How do you evaluate the robustness of an Artificial Intelligence system against security attacks? This question is at the heart of an SnT paper that was recently awarded as spotlight paper at the prestigious NeurIPS conference.
More than 15,000 papers were submitted for this event, and 25% have been accepted for publications. Only 3% have been selected as a spotlight.
“This is the most prestigious publication we are rewarded, after more than five years of research investment in the field of AI trustworthiness”, says a delighted Maxime Cordy, Research Scientist and co-author of the article with Thibault Simonetto and Salah Ghamizi.
Keeping up with AI technology
For years, SnT has been working on developing various aspects of AI technology and its application to areas of economic importance such as Fintech, Cybersecurity, Space Systems and Autonomous Systems.
An important element of this work is to make AI technology more trustworthy. From ensuring that Ai tools are unbiased, robust and explainable, to creating solutions that protect AI systems from security attacks.
As an increasing number of sectors use AI systems in their processes, it’s important these are secure. Take an AI system that has the objective of monitoring transactions to detect money laundering, for example. A security attack could mean that this fraudulent transaction would go unnoticed. This is called an evasion attack, where “noise” is created in the data so that AI can’t do its job properly.
“The difficulty is to make these simulated attacks effective and realistic. Our research takes an important step in this direction and proposes a new approach – Constrained Adaptive Attack – which sets new standards for assessing the robustness of tabular AI”, explains Maxime Cordy. Tabular AI systems deal with data presented in tables (like in database and excel sheets) and are therefore omnipresent in the industry.
From a project with BGL BNP Paribas to scientific recognition
The paper represents the pinnacle of Thibault Simonetto’s PhD journey at SnT, and also follows the partnership announced in January 2024 with BGL BNP Paribas. The interdisciplinary research centre supports the bank in developing an intelligent monitoring system for the AI solutions implemented in its services.
“This paper is an important contribution to the realization of our research program and the proposed approach is applicable to the banking AI systems that operate on industrial scale data. This success emphasizes the benefit of a partnership between Industry and academia” comments Anne Goujon, Head of Data Science Laboratory at BGL BNP Paribas.
The Serval research group, in which Maxime Cordy works, is experiencing a particularly dynamic period. Another of his papers has been accepted as a “benchmark and dataset” by NeurIPS. This contribution develops a suite of benchmarks for evaluating the ability of defense algorithms to protect tabular AI systems against attacks.
Thanks to these two distinctions at the NeurIPS conference, SnT receives a publication at one of the leading AI conferences where it had not previously been published. “In general, AI trustworthiness is a huge topic in Europe right now. This spotlight, together with our publications, demonstrates our legitimacy, as experts, to work on this topic and support society in the AI transition”, adds Maxime Cordy.
Two other SnT papers have been selected for the NeurIPS conference. These are publications by the CVI2 research group, produced in partnership with Artec3D and POST respectively.
“In total, four papers have been accepted this year, which is exceptional and positions SnT as contributor to the AI field at a top-level”, stresses Yves Le Traon, Director of the SnT.
“These achievements show that getting research and industry working together can lead to high-quality scientific results, contrary to an old prejudice. This requires trust, mutual understanding and a willingness to cooperate in the long term between research and industry”, adds Carlo Duprel, Head of SnT Technology Transfer Office.
This article was first published on 20 November by University of Luxembourg.
Receive the Funding Newswire each Tuesday, our Policy Bulletin each Thursday, and news about bridging Europe’s east-west innovation gap twice a month in The Widening. 
A unique international forum for public research organisations and companies to connect their external engagement with strategic interests around their R&D system.
