The Ecosystem: Artificial Intelligence Act restrictions mean make or break for Europe’s emotion AI start-ups

The European Parliament has introduced a series of restrictions into the draft EU Artificial Intelligence (AI) Act that would prevent AI-driven facial recognition technology being used in public spaces for surveillance or law enforcement purposes. But it has also set out a broader ban on AI systems that detect human emotions, not just for surveillance but also in schools and colleges, and in the workplace.

This move has been welcomed by some start-ups that are applying AI methods to the detection and interpretation of human emotions, but caused consternation for others. “Essentially, instead of receiving support for our development, we face the threat of extinction,” said Stefano Bargagni, founder and chief executive of MorphCast, an Italian start-up developing an AI system for digital video communication.

MorphCast’s technology can detect and analyse the emotional responses of viewers in real-time, allowing for videos that are intended to interact with viewers' emotions and contain customised empathic content. The system will also measure and certify how much attention viewers pay to entertainment, advertising, and educational content.

“We are witnessing numerous halts to projects that we had painstakingly initiated with major clients,” Bargagni said. “These clients fear the forthcoming regulations as potential risks to their brand reputation.”

This has affected MorphCast’s funding. “We are experiencing difficulties raising capital, as our company may potentially belong to those that will soon be substantially restricted in their development, or may even see their business operations eradicated,” said Bargagni.

Other start-ups have developed emotion AI applications for use in highly controlled settings. As a result, they find the limitations introduced by the Parliament much less threatening.

Polish start-up UX Mining is developing tools for product useability testing in which volunteers are recorded using a new product or service and a transcript analysed via AI to assess their feelings. At the same time, the company is working on a more sophisticated system that will monitor voice, facial expressions and body movements.

This will tag an interview video when particular emotions are detected, such as discomfort completing a task, so that human analysts can pick up key moments in the test without spending hours watching recordings.

“We selected this application precisely because it takes place in a controlled environment, and consent is quite clear,” said Jean-luc Momprivé, co-founder of the company. Care has been taken to build in strict controls over how the data is used and who can access it. The role of the system is also important. “We are using AI as a helper and not as a decision-maker,” Momprivé said.

The same tight focus and concern about data protection are apparent at Beyond Emotion, a spin-off from the Hamburg University of Applied Sciences. It is developing an AI system that tracks the mood of elderly people, alerting relatives to strong negative emotions or prolonged inactivity. The system is built into a picture frame, which displays images for the subject to look at, such as family photos. At the same time, it monitors their facial expression.

“The software carrying out the analysis is on the device, so the video stream is not sent to the cloud or anywhere else,” said Hanne Butting, co-founder of the company. “So, this is data protection through edge computing.”

The product has been tested with users and presales to consumers will begin at the end of June. “In two months, we want to ship the product,” Butting said.

Strict regulations support emotion AI

Both UX Mining and Beyond Emotion see the limits on emotion AI in the AI Act as useful red lines that can help build confidence in the use of this technology.

“They limit the people who want to use information gathered in the street, in a shop, or in an interview without asking explicitly for consent,” said Momprivé. “I think that is a good thing, and there are still a lot of applications where this technology can be very helpful in assessing people’s emotions.”

Butting agrees. “This is such a sensitive area, and it affects so many people if AI is used in public spaces,” she said. Rather than hinder further development of emotion AI, the Act may be a stimulus. “It shows that it is possible to innovate in this area, but still think about the people.”

A strong regulatory environment will also support companies that take their data protection responsibilities seriously. “Elderly people, especially here in Germany, always ask how their data will be handled,” said Joscha Lordieck, Beyond Emotion’s marketing and communications manager. “We have an answer to that with our product, but the regulation will also help.”

Bargagni sees the value of regulation, but argues that the EU is going about it the wrong way. “While I support a regulatory framework that mitigates risks and prevents the discriminatory use of technology, it should be formulated in concert with other major countries where global market players are, to avoid disadvantaging our own companies,” he said.

Given the global scope of AI and its relevance to users worldwide, an international approach would be much more appropriate. “However, it seems to me that Europe often rushes to be the first to impose obstacles, regardless of the circumstances,” Bargagni said.

Instead, the EU should follow the example of the US or China and foster a conducive environment for its cutting-edge technology companies. “Simultaneously, unnecessary bureaucracy and poorly designed, detached regulations should be avoided,” said Bargagni

Egregious uses

For Andrew McStay, director of the Emotional AI Lab at Bangor University in Wales, the AI Act could have a positive impact on the development of the sector, while still leaving it room to manoeuvre.

“I think the law has signalled an end to some egregious uses, but we are still entering a world where technologies will be used to profile and interact with us in more intimate ways,” he said. “I do hope legacy tech and the start-up community decides to innovate in ways that work with and for people, rather than against them.”

The restrictions imposed by the European Parliament leave untouched a lot of potentially interesting applications for emotion AI. “It has potential to be applied in any situation where understanding of emotion is useful,” McStay said. “Seen through this prism, law enforcement and border management, and to a lesser extent the workplace and education, are actually quite odd examples.”

More fruitful applications might be found in communication, art, entertainment, well-being, and mental health. Whether emotional AI will be able to grow in these areas is another question.

“As is well known, current iterations of these technologies are marred by accuracy questions and issues,” McStay said. “That said, my hunch is that we are at the beginning rather than the end of something.”

AI Act will have a positive effect if it moves the industry away from public space applications. “While one might speak of limiting the scope of developing emotional AI systems, it is perhaps better to think of the situation as one where the industry is nudged in more pro-social directions,” Mc Stay said.

That will require more innovation rather than less. “The task for start-ups in this area is to avoid lazy reliance on labelling techniques that are massively over-simplistic,” he said.

At its most basic, emotion AI relies on the idea that internal states can be inferred from outward appearances. “There is good science showing that current industry approaches are only slightly better than chance in terms of accuracy,” McStay said. “If correct, the opportunity is to move away from passive profiling, to one with active buy-in and input from people. This is a very different approach, but I suspect it is one that could win out.”

For companies such as Beyond Emotion, there is no lack of interest from the market. It often receives requests to develop systems for other uses, including in education and e-learning, for example to support personal training in presenting. But it is cautious about what it considers.

“The main thing for us is that the person being analysed knows about it, and must agree as well as be the beneficial owner of the analysis,” said Butting.

One offer the company turned down involved a call centre that wanted to monitor its employees and optimise their concentration levels. “We had concerns about guaranteeing its responsible use, and so we said no,” said Butting.

Elsewhere in the Ecosystem…

• The EU Prize for Women Innovators is shrinking, with smaller awards and less money over all in the 2023-24 competition. Previously, the European Innovation Council gave €100,000 each to three women innovators and €50,000 each to three rising innovators. Now, only one woman in each category will get this amount, with the runners up taking home smaller prizes. Even with a new set of awards for women leaders with connections to the European Institute of Innovation and Technology (EIT), the total prize money still falls short of the 2022 round by €30,000. The deadline for applications is 27 September. The winners will be announced at the EIC Summit in March 2024.
• Six additional countries have joined the Europe Startup Nations Alliance, an organisation set up to help signatories to the EU Startup Nations Standard make progress towards its implementation. The standard sets out eight areas of policy to support the creation and growth of innovative start-ups, including supportive rules on stock options and access to finance. The expansion brings the alliance’s membership to 15 countries, just over half of the 27 signatories. The new members are Luxinnovation for Luxemburg; Germany Trade & Invest; Lithuania’s Innovation Agency; the Startup Poland Foundation; the Research and Innovation Foundation in Cyprus; and Elevate Greece.
• Nia Health, a spin-out from the Charité university hospital in Berlin, has raised €3.5 million in seed funding. The company is developing digital therapeutics for people with chronic skin conditions, plus systems for monitoring patients and conducting clinical trials. The money will be used to expand market share in German-speaking countries, launch in other EU territories, and prepare for the US market. The lead investor is High-Tech Gründerfonds.
• Voima Ventures has raised €90 million for a new fund that will support Nordic and Baltic early-stage deep-tech companies to scale-up. Its focus will include climate tech and other contributions to the green transformation, life science and health technology, and quantum. Initial investments will range from €200,00 to €3 million.