Fraunhofer launches internal AI chatbot

The Fraunhofer-Gesellschaft, in collaboration with Microsoft, is one of the first research organizations in Europe to launch an internal AI chatbot that works similarly to OpenAI’s ChatGPT. The FhGenie generative AI solution lets employees securely create, edit or modify texts on the basis of non-public data. The GDPR-compliant service was introduced for Fraunhofer employees in line with guidelines drawn up by the Chatbot taskforce at the Fraunhofer ICT Group.

he disruptive potential of generative AI systems became evident at the latest with the release of ChatGPT. The expected economic and societal impact is enormous. Fast and secure access to existing systems is therefore essential for large research organizations to be able to gain comprehensive first-hand experience, drive technological developments and maintain innovative strength.

The Fraunhofer-Gesellschaft has now released the AI chatbot FhGenie for its employees. It is based on Microsoft Azure OpenAI Service, which enables organizations to use OpenAI’s latest generative AI technologies such as GPT-3.5 or GPT-4, Codex, ChatGPT and DALL-E 2 from the Microsoft Azure Cloud.

“I am very pleased that, together with Microsoft, we have succeeded in developing an AI chat system tailored to the requirements of the Fraunhofer-Gesellschaft — and we did so in a very short time,” says Prof. Ingo Weber, director of Digital Transformation and ICT Infrastructure at the Fraunhofer-Gesellschaft. “We have found that many colleagues wish to use chat-based AI applications for their work and for research. However, public solutions available so far are problematic for work-related purposes, especially when it comes to data protection, confidentiality and information security.”

FhGenie provides Fraunhofer employees in research, management and administration the chance to gain experience with these systems: “Microsoft Azure OpenAI Service on which FhGenie is based enables Fraunhofer employees to harness the potential of generative AI systems for their own work. Against the backdrop of rapid technological developments, it is important to become familiar with the solutions already available and to take advantage of their benefits,” says Alexander Britz, Public Sector Lead at Microsoft Germany.

GDPR-compliant internal chatbot

The generative text AI solution FhGenie is based on the OpenAI language model GPT-3.5-turbo and will be upgraded as quickly as possible with the more powerful and multimodal GPT-4.0, which can also process images. Currently, FhGenie comprises a web application with a Python-based framework and the JavaScript runtime environment node.js. Employees can easily access it in their browser with SSO authentication using their smartcard. To ensure data security, the instance was configured in a European, Fraunhofer-owned Azure subscription, a closed section within Microsoft's cloud platform.

Requests via Azure OpenAI Service, which is also a dedicated part of the Fraunhofer subscription, use an API prompt. This is a customized interface that passes commands to the AI system. Users receive their results from a completion endpoint (autocomplete functionality). All data entered remains within the Fraunhofer-Gesellschaft and is not used to train the AI. Apart from an automated filter program that checks for improper use, content entered is not monitored or saved.

The works council has approved the use of the GDPR-compliant Azure OpenAI Service application throughout the Fraunhofer-Gesellschaft, access authorizations are assigned locally by the IT managers at the individual Fraunhofer institutes and entities. Current usage figures show that the new tool is well received by Fraunhofer’s workforce: As of today, 19,200 of Fraunhofer's more than 30,000 employees have already registered for the service, of these about 6,500 can be considered active users.

From reporting to knowledge management: a wide range of possible applications

There are numerous possible applications for the AI chatbot, the full range of which will probably only become clear in the future. FhGenie already offers Fraunhofer employees not only support in creating and improving texts and presentations, such as for internal reports, but can also help develop software, prototypes or data analysis scripts faster. There are also potential use cases for further developing internal knowledge management systems, such as in the form of an interoperable intranet. Researchers at the Fraunhofer institutes can use the experimental space to quickly expand their expertise and test AI models together with customers.

How they use the solution, however, is up to the users themselves. Personal data is the only thing that may not be processed with FhGenie — for legal reasons. “We are currently in a transitional and experimental phase in which it was important for us to quickly offer a practicable solution to be among those at the forefront of the rapid technological developments currently taking place,” explains Prof. Weber. "Of course, there are still many uncertainties, such as copyright or liability for incorrect results. We therefore asked those responsible at the institutes and departments to make sure that all users are aware that they are responsible for and must verify the quality of AI-generated results themselves.”

AI language models and Fraunhofer: access, adapt, advance

Numerous Fraunhofer institutes are already actively involved in initiatives and projects for the research and application of generative AI and foundation models. One example is the OpenGPT-X consortium created for the development of large European AI language models. This is another reason why a task force from the Fraunhofer ICT Group drew up guidelines for the Fraunhofer-Gesellschaft on how to deal with AI language models when ChatGPT was launched. The release of FhGenie is part of the first of three steps (access, adapt and advance) required for large-scale AI models. At the same time, Fraunhofer aims at winning over more cooperation partners from industry.

“Large language models and generative AI make new products possible and our world of work more productive. As an applied research organization with an interdisciplinary approach and strong links with the industrial sector, the Fraunhofer-Gesellschaft is in an excellent position to put these innovations effectively into practice. Similar technology is already in daily use at companies, for example, in the financial and health care sectors, resulting in efficiency gains of up to 50%. FhGenie helps us ensure that everyone at Fraunhofer can immediately and securely leverage this potential and tap into it for our customers and partners,” says Prof. Stefan Wrobel, deputy chair of the Fraunhofer ICT Group and director of the Fraunhofer Institute for Intelligent Analysis and Information Systems IAIS.

This article was first published on 30 August by Fraunhofer.