German universities still wary of EU push to reform research assessment

The European Commission is making a big push to reform research assessment, but Germany’s university leaders are not convinced the call for change from above is the right way to deliver it.

While the German Rector’s Conference sees value in rethinking how research is assessed in Europe, it believes the debate is not independent enough. “The agreement itself was designed and conceptualised along the policy lines of the EU Commission and, at its core, is basically the result of a top-down process,” says Walter Rosenthal, the Rectors’ Conference vice president for research, academic career paths and transfer.

The reform process was kicked off by the European Commission in the beginning of 2022, as part of moves to create a single European Research Area (ERA).

With research becoming more impact-driven, multidisciplinary and open, the aim is to align research assessment with these changes, moving away from metrics such as the number of papers published and the number of times these are cited, to a broader set of achievements reflecting the wider impact of research.

The initiative has both faced a backlash and received strong support in different parts of the EU research ecosystem, but last month moved on to the next stage with the establishment of the Coalition for Advancing Research Assessment (CoARA), to carry on the discussion and work on reforms.

The German Rectors’ Conference has decided, at least for now, not to join coalition, saying it is too political a mechanism. This is at odds with other university groups, with the European University Association (EUA) spearheading the agreement, and over 400 research organisations of different sizes signing up.

Lidia Borrell-Damian, secretary of Science Europe, one of the associations at the forefront of the initiative, says CoARA will be a place to share knowledge and streamline efforts, rather than dictate what changes institutions should implement. “What we offer is a platform of reference,” she says. “It’s a framework for discussion, sharing and learning.”

CoARA is fully independent but Borrell-Damian says the Commission’s steer at the beginning was essential. “To expect that the academic community would have come together on this fully bottom up is too hopeful,” she told Science|Business.

The holistic view

The politics is one issue, but the German Rectors also find fault with the ideology behind the agreement.

While the reform is framed as shifting the focus from quantitative measures of research to a more holistic view of its value in terms of societal impact and beyond, Rosenthal argues the agreement involves a “much broader and less aspirational concept of quality” that is at odds with using the tried and tested principle of excellence to benchmark research.

The agreement, “tends to confuse criteria with prerequisites,” Rosenthal said. “It is undeniable that diversity and cooperation are necessary prerequisites for research excellence. Still, both as such are no criteria for evaluating the scientific substance of a research proposal.”

The concern with such an approach, he notes, is that the initiative goes beyond research assessment reform and affects the way in which funding is distributed in Europe. “This would be a mere political result, not a scientific one, and [the German Rectors’ Conference] therefore can’t support it,” said Rosenthal.

He also fears a more relaxed approach of ‘striving for quality’ could endanger the European Research Council (ERC) focus on funding excellent research.

The ERC hopped on the research assessment train last month, announcing it will be adjusting its evaluation process. From 2024, ERC evaluators will give more weight to project proposals per se, rather than the past achievements of each applicant, reviewing short narrative descriptions of their careers rather than the current CV and track record templates.

That is intended to allow researchers to highlight unconventional career paths and contributions to the research community, rather than how much extramural funding they have received and PhDs they have supervised.

But instead of big changes to assessment systems, German rectors want to see a more diverse and inclusive science system promoted, which would provide applicants with the means to compete successfully, but with excellence as the key criterion.

They also support bottom-up initiatives, such as the Declaration on Research Assessment (DORA), an international grass roots movement that looks to improve evaluation.

Stephane Berghmans, director of research and innovation at the EUA, says moves to reform research assessment are taking place in institutions and countries around the world. CoARA, as a structured coalition, is providing much-needed impetus in Europe. “This is nothing new to most researchers and universities, but today we have a momentum we’ve never had,” he told Science|Business.

Yet, not every institution is ready to join the effort, or knows enough about the objectives to be comfortable joining. In some cases, this is because the institutions themselves are rather advanced in their own reform initiatives.

“One thing I usually say to people who ask me, is that it’s better in than out. Being in, you can influence the process, be aware of what’s going on, and shape it,” said Berghmans.