EU push for research assessment reform poised to enter a new phase

The European push to reform how the value of research is assessed is about to enter a new phase, with the action plan that was finalised in July opening up for organisations to endorse it next week.

For research stakeholders who have been working on the process since the beginning of the year, this marks the start of the real business, when organisations that carry out research assessments pick up the baton and start work on reform.

For Mattias Björnmalm, deputy secretary general of the CESAER university association, one of the stakeholder associations steering the process, this is the end of the conceptual stage of the initiative. “I think now it’s really important that we move to the phase where those that do research assessment themselves are empowered take ownership,” he told Science|Business.

The European Commission got the ball rolling on the reform in the beginning of this year as part of moves to revive the European Research Area (ERA). The aim is to revamp the way research is assessed, moving from bald statistics, such as the number of scientific papers published and how many times they are cited, to measures that more accurately reflect the value and impact of research.

 In June the process was endorsed by EU science ministers, leading on to the action plan setting out the main lines across which the evolution of assessment systems should take place. Now, it’s up to funding agencies, researchers, universities and organisations to sign the agreement and start the hard work.

They can start signing up next week and have until 15 November to do so if they want to take part in the first big meeting of the coalition on 1 December.

“It’s been sold a little like the process is finished and co-creation is done, and we managed to get a framework that everyone is comfortable with. But the key issue now is to see how many institutions will become part of this initiative once it really starts,” said Silvia Gómez Recio, secretary general of the Young European Research Universities (YERUN).

Setting the scene

Some organisations have signalled willingness to sign, while others are hesitant to jump on board early on. The good news, says Stephane Berghmans, director for R&I at the European University Association, is that universities which have indicated they will sign represent a diverse set of institutions and are based all around Europe.

“Given the time the process can take for big decisions [at universities], it’s already a very good sign,” said Berghmans. “I think the coalition is going to be the place where a lot of questions will be addressed, if not answered straight away. I do think it’s better to be in than out. But it’s down to each university to decide.”

Lidia Borrell-Damian, secretary general of Science Europe, expects a multitude of different organisations to sign up, although exact number is anyone’s guess. Once on board, the different members will have to decide how to work together. “We will have to define a system in which each actor can take up the role that is natural for them,” Borrell-Damian told Science|Business.

The goal is to get the signatories to commit more than a signature and to actively engage in the process. “What is important is that those that will commit will have the resources – human and monetary – and the dedication to coordinate inside the institution, inside academic circles, to go deeper into practicalities of implementation, and how it will affect researchers and research more generally,” said Borrell-Damian.

Meanwhile, the drafting team, now renamed the interim secretariat, is finalising the draft documents proposing structures for governance, rules of procedures, and code of conduct, which will be presented to signatories at the meeting in December for their approval.

Björnmalm believes the key to making progress lies in not settling on a top-heavy governance structure dictating the flow of the coalition, which could impede free exchange of ideas. “It’s a very evolutionary, learning-based process, and to be this implies that we should let the structure be lean and flexible,” said Björnmalm. “Bureaucracy and rigidity can be detrimental to the learning approach.”

Evolution, not revolution

The next, and final stakeholder assembly of the current stage of the initiative, takes place on 13 October. Gómez Recio sees this as an opportunity to ask representatives what would help them to engage with the process. “We can propose many things and have many ideas, but in the end, it’s really up to them to decide what shape they want it to have, how they would feel more encouraged, included, able to be part of it, more willing to contribute,” she told Science|Business.

There is no single, perfect system. Rather it’s about having a conversation and respecting the diversity and multitude of research outcomes and their impact. “It’s about all of us understanding a different way of assessing each other and understanding the contributions researchers make to different areas of society,” said Gómez Recio.

For Borrell-Damian, the process is about bringing openness to alternatives to the research world. Research outputs are diverse, but not all assessment systems recognise them. And many pay too little attention to the contribution of science to society when it comes to assessing the work of researchers. “We want to change this paradigm,” said Borrell-Damian. “I do hope this will install some openness in the research sector.”

Whether formalised or not, change is in the air, and the process is providing the forum for exchange on the evolution of research assessment. “In a lot of places, there is already an atmosphere and feeling that change is happening. When I talk to members, leaders of universities, students, change is happening already,” said Björnmalm. “This idea with the coalition is to help capture the momentum and take it to the next level.”

Much will depend on who signs up to it. “If people don’t take [this agreement] and do something with it, it’s nothing,” said Gómez Recio. “The most important thing is to have an engaged and active community of signatories.”