Academics in central Europe debate the fairness of European Research Council evaluations

Central European research leaders have expressed scepticism about efforts to increase success rates in winning European Research Council (ERC) grants in member states with underfunded research systems that joined the EU after 2004.

At a meeting in Budapest last week organised by the science academies in Czechia, Hungary, Slovakia and Poland, academics in central Europe and representatives of the ERC debated strategies for boosting the number of EU grants that go to researchers in countries with less developed science systems. A recording of the meeting was made available this week.

As academics at the meeting complained, in Czechia, Hungary, Slovakia and Poland, ERC success rates are significantly lower than in the Netherlands, Germany and France.

According to data presented by Philippe Cupers, head of the life science unit at the ERC executive agency, between 2007 and 2021, 15% - 16% of applicants in the Netherlands, Germany and France got an ERC grant, but less than 8% of applicants in the central and eastern European countries succeeded in ERC grant competitions.  

Some non-EU countries did even better than the top EU countries, as is the case for Switzerland and Israel, which had success rates of 21% and 18% respectively.

“We have really put in a lot of effort to increase the participation of our countries and let's be frank, we did not succeed,” said Jerzy Duszyński, the president of the Polish Academy of Sciences. “The numbers are very, very small,” he said.

Because of the low numbers, researchers begin to doubt their chances of success in ERC competitions and decide to not apply. István Szabó, vice president for science and international affairs at Hungary’s research, development and innovation office NKFIH said researchers “are a bit reluctant” to apply. “We should somehow make some joint efforts […] to convince them that they really have actual opportunities and then they can actually have ERC grants.”

Duszyński suggested national agencies and academies in central Europe could organise search committees to recruit the scientist with the best chances of winning ERC grants. “We can find candidates that we will encourage to enter ERC competitions,” he said.

The issue has long been a sore point and is not limited to the ERC. According to data from the European Commission, less than 6% of the EU budget for the 2014-2020 research programme Horizon 2020 went to the so-called widening countries, the member states with the lowest research and innovation performance in the EU.

The Commission has been trying to convince member states to increase public and private investments in R&D. Governments have recently renewed a previous commitment to raise R&D public and private expenditure to 3% of the national GDP, but very few countries have passed that threshold so far. Some countries in the widening group are making progress, but others are struggling with budgets that account for a lot less than 1% of GDP.

Cupers said the countries that are most successful in getting ERC grants are those that got close or surpassed the 3% target. The ERC also notes that research institutions in countries with most successful ERC applicants spend more money and effort on administrative support for promising researchers. “We see that the support of the host institutions and the research organisation is something critical,” said Cupers.

Evaluation criteria

According to data presented by Cupers, most applicants from the widening countries never make it through to the final evaluation round and are rejected in the first step of the evaluation. In the first step, a panel looks at an extended synopsis of the proposed project and the investigator’s track record. “This is where some efforts have to be made to convince this panel of generalists,” Cupers said.

Tamás Freund, president of the Hungarian Academy of Sciences said the first step does not favour researchers from the widening countries because it focuses too much on the publication and citation record of the applicants. “Due to our special situation, due to the decades and decades of political problems and the poor development of our industry, we have very, very few scientists who have that many publications which would allow them to pass,” he said.

Freund went on to say that researchers are reluctant to apply for ERC grants because they feel they do not have “enough” papers in prestigious publications like Nature and Science.

Cupers said the ERC strives to be fair towards applicants regardless of their background and the first step of the evaluation focuses on the quality of the proposal. “This is what we are really asking our panel members to do, they have to focus on the quality of science,” he said.

Representatives from other countries suggested the ERC could use blind evaluation in the first step of the project assessment. Cupers said a similar process had been implemented under the Commission’s fifth research framework programme in the late 1990s, but the system did not have the intended impact. “As soon as there was a little clue on who could be the person behind, [the project] was of course deemed ineligible.”

Eva Kondorosi, a Hungarian biochemist who is a member of the ERC scientific council said at one point her peers debated the possibility of introducing parallel panels, one that would evaluate both the CV of the applicant and the proposal, while a second panel gets to look only at the proposal. However, the plan was never implemented “for some administrative reasons.”

Better framework conditions

But not everyone in the room agreed on what should be done next. Research institutions in central and eastern Europe are often split in two camps: those who want to emulate successful counterparts in western Europe, and those who would like the rules of the game to change.

Pavol Šajgalík, the president of the Slovak Academy of Science said thinner CVs are not to blame for low success rates in the ERC. “I think that our problem is to express in a very short [space] what is the breakthrough of your research.”

According to Kondorosi, researchers should ask their peers to review the proposal before submitting it to the ERC, as each application should be criticised “very thoroughly” by other scientists. Kondorosi often found some applications “either too specific, or too boring or not ambitious enough.”

Kondorosi said widening countries can start implementing some changes to improve their standings in ERC competitions. Researchers should spend more time brainstorming to come up with original ideas and to learn how to present them. In addition, research institutions in low-performing countries should also encourage young scientists to apply and offer them more support. “The other problem is that I think our countries are really not attractive to foreigners,” said Kondorosi.

Katalin Solymosi of the Young Academy of Europe said early career researchers should be encouraged to work independently, travel more often in exchange programmes and learn to communicate their work in English. Early career researchers “do not necessarily have the experience to write their projects correctly.”