Improve evaluation and transparency in Horizon 2020, urge MEPs

The process for evaluating Horizon 2020 proposals needs more rigour and first-time applicants for grants from the research programme should get better information on the odds of success.

These were among the main recommendations put forward in a  report by the European Parliament’s Committee on Industry, Research and Energy (ITRE) which looks at the performance to date of a selection of Horizon 2020’s main funding schemes, investigating complaints from researchers about the consistency in evaluating applications to the SME Instrument and Fast Track to Innovation.

There is a general plea for the EU Commission to improve on the detail it provides on the various websites uses to promote Horizon 2020 funding, with the report saying for example that there is not enough information for young companies on how to take part in competitions run by the European Institute of Innovation and Technology.

“Horizon 2020 is a big step forward compared to the last research programme,” said Belgian MEP Philippe De Backer. “Nevertheless I still see rules that are too complex for small businesses, gaps in the quality of feedback from evaluators, and proposal re-submissions which further depress the low success rate,” he told Science|Business.

Danish MEP Jeppe Kofod wants researchers and companies to be given a stronger sense of what they are getting themselves into when they apply for Horizon 2020 grants. With success rates as low as 12 per cent, the €77 billion research programme is one of the most competitive in the world.

The Commission should ensure applicants are given clearer indications of their likelihood of succeeding. This would, “minimise the effort wasted on long application procedures that fail to [translate] into funding and actual research,” Kofod said.

Make SME Instrument more accessible

Improvements are needed to the SME Instrument, which funds companies with close to the market products. The low success rates for applicants to the scheme, introduced in 2014, could become a huge issue, said German MEP Hans-Olaf Henkel.

“With odds of success as low as 6 per cent we [are] creat[ing] huge potential unsatisfied, disenchanted - and finally EU-critical audiences,” he said.

MEPs say it should be standard practice to interview applicants for SME Instrument funding and statistics on success rates should be more prominent. The figures on the Commission’s website lack detail, which MEPs say, “might raise a question about full transparency”.

In addition, the level of communication with grant managers is unsatisfactory, according to Horizon 2020 national contact points, a network which gives guidance to applicants. Officials from the Executive Agency for Small and Medium-sized Enterprises were contacted for comment but did not reply before publication.

The report cites meagre feedback on failed proposals as a weakness – a complaint commonly heard from researchers about all Horizon 2020 competitions. The Commission’s argument for shorter appraisals is that it helps trim bureaucracy.

Henkel also complained about a lack of flexibility within the SME Instrument which makes it impossible to move money from unpopular to oversubscribed topics.

There is also a call for EU joint technology initiatives to make themselves more visible. "It is positive that the report calls for more openness, transparency and effectiveness in [these partnerships] as presently these are still closed clubs," said Kaja Kallas, an Estonian MEP. 

Spreading excellence?

The question of whether Horizon 2020 grant funding is spreading excellence is questionable, the report said. Winning EU research contracts, for the most part, remains the preserve of big institutions in western European countries.

To balance this out, the Commission encourages researchers from Eastern and Southern Europe to apply for research money from initiatives such as “European Research Area Chairs”, “twinning” and “teaming”.

However, MEPs say this dedicated budget does not seem to be large enough to have a real impact and the success rate is very low.

The recipient of a €2.6 million grant under an ERA Chairs competition, Nuno Nunes, associate professor in informatics engineering with Madeira Interactive Technologies Institute, agrees with this assessment. “All of these grants which are for institutional development require a huge amount of effort that hardly compensates in terms of the career of researchers,” he said.

Similarly, MEPs lamented a failure to widen participation in European Research Council (ERC) competitions. Again, rich nations lead the way. “Perversely, ERC grants are increasing inequality among researchers in the [EU] by only providing generous funding to a small percentage of researchers,” the report claims.

They also say that at €150,000 to €2.5 million, ERC grants are too big. “For some stakeholders, the grants are so generous that the few ERC grant holders might abandon ‘real’ research and become mini-funding managers.” 

Co-financing options remain vague

For some, ITRE’s investigations do not go far enough. For example, there is a lack of information on how easy or difficult it is for researchers to co-finance their research or business idea using both Horizon 2020 and EU structural fund money.

Nunes said that in practice co-financing is very frustrating and difficult for researchers and businesses. ”We had several proposals very well evaluated under Horizon 2020, but trying to get them funded with regional structural funds is a different challenge.”

“At the regional and interregional levels there is simply no culture of excellence – on the contrary, centres that are successful in attracting funds are considered a threat and this limits the capability of combining money pots.”

The report says it could not gather enough evidence to judge the success of the Commission’s “seal of excellence”, a certificate scheme introduced by Research Commissioner Carlos Moedas for applicants that are evaluated as excellent but cannot obtain financing from Horizon 2020. In theory, the seal could persuade regional fund managers to invest.

“It is not clear how [the scheme] will be implemented by the member states and their regions. It is unlikely that the regions will be able to award the same amount of grants as [Horizon 2020],” the report says.