Brussels   |     |   By invitation only

Expected outcomes: What should research assessment reform aim to deliver and how to get there?

A Science|Business hybrid expert workshop, organised in partnership with Elsevier (10:00 – 12:45 CET)

Assessment is one of the most significant processes in the global research arena, encompassing everything from policy goals and programme budgets to project selection, awards and career development. Moreover, its profile has never been higher, both in Europe and beyond. Since the 2008 financial crash, economic constraints on public funding have multiplied, leading to increased political demands for the scientific community to measure impact and demonstrate its value to society.

Yet in the past few years, a widespread belief has grown that the current system needs to be rethought – not least in an age of increasing digitalisation, open science, big data, and the demand for cross-disciplinary methods and skills, to name but a few.  Among the consensus issues that require attention: improving the use of metrics, balancing quantitative and qualitative factors, and broadening the scope of assessment to reflect the full diversity of inputs, outputs and practices in 21st century science.   

It has thus become clear that new indicators and approaches are needed to drive the emergence of a fairer, more nuanced system.  Against this backdrop, the European Commission has run a consultation and “co-creation” exercise with R&I stakeholders to define principles and boundary conditions for a new, enhanced assessment system. The resulting Agreement – published in July – provides a pathway for change, to be backed by a “coalition of the willing” who commit to run a full cycle of assessment reform and review by the end of 2027. European University Alliances have been touted as one potential environment within which to get the ball rolling. Importantly, however, the text proposes to cover all research funding and performing organisations, assessment authorities and prize-giving bodies – in other words, not just the world of universities.

Inevitably, the key question now becomes: what’s next in terms of implementation? Can the shift from rhetoric to action be driven by genuine innovation and experimentation, towards an agreed set of defined and pragmatic outcomes? To encourage success, how can the way forward reach beyond academia to involve the full R&I community? And while Europe embarks on its own transformation, what can it learn from other parts of the world that are undertaking similar processes, with a view to future international cooperation and alignment?    

On October 25, Science|Business – with the support of Elsevier – will convene a selected group of senior figures and experts from across the European and international R&I spectrum. Drawing in part on the latest policy developments around the Agreement on reforming research assessment, the ERA Roadmap, the European Strategy for Universities and more, the meeting will seek answers to the above questions, and identify key considerations for policy makers as consensus moves into implementation.

Silvia Gómez Recio
Secretary General, Young European Research Universities Network
David Sweeney
Executive Director, Research England
James Wilsdon
Vice Chair, International Network for Government Scientific Advice
Ludovic Thilly
Chair of the Executive Board, Coimbra Group
Thom Palstra
Full Professor & former Rector Magnificus, University of Twente
Mattias Björnmalm
Deputy Secretary, Conference of European Schools of Advanced Engineering Research and Education
Amane Koizumi
Research Director, Japan National Institutes for Natural Sciences
Jean-Claude Burgelman
Professor & Academic Coordinator, Vrije Universiteit Brussels
Lidia Borrell-Damián
Secretary General, Science Europe
Michael Arentoft
Head of Unit, Open Science, DG Research and Innovation, European Commission
Nick Fowler
Chief Academic Officer & Managing Director, Research Networks, Elsevier
Ricardo Migueis
Head of Office, INESC Brussels Hub
Giulia Malaguarnera
Executive Committee Secretary, Marie Curie Alumni Association
Oleksandr Berezko
President, European Council of Doctoral Candidates and Young Researchers
Gunnar Sivertsen
Researcher & Senior Advisor, Nordic Institute for Studies in Innovation, Research and Education
Torsten Fischer
Director, European Liaison Office of German Research Organisations
Marjolijn Brussaard
Chairperson, UAS4EUROPE
John Byron
Principal Policy Adviser, Queensland University of Technology
Stephen Curry
Chair, Steering Committee, DORA & Professor, Imperial College London
Monica Dietl
Executive Coordinator, Initiative for Science
Verena Fennemann
Head of EU Office, Fraunhofer
Vinciane Gaillard
Deputy Director, Research and Innovation, European University Association
Sandra Hesping
Advisor, International Research, Eindhoven University of Technology
Sarah Howard
Branch Manager, Research Excellence, Australian Research Council
Bianca Langhout-van der Bulk
Project Manager "Recognition & Rewards", Erasmus University Rotterdam
Jordi Molas Gallart
Research Professor, CSIC & Director, INGENIO, CSIC-UPV
Katia Pistrin Rosell
Manager, Personnel Grants, Barcelona Supercomputing Center
Andrew Plume
President, ICSR and Vice President, Research Evaluation, Elsevier
Karen Stroobants
Policy Adviser, Royal Society of Chemistry
Hirotaka Yamada
Director Paris Office, Japan Science and Technology Agency

10:0Welcome and opening remarks

10:05  Science experiments: Is the research world ready for change?

An opening discussion regarding the state-of-play in research assessment reform – both in Europe and beyond – as multiple countries move forward with attempts to rethink and redesign existing systems. In parallel, the first session will explore where potential synergies lie between Europe and elsewhere, and how those might be harnessed in the years ahead.   

Indicative discussion points:

  • How does Europe’s new initiative compare with international equivalents?
  • Do beacons of excellence in assessment reform already exist, and what can be learned from their experience in breaking new ground?
  • Will Europe’s initiative affect research cooperation with other parts of the world, as well as strengthening its science diplomacy?
  • When and how would it make sense to engage with other countries and regions in pursuit of a common approach (principles, metrics, criteria, etc)?

10:40   Desired outcomes: What should a post-reform assessment system look like?

A forward-looking debate of what the European research community would consider ideal outcomes from the assessment reform exercise, as well as the fundamental building blocks of a system that has relevance and value across the R&I landscape.

Indicative discussion points:

  • Is there a common understanding of what Europe’s research assessment system should look like by 2027? Does a one-size-fits-all assessment model make sense, given national contexts and research diversity?
  • What are the potential risks in introducing a new approach, and how can they be managed?
  • Can assessment models be made relevant to all research funding and performing organisations beyond the “ivory tower”? What are the related implications for career development in both public and private sectors?
  • How should future assessment models measure the contribution of research and researchers to tackling societal challenges? 

11:25  Coffee Break

11:45   The road ahead: Defining concrete steps towards systemic change

A focused discussion seeking to map out a clear direction of travel and potential vehicles for innovation and experimentation in research assessment reform, and to identify clear recommendations and ideas for future policy consideration.

Indicative discussion points:

  • What can Europe’s R&I community expect in terms of next steps in implementing the Agreement on Research Assessment Reform?  
  • Are there good examples of multistakeholder collaboration to follow? How to ensure that industry, SMEs, service providers and others are actively involved? 
  • What kind of expertise and resources will be needed to support experimentation platforms and “sandboxes” for innovative ideas?
  • Will evaluators and assessors themselves require new skills and competences?
  • Which role for Europe’s University Alliances in the years to come?   

12:45  End of meeting

            Followed by networking lunch for participants in Brussels