Why Sorbonne pulled out of university ranking

France’s Sorbonne University plans to leave the Times Higher Education (THE) Rankings, adding its name to a growing number of universities rejecting lists that play one institution off against another. According to its president, most of these rankings are “black boxes” whose methods not only raise ethical questions but also fail to cover the breadth and diversity of university contributions.

“By deciding to stop sending our data to THE, we are leaving this specific ranking, but our criticism of major international university rankings is global,” Nathalie Drach-Temam, president of Sorbonne University, told Science|Business. “These rankings, built on selected quantitative indicators amalgamated into a single score, are not designed to evaluate research nor reflect the breadth and depth of the missions of research and higher education institutions.”

From the UK-based Quacquarelli Symonds (QS) ranking to the US News and World Report (USWR), university rankings set out to measure how well a higher education institution performs and how its performance and quality compare to its peers. Prospective students turn to them for guidance, and governments and investors base their research funding decisions on them.

But more and more institutions are opting out of university league tables, as part of a wider movement for universities and researchers to be judged on quality, and not quantity. In late 2022, the law schools at Yale, Harvard and Columbia dropped out of USWR; a few months later, South Korea’s top research universities called for a boycott of the QS ranking; more recently, the University of Zurich decided to abandon THE.

“These rankings are [. . .] black boxes that operate in a closed system,” Drach-Temam said. “The data on which they are based are not shared, the methodology is partially disseminated, which means that these rankings are not reproducible and cannot be questioned by the universities concerned, which therefore cannot fully appropriate them.”

Sorbonne University is a member of the Coalition for Advancing Research Assessment, set up to advocate for sounder assessment methodologies. Its Agreement on Reforming Research Assessment states that avoiding the use of the global rankings “will help the research community and research organisations regain the autonomy to shape assessment practices, rather than having to abide by criteria and methodologies set by external commercial companies.”

Biased methodologies

Global rankings are based on criteria as diverse as illustrious alumni and industry income, but their critics think they still promote a view of institutional success that is too narrow and unfair. Not only do they overlook many contributions a university makes to society, but they also tend neglect the diversity of their social, economic and political context.

“They cannot reflect our policies on inclusion, sustainable development or science-society dialogue, for example,” Drach-Temam said. “Their methodology, by focusing on English-language journals, favours certain disciplines, while social sciences and humanities, whose publication methods and dissemination languages are more diverse, end up being disadvantaged.”

She also said that some methodologies raised scientific and ethical concerns. 

Another risk is that universities may be inclined to take temporary, superficial measures to boost their ranking rather than focusing on long-term quality enhancement, undermining the core purpose of these rankings.

Related articles

• The Draghi report one year on: what has changed?
• Utrecht University withdraws from global ranking as debate on quantitative metrics grows

These problems have led the International Network of Research Management Societies to create the More Than Our Rank initiative for academic institutions to show that the success of universities goes beyond their ranking position.

However, Drach-Temam did credit THE for allowing universities to leave its ranking, an option not available for the Shanghai and QS rankings. Sorbonne University is also part of the CWTS Leiden Ranking which, she explained, does not conflate very different criteria into a single score, has an open methodology, and allows universities to measure their open access rate.

Open data infrastructures

Sorbonne University has also decided to leave the Web of Science, a subscription-based citation database operated by the analytics firm Clarivate, “mainly because it no longer seemed possible to us to rely on proprietary and closed data to manage our research, let alone for its evaluation,” Drach-Temam said.

Just like the rankings, she finds these databases skew towards English-language journals and fail to correctly reference literature and humanities publications, particularly in French. “A considerable and particularly rich part of the activity of our university is therefore quite simply ignored in these databases,” she said.

Sorbonne University now plans to focus on open and participative infrastructures such as OpenAlex, which offers free access to academic publications. “Beyond financial issues, supporting these infrastructures also means getting involved in their governance and making our voice heard in their directions and projects,” Drach-Temam pointed out.

The French university is already represented on the OpenAlex advisory board.

The idea, for Drach-Temam, is not to replace one tool with another but “to change the paradigm, and to give universities and research institutions the means to reclaim ownership of their data [and] make them accessible to society.”