U-multirank demands across-the-board data collection effort but delivers patchy returns

My university, Sweden’s Royal Institute of Technology (KTH), recently participated in the compilation of the new university league table, U-multirank, receiving an overall ranking and individual rankings for our curricula in electrical engineering, mechanical engineering and physics.

In drawing up the ranking, the developers of U-multirank requested numerous data. It was very time-consuming to collect and a large number of staff from the university administration and three of KTH’s schools were involved in this work.

KTH got very good results in the overall ranking: very good in 15 indicators; good in six indicators; average in one indicator; below average in four indicators; and weak in one indicator.

It’s apparent that U-multirank is designed in such a way that no university could possibly get good results in all indicators. Some indicators promote the international research-intensive university, while others favour the regional university.

It's very difficult for a large, international and research-heavy university to get good scores for regional engagement, although this also constitutes an important focus for our university.

Rating scales

The rating scale for the indicators is sometimes odd. For example: Why is it better if students do their internships in the university’s locality rather than further away? Why is it important to have many bachelor programmes in the English language?

I am also hesitant about the use of student surveys in rankings. KTH students were not interested in responding when we polled them, and it seems likely the results of other universities too are based on very few answers – and thus a weak statistical base (if there were less than 15 responses, U-multirank did not present any results). I do not think it is a good idea to rate the quality of education solely from a student survey.

Putting demands on the users

The tool as presented on the U-multirank website - the finished product, is difficult to use and makes great demands on the user.

I don’t believe the tool is especially useful for an 18-year old student who is considering which university to apply for. For example, for a young student, the indicator measuring regional income must appear almost incomprehensible.

Considering all the work that KTH put into this project, and given the results for the subject rankings, it was not really worth the effort. I do not think this ranking will be a great success.

Per-Anders Östling is a senior administrative officer with the Royal Institute of Technology (KTH). The opinion in the article are his, and do not necessarily represent the position of the Royal Institute of the KTH.