Commercial rankings are increasingly at the centre of discussions as universities reconsider their role as benchmarking tools and question the validity of their methodologies. Universities are exploring alternatives that prioritise quality over quantity and more fully recognise modern practices like those supporting open science. User-driven alternatives to traditional rankings are gaining attention, and their integration with existing rankings is a key focus.
This evolution is happening alongside efforts to reform research assessment, highlighted by the Agreement on Reforming Research Assessment signed by our association in November 2022. Despite the criticism, commercial rankings still significantly influence student decisions, and some institutions have successfully used them in outreach and engagement.
In that context, our Task Force Benchmark and Board of Directors held a joint workshop ‘Reimagining university rankings: exploring strategic priorities and alternatives’ on 19 June, hosted by KTH Royal Institute of Technology, in Stockholm. This workshop provided a platform for in-depth discussions on the role of commercial rankings.
In the first part of the workshop participants discussed movements in the landscape of university rankings including decisions of major universities opting out of rankings. Peter Elspass, Chair of Task Force Benchmark and moderator of the workshop, outlined the purpose, added value and shortcomings of commercial rankings. He provided insights on the practical implications of rankings in his daily work as Head of the President's Staff Department of University Development & Controlling at Leibniz University Hannover. Peter emphasised their role in simplifying complex institutional evaluations, ensuring comparability across diverse university sectors through standardised metrics and methodologies. Additionally, rankings serve as strategic marketing tools to attract students, faculty, researchers, and funding.
We then invited contributions from Paul Boselie (Chief of Open Science at Utrecht University) and Manuela Höfler (Co-Director Open Science Office, University of Zurich) and Rüdiger Mutz (Senior researcher, University of Zurich) to share insights from the strategic decisions of their universities not to submit data to Times Higher Education rankings, thus effectively excluding themselves from the ranking.
Paul and Manuela highlighted that the decision of their institution to opt out of commercial rankings was driven by commitments to open science. Paul cited concerns over the validity and commercial biases inherent in ranking systems. He emphasised broader support for this move among institutions globally, particularly from universities in the global south. Manuela echoed these sentiments, explaining University of Zurich's similar stance and its comprehensive integration of open science practices across faculties and central units. She pointed out the problematic aspects of investing resources in rankings, emphasising that the University of Zurich prioritises creating good conditions for research, teaching and learning and open science practices.
Paul recognised a global movement towards reevaluating ranking systems, anticipating a pivotal shift away from ranking-based learning assessments in academia. He discussed the growing influence of open science initiatives on research funding and interdisciplinary collaboration, without observing any negative effect in grant acquisition. Both speakers emphasised the internal and external benefits of opting out, including enhanced institutional visibility and reputation, and attractiveness to prospective academics.
Ludo Waltman (scientific director of the Centre for Science and Technology Studies (CWTS), Leiden University) opened the second part of our workshop by presenting the new Open Ranking of the CWTS Leiden Ranking. Ludo underlined the importance of providing universities with statistics and benchmarking aligned with open science principles.
He introduced the Leiden Ranking Open Edition, launched in January 2024. Unlike traditional rankings that rely on closed data often from commercial sources such as the Web of Science, the Leiden Ranking Open Edition uses open data sourced from OpenAlex. Ludo referred to key initiatives in the sector, such as the report from the University of the Netherlands "Ranking the University" and CoARA (discussed here). Ludo also highlighted the Barcelona Declaration on Open Research Information, a collaborative effort involving over 25 research information experts from organisations involved in research funding, evaluation, and open infrastructure, published in April 2024.
Ludo encouraged participants to move away from a ranking system dominated by a few entities and instead encouraged universities to focus on rankings that align with their specific purposes and goals. He expressed hope that increased transparency would lead to improvements in ranking methodologies and outcomes.
Following up on a short report on the ‘More than our rank’ initiative, Aldo Torrebruno (Senior officer, Planning and Control unit, Politecnico di Milano), mentioned that the university is currently evaluating whether they should continue participating in the initiative. He emphasised the importance of providing universities with benchmarking opportunities without the pressure imposed by traditional rankings.
During both workshop sessions, participants engaged in small group discussions. At the end of the first session, they mapped their opinions visually on a scale from “This is the beginning of the end of commercial rankings” to “Status quo: nothing much will change,” showing how they viewed the ranking landscape. Based on these maps, participants split into five groups with similar views to discuss their thoughts together.
After the second session, participants formed small groups, each focusing on a specific question. Some groups explored how to improve traditional league tables, while others looked into alternative ways to analyse data and set new goals for evaluating universities. Each group brainstormed ideas and explored potential steps to take next, both for their own university, and for our association.
The conclusions from these discussions were shared during a final session, and they will be included in an upcoming workshop report.
Presentation by Paul Boselie (UU)
Presentation by Manuela Höfler and Rüdiger Mutz (UZH)
Presentation by Ludo Waltman (CWTS Leiden Rankings)
Pictures of the meetings at KTH Royal Institute of Technology in Stockholm are available on our Flickr.
For more information, please contact our Junior Advisor for Education and Benchmark Touko Närhi.