Does the World Need More University Rankings?

This question was among those discussed at last week’s IREG-7 Conference, jointly organized by the IREG Observatory on Academic Ranking and Excellence and the QS Intelligence Unit – a rare occasion which brought together representatives of the world’s major international university rankings providers in a (reasonably) amicable panel discussion. The focus of the session was “New International Academic Rankings”, kicking off with an overview of recently released and forthcoming university rankings projects.

Bob Morse, director of data research at US News & World Report, outlined plans for a project to rank universities in Arab countries, being developed for initial release towards the end of this year. Then Gero Federkeil, manager in charge of rankings at higher education think tank CHE, gave an introduction to the newly launched U-Multirank project, which aims to provide in-depth comparisons between similar types of university on more than 30 indicators.

Times Higher Education’s Phil Baty discussed newer additions to the THE university rankings suite, including the Top 100 Under 50 and the BRICS & Emerging Economies Rankings. And QS’s own Ben Sowter outlined plans to extend the data provided by existing projects such as the QS World University Rankings by Subject, as well as discussing potential new regional rankings covering areas such as Eastern Europe and Central Asia.

Why do we need international university rankings?

International university rankings are still relatively new, having first been developed about a decade ago. They appeared at a time when the number of students choosing to study abroad was growing rapidly, leading to growing demand for ways to compare universities across national borders. At the same time, more universities and governments were looking for ways to raise their profile in the international higher education community, and seeking data that could help with global-level benchmarking.

Today, international university rankings are widely used by many groups, including students, academics, university staff members, private-sector researchers and graduate employers, governments and public bodies. Every year, the online rankings tables attract more visitors and more media coverage, showing that these data sets really do hold interest and usefulness for a large and growing audience.

Why MORE international university rankings?

The extension of existing international university rankings – to cover more universities and provide more detailed data breakdowns, for instance – and the launch of rankings focused on particular subsections of the global higher education sector, has come in response to demand for more specific and extensive data.

To this end, more sophisticated ways of publishing the rankings online have been developed, making it possible for users to compare universities on specific indicators (such as employer reputation), while new rankings have been created to provide more detailed and meaningful comparisons. For instance, the QS World University Rankings by Subject currently covers 30 separate subjects, and is being expanded each year.

Then there are regional rankings, such as the QS Asia and Latin American rankings, and most recently a ranking dedicated to universities in the BRICS countries. These have two main goals. First, to provide information about a greater number of universities than appear in the overall international rankings. Second, to facilitate deeper comparison of universities within a region; while based on the methodology used for the world rankings, the regional sets also include additional indicators. For instance, the QS rankings of universities in Latin America and in the BRICS countries each provides information about the percentage of academic staff members with a PhD, while the Asia ranking includes comparisons of incoming and outbound exchange students.

What are the limitations of international university rankings?

While of course highlighting the usefulness of international university rankings for various groups, the IREG-7 session was also an opportunity to discuss some of the limitations of these projects.

Several critics pointed out the element of arbitrariness inevitable in methodologies which combine multiple indicators to give one overall score; why should certain types of measure be given more weight than others? Another participant challenged the use of reputation surveys, arguing that these simply serve to keep the same well-known universities at the top of the rankings, in a self-feeding cycle. Others were critical of the use of data provided directly by universities themselves.

The rankers themselves were more than willing to recognize the limitations of their systems, while maintaining that the information they provide nonetheless remains useful (not to mention impressive, given the amount of data that has to be collected and analyzed every year). Phil Baty acknowledged the importance of being a “franker ranker” – being as transparent as possible about the methodology used and the sources of the data collected. And Ben Sowter summed up the situation with a quote from statistician George E. P. Box, who famously said, “Essentially all models are wrong, but some are useful.”

More sophisticated rankings, more active users

U-Multirank is interesting as a project which attempts to move beyond some of the criticisms university rankings commonly face, by placing greater onus on users to create and interpret their own set of results. First, users narrow down the type of institution they are interested in, so the results show a more meaningful “like for like” comparison (for example, universities of a similar size, specialization and research scope). Second, they select the indicators which hold most significance for them (such as graduation rate, opportunities to study abroad or patent applications).

The advantage of this approach is that users are forced to think more carefully about what the data they’re being shown means, and what’s really important to them. What U-Multirank deliberately refuses to provide is an overall amalgamated list – the one that makes media headlines, gets attention and shows regional and global trends at a glance.

It seems unlikely that the appeal of this “at a glance” level of international university rankings will fade away any time soon. (This is, after all, the era of the “listicle”, and the impulse to “rank” seems to be pretty deeply embedded in the human psyche.) But, especially as the rankers find ways to make the information even easier to access and interpret, it seems likely that users will choose to take on more active roles. And in the meantime, the ranking providers will be busy thinking of new ways to extend, improve and innovate, bringing even more information to the screens and fingertips of students, academics and policy makers worldwide.