Using international rankings to help choose a Masters or PhD

By: Ben Sowter

People love rankings. Even the people who hate rankings love rankings… or so it would seem by the amount of time they spend talking and writing about them. And not just university rankings, but schools, celebrities downloaded, TV shows, hospitals, footballers, rich people – you name it and there has been a ranking of it. Using university rankings to help select a graduate program is now commonplace and can play a valuable role in the decision making process.

Many rankings focus on the more frivolous aspects of the world in which we live and are unlikely to influence your life – which movie you rent or which album you download are ultimately low investment decisions, and if you are misled by a ranking it is unlikely to prove disastrous.  When choosing an international Masters degree or PhD program, however, you are taking a big step and one that ought to have a profound influence on your life.  In this context, to depend too much on an international ranking of institutions, not programs, that are based on somebody else’s interpretation of what represents quality, would be a grave mistake.

This is not to imply that university rankings should be dismissed entirely, simply that they should be considered at the appropriate stage of the decision-making process and with a complete understanding of what the rankings are evaluating and how the various measures are aggregated to yield an overall relative position.  These key principles for the use of rankings in decision-making could be applied in almost any context, but when applied to international rankings of universities, they are crucial.

For use domestically

On a domestic level, university rankings have been in existence for many years, US News & World Report had operated university rankings in the US since 1983, The Times in the UK since the early 90s, and they have evolved in that time to include an impressive number of contributory factors.  In some cases they have been able to drive measurement factors in the education sector.  The US News & World Report project draws on 18 distinct assessment criteria, but they are different from those considered by The Times.  Yet even with such a comprehensive set of measures, often at both an institutional and a subject level, it still remains unlikely that their criteria will perfectly reflect those of every applicant considering either undergraduate or graduate education. Why then does it seem that the most typical usage of rankings seems to be, “I should go to the most highly ranked institution that will admit me”?

The truth is that the rankings should, at best, provide a tool for short listing potential options early in the decision making process – you can’t visit every university, you can’t even spend sufficient time reading prospectuses and visiting websites from every university – you need to narrow the field.

… and internationally

At an international level, this is even more important and even more necessary (as the initial field of options is much greater).  International rankings have an additional constraint – they can only measure criteria that are globally accessible – at present there is no dependable international ranking taking into consideration more than six measures.  So not only are the people behind the ranking likely to have a different opinion on which factors are more important than others, there are likely to be some that you would consider crucial that they are not measuring at all.

The two most prominent international rankings at present are the THES – QS World University Rankings, originating from the UK and the Academic Ranking of World Universities compiled by a team at Shanghai Jiao Tong University in China. 

The criteria for both rankings are different and can be seen in their separate elements in the tables below.

THES – QS World University Rankings Criteria & Weights

Academic Ranking of World Universities Criteria & Weights

Clearly, these rankings take very different approaches and as a result also yield very different results – particularly outside the Top 10 or 20.  So, which one is right?  Answer: neither.  Despite the apparent objectivity of a ranking, the selection of criteria and the weights applied is a subjective process – there is no right answer – at least not in terms of selecting an institution with a Masters degree or PhD program in mind.  Both of these rankings measure well that which they claim to measure and nothing more.

Neither ranking considers the value of scholarships available, quality of facilities, student satisfaction, alumni giving, return on investment, accurate teaching quality assessment, entry requirements, average class size, international student support or availability of student accommodation all of which, amongst others, could be of great interest to a prospective international graduate applicant.  These are all things that need to be independently researched after, perhaps with the help of these rankings, a short list has been drawn up.

These rankings, however, do reveal interesting and different, things about the institutions they examine. 

The Shanghai ranking is particularly focused on institutions whose main strengths lie in scientific areas – as reflected by the ability of LSE to perform well in the THES – QS ranking but barely register here.  The main reasons for this scientific emphasis are down to the selection of criteria – citations and publications contribute to 70% (they are factored against size for the last 10% along with all of the other criteria) of the weighting and yielding significant advantages where journal publications are a significant reality of academic life – not yet the case in Arts or Humanities and only a fledgling culture in Social Sciences.  There is also a geographic influence; the US has an exceptionally strong publication culture the influence of which may be more significant than the relative strengths of their institutions’ actual work.  Of course, the THES – QS ranking is also subject to such bias, but with only 20% influenced by citations, it is less profound.

Nobel Prizes also lend a scientific bias to the results (the Nobel Prize for Literature is rarely, if ever, awarded to an academic and associated with a university) and to gather sufficient data must look back over 50 years.  Whilst it is true that most of the world’s strongest institutions 50 years ago are still strong today, the same cannot be said in reverse – in fact, some of the world’s strongest universities today barely existed 50 years ago, leaving them, despite their virtues, at a woeful disadvantage.  Nonetheless, if you are interested in pursuing a Masters degree or PhD program in a scientific or technical field then ranking is worth your consideration in helping you build your shortlist.

The THES – QS ranking has a very different approach, with 50% of its weighting attributable to opinion – both of academics (40%) and employers (10%).  Whilst delivering a more balanced picture across geographies and subject area, this approach raises different challenges – such as respondents assuming Harvard is great, not necessarily due to familiarity with their work but just because they assume it must be.  With this in mind, however, the ranking provides a reasonably balanced, if imperfect, view on comprehensive universities and should make the process of building your shortlist straightforward – particularly for those applicants that may not be dead set on a particular academic subject yet.

What lies ahead

Ultimately, however, this is just the first generation of international university ranking. 

Before long rankings will help guide applicants not only on institution choice but specific program selection as well and increasing level of detail will appear alongside the opportunity for applicants to assign their own weights to the criteria they think are important.

However sophisticated they become, though, rankings will never become an entire substitute for the decision making process – any applicant should and must ensure that they go hand in hand with more qualitative factors and, perhaps most importantly, actually meeting and talking to people who are at the institution, or once were.  Nothing replaces the experience of firsthand contact with a university or academic programme for contributing to your final and well-informed decision.

Click here for more information on international university rankings in general and a complete explanation and results of the THES – QS World University Rankings.