This is a letter written by the President of Nanyang Technological University, Dr Su Guanning, and it first appeared in The Straits Times on 27 September 2010.
Setting record straight on uni rankings
The 2010 Times Higher Education World University Rankings was released recently. The Straits Times carried a story on 17 Sept with the sub-heading: “NTU’s ranking plunges in revamped Times Higher Education list”. The opening paragraph said that Nanyang Technological University had tumbled 101 places.
The story has caused consternation among some students, faculty and alumni. An examination of the methodology behind such rankings would have made it clear there is no cause for alarm.
To begin with, let me state categorically that NTU did not drop 101 places in the global university rankings. The story has done a grave disservice to the university. Clearly, its leadership cannot simply look on with bemusement at the way the rankings were reported. We need to set the record straight.
Since 2004, Quacquarelli Symonds (QS) and Times Higher Education were co-producing the Times Higher Education-Quacquarelli Symonds (QS) World University Rankings. This year, the two organisations went their separate ways.
The QS World University Ranking was released in early Sep 2010. It continued with the same methodology it had used since 2004. The Times Higher Education, however, designed a completely different methodology.
It would be erroneous to compare the two rankings because they are based on entirely different assumptions. Comparing them would be like comparing apples with durians.
The importance of not doing so was stressed by none other than Times Higher Education rankings editor Phil Baty and reported in The Straits Times story: “Any movement up or down the ranking table cannot be seen as a change in performance by an individual country or institution, and it’s not relevant to make comparisons.”
The QS 2010 Ranking is today the most widely-used world university ranking. Nanyang Technological University was placed 74th in the list and the National University of Singapore 31st, both down by one position from last year. The performance of both universities has been consistent over the last four years.
NTU started off as a practice-oriented, teaching university in 1991 and is in fact the youngest university in the Top 100 ranked in the QS 2010.
The Times Higher Education 2010 rankings is entirely new. Its criteria have yet to be accepted by many universities. A detailed analysis reveals it is 88 per cent computed from research-related indicators with unusual normalization of data resulting in some bizarre results. In a joint article in Edmonton Journal, Indira Samarasekera, the President and Vice-Chancellor of the University of Alberta, and Carl Amrhein, the Provost and Vice-President (Academic) of the same university, wrote that the Times ranking has very peculiar outcomes that do not pass the “reasonableness test”. They advised the public to take the “rankings with a truckload of salt”.
Malcolm Grant, President and Provost of University College London pointed out to The Guardian that research citations, if not intelligently applied, can lead to bizarre results. He cited the example of Egypt’s Alexandria University being ranked above Harvard and Stanford universities in research influence in the Times Higher Education 2010.
The methodology used by Times leans towards classical research parameters that privileges older, more established universities. The citation parameter, for example, measures research papers cited between 2004 and 2008.
From the time a researcher starts on a research project and publishes the findings to the time the citations of that research paper build up, it may take about 10 years, usually more. The effect of this parameter being retrospective is that it is actually a reflection of universities’ performance from as far back as 1995.
For rapidly rising universities, the results of recent world-class research work are not immediately captured in rankings such as the Times.
Universities know well their rankings do not tumble overnight. It takes years to build up a university; its standing will not swing wildly within a year or two.
NTU started off as a practice-oriented, teaching university in 1991 and is in fact the youngest university in the Top 100 ranked in the QS 2010. We ramped up our research intensity more recently, accelerating in 2006. Hence, it will take many years for NTU’s research efforts to make an impact in rankings such as the Times.
NTU is confident that the great strides it has made in the quality of its faculty, staff and facilities, as well as in the partnerships it has established with top institutions worldwide, are transformational in their impact, and are a more accurate reflection of its quality.
For example, Professor Kerry Sieh, a world authority on earthquakes and former Chaired Professor in Caltech who joined NTU’s Earth Observatory of Singapore as its founding director three years ago, has published nearly 100 research papers, some of which have more than 200 citations each.
His first research paper published after he joined NTU appeared in the top journal Science in Dec 2008. It has nine citations so far. This illustrates the fact that even for outstanding researchers, it takes years to build up citations.
For rapidly rising universities, the results of recent world-class research work are not immediately captured in rankings such as the Times. But the reality for NTU students is that the benefits of being taught by world-class faculty such as Professor Sieh are immediate.
Not many Singaporeans realise that in the more established QS World University Rankings, Singapore has two universities in the Top 100 universities worldwide. Aside from Switzerland, Singapore is the only other small country to have achieved this feat. We should be proud of Singaporean universities and cheer them on.
The PDF for the letter in The Straits Times can be found here.