GLOBAL: New university ranking aims for objectivity

A new university ranking seeks to use a sophisticated set of bibliometric indicators to rate scientific performance to establish the world's top 500 research universities.

The Leiden Ranking 2011/2012 aims to provide highly accurate measurements of the scientific impact of universities and of universities' involvement in scientific collaboration.

Of the top 20 universities, 18 are from the United States and two from Switzerland (École Polytechnique Fédérale de Lausanne and ETH Zurich).

The Massachusetts Institute of Technology heads the table, followed by Princeton with Harvard in third place. Cambridge is the top UK university in 31st place, with the Hong Kong University of Science and Technology the highest placed university outside the US and Europe in 58th place.

The US has 127 universities in the 500, followed by Germany (39), the UK (36), China (31), Italy (25) and Japan (24).

The ranking, based on more than 25 years of bibliometric experience at the Centre for Science and Technology Studies (CWTS) at Leiden University in The Netherlands, is claimed to offer more advanced indicators of scientific impact and collaboration and uses a much more transparent methodology than other rankings.

Unlike the Times Higher Education or QS World University Rankings, it does not draw on reputational surveys or on data provided by the universities themselves, which it dismisses as subjective.

Among the improvements in the 2011-12 ranking are an impact indicator based on the proportion of top 10% publications, collaboration indicators based on geographical distances, fractional counting of collaborative publications, and the possibility of excluding non-English language publications.

"Comparing the impact of non-English language publications with the impact of publications written in English may not be considered fair," the compilers say. "Non-English language publications can be read only by a small part of the scientific community, and therefore these publications cannot be expected to receive similar numbers of citations as publications written in English."

The ranking also makes use of a statistical technique known as bootstrapping to smooth out variations in the data.

Its reliance on bibliometrrics places the Leiden Ranking closer to the Shanghai Jiao Tong University Academic Ranking of World Universities in approach. An exclusive focus on research metrics opens the ranking to the criticism of painting a partial picture of a university.


Comment

As with technology standards, the good thing about university rankings is that there are so many of them. In this instance, note that the Leiden website provides for numerous alternative rankings. For example, the results cited in the article are "normalised for university size (enrolment?)", and based upon one of four criteria, "proportion of the publications of a university that, compared with other similar publications, belong to the top 10% most frequently cited".

Quite different rankings result from changes in these characteristics. To its credit, the Leiden website also provides links to documentation of data collection and rankings methodology.

Daniel Updegrove, consultant on IT in higher education, Austin, Texas, USA