Multidimensional university ranking system developed

Russian universities, like universities from other countries, increasingly compete not only at the national level but also globally. This trend is reflected in the growing interest in global university rankings. Despite criticism, rankings are in demand and influence universities’ positioning in the international higher education market.

In Russia almost 30 different approaches to ranking have been developed recently and tested in a bid to satisfy the needs of various stakeholders.

All of these approaches are single-dimensional rankings that use a composite indicator and weight coefficients. They have drawn the interest of prospective students, universities and the academic community. They have also been criticised by various stakeholders.

A template methodology

Thus in 2011 the Ministry of Education and Science contracted the National Training Foundation, or NTF, to develop and approve a template methodology for a Russian university ranking, as part of the project “Developing and Approving a Template Methodology for National Ranking of Higher Education Institutions”, to be piloted between 2011 and 2013.

The NTF team was guided by several principles: the methodology should reflect the complexity and diversity of the Russian higher education system and its development objectives; it should be based on best international practice and International Observatory on Academic Ranking and Excellence (IREG) criteria; and it should provide reliable data on the performance of institutions and their position in the national higher education system.

The four target user groups include: prospective students and their parents; government (central and local); employers; and the academic community.

Thus a multidimensional approach to ranking higher education institutions was adopted.

Five functions of higher education institutions have been identified for assessing university performance: research, teaching and learning, internationalisation, knowledge transfer, and engagement with regional stakeholders.

It should be noted that the assessment is based on quantitative indicators and does not use qualitative data from surveys of students, academic staff and employers. However, inclusion of qualitative data in the multidimensional ranking methodology is perceived as one of the ways it can be developed in future.

The pilot project

There are more than 1,000 higher education institutions in the Russian federation, and 103 took part in the methodology pilot, including eight federal universities and 28 national research universities.

The results demonstrate that the multidimensional ranking methodology reveals the qualitative characteristics of Russian higher education institutions, identifies their relative strengths and weaknesses in the five areas, and helps them to shape development strategies.

The template methodology lays the foundation for a national approach to assessing the performance of national higher education institutions, which takes account of the Russian higher education system's diversity and development goals.

One future development will be the selection of a core of ‘excellence’ indicators for monitoring the performance of Russian higher education institutions, which will receive government support for enhancing their competitiveness among the world's leading universities.

A detailed description of the methodology is available here.

The outcomes of the project and the future application of the ranking system will be discussed at a conference titled “Lessons from Multidimensional Ranking of Russian Higher Education Institutions: From piloting to practice”, to be held from 25-26 April 2013 in Moscow.

* Dr Marina Larionova is head of the Higher School of Economics International Organisations Research Institute and head of international programmes at the National Training Foundation in Russia. She will be speaking at the rankings conference in Moscow next week.