China is choosing its own path on academic evaluation

On 18 February 2020, the Ministry of Education and Ministry of Science and Technology jointly issued a notice announcing a reform of China’s academic evaluation system.

Its main points include a proposal that the number of papers published in internationally indexed journals, especially those included in the Science Citation Index (SCI) or the Social Sciences Citation Index (SSCI), and the citations of papers in related journals should not be used as key indicators in China’s academic evaluation system as they currently are.

This means that SCI papers, citations of papers and other related indicators should not be used to measure the level of innovation or innovative research. Neither should they be used as required standards for universities or research institutes when awarding doctoral degrees, hiring new researchers, promoting young researchers to a higher rank or determining academics’ bonuses, research funding or their application for academic prizes or rewards.

Further, universities are to be banned from setting up any quantitative targets for individual researchers to publish SCI papers or citations of their papers.

More importantly, the numbers of SCI papers, ranking of the essential science indicators or other related indicators should no longer be used to measure the quality or reputation of individual researchers, specific disciplines, Chinese universities and research institutes.

Therefore, the strategies or action plans relating to SCI papers and citations of papers and the global university ranking table ambitions which have been developed by individual universities will have to be amended.

What could happen next?

This announcement will result in significant changes in Chinese academic evaluation systems, the degree awarding system, institutional evaluation schemes and especially the evaluation of researchers’ and doctoral students’ academic performance.

For example, in general, researchers and doctoral students, especially those from the humanities and social sciences, will make less effort to publish their papers in indexed journals like SCI or SSCI journals and pay less attention to the importance of enhancing the citations of their papers and to international impact.

Also, it is highly possible that researchers focusing on theory or basic research in some disciplines may show less interest in publishing in indexed journals. Key reasons for this change may include the fact that it is more risky for Chinese researchers and doctoral students from the humanities and social sciences to publish in indexed journals than those from natural sciences due to the fact that much of their research is concerned with sensitive topics and questions in China.

Many of their research fields or themes are not allowed to be researched in China, for instance academic freedom, institutional autonomy, the dominant control of the Communist Party over higher education and research, and academic corruption.

Additionally, researchers from the humanities and social sciences seem to have smaller international networking communities and undertake less international collaborative activities with colleagues abroad than those from natural sciences in China.

Similarly, less emphasis may be placed on using SCI papers or citations of papers in natural sciences, including medical sciences, to evaluate researchers. For example, more domestic and other qualitative indicators may replace SCI papers or citations of papers.

Moreover, doctoral students from leading universities and research-intensive universities as well as national research institutes could graduate and earn their doctoral degrees without having to publish SCI papers.

Young academics could also be promoted to a higher academic rank without being asked to publish SCI papers, even those working in leading universities. And the numbers of SCI papers and citations of papers may no longer be used as direct and important indicators in the selection of national-level talent, principal investigators of national research projects or the award of national-level prizes in research.

Finally, as researchers and doctoral students, especially those from medical sciences or life sciences, will no longer be required or forced to publish SCI papers to get degrees, promotion or research funding, it is possible that the number of low-quality or fraudulent papers by Chinese researchers might reduce as the total number of papers in circulation are likely to decrease.

Although no practical or operational plans or recommendations have been referred to in the announcement, it is assumed that more diverse indicators based on different levels and types of universities, research institutes, disciplines, types of research activity (basic, applied, development or military) and public engagement will be created and introduced in the new evaluation system.

For example, for those from the soft sciences, it seems that the more qualitative and practical aspects of their research will be more highly valued, such as social impact, to what extent policy recommendations have been accepted by government, what contributions they have made to the healthy development of Chinese society, how much they have met political and ideological requirements and to what extent they have solved Chinese problems as well as demands from industry and business.

Furthermore, more peer reviews by domestic experts will be encouraged to evaluate researchers’ performance and researchers’ loyalty and service to the Party, and national economic development and Chinese society are likely to be further emphasised.

What is more, the number of SCI papers and citations of papers is likely to play a less decisive or prominent role in the prospective evaluation frameworks; rather, a mixture of both quantitative and qualitative ways of assessing research and evaluation frameworks based on subjective observations and reviews by Chinese researchers will become increasingly important.

Background

In September 2018, when the national educational conference was held, President Xi Jinping announced the need to reform the current academic evaluation systems in China. In the aftermath, the Chinese government, several top researchers and some members of the mass media have continued to criticise the current evaluation systems used in higher education and their harmful effects on the development of China’s higher education and research.

For example, it is said that in some leading universities, researchers cannot be promoted to a higher rank simply because they have not published any SCI papers, no matter how excellent they are.

Similarly, doctoral graduates, including those who have graduated from foreign universities, cannot be hired at top universities if they have not published any SCI papers. Some universities even allocate a huge bonus or other funds to researchers who have published in indexed journals as a reward.

More importantly, even some top scientists have faced questions about false or made-up data or photos in indexed journals because of the pressure to publish in these journals.

A Chinese approach

By removing the emphasis on the number of SCI papers and citations of papers, as well as their dominant influence on measuring research quality, China wants to make fundamental changes in its academic evaluation systems and aims to establish new evaluation systems which are more relevant and responsive to the Chinese academic context and Chinese problems.

According to the notice, the new evaluation systems should encourage researchers to contribute to the establishment of a nation that is strong in education and science and technology, rather than merely serve the international community, international-indexed journals or foreign publishers.

However, there are numerous challenges to come. The biggest appears to be how to develop a new evaluation system which can really measure the research quality of Chinese universities, research institutes and individual researchers while also being compatible with globally accepted standards and acceptable to the international academic community.

Issues such as who will create the new systems, to what extent they can avoid the problems identified in the current systems, potential challenges and questions about the extent to which they will affect Chinese doctoral education, the production of researchers and academics, the selection and promotion of young promising researchers, the international reputation and quality of Chinese higher education and research and the direction of Chinese higher education and research have not yet been explicitly addressed.

This lack of specific and operational suggestions or recommendations on forming the new academic evaluation frameworks, especially at institutional and individual levels, seems to be the biggest challenge facing the Chinese government now.

However, other issues include: how to develop more diverse academic evaluation systems by taking into consideration different types and levels of higher education institutions, different disciplines, different types of research activities, and different types of researchers, academics and doctoral students.

Furthermore, as some of China’s domestic journals have not established strict peer-review systems like international indexed journals, incidents of academic corruption are a distinct possibility.

Fall in global ranking expected

It is extremely difficult to predict precisely what outcomes the implementation of the announcement will bring to China’s higher education and research in the future.

However, the impact on China’s current academic evaluation system, missions and goals of producing and training researchers and doctoral students, doctoral education, academics’ research activities, the status of Chinese universities in the main global ranking tables and even the internationalisation of China’s higher education and research are likely to be significant.

It is likely that the numbers of research papers to be published by Chinese researchers in indexed journals will decline enormously and that the ranking of several Chinese universities will fall in global university ranking systems.

This latest announcement is part of a major shift in China’s national policy from using international academic standards, typically represented by the numbers of SCI papers and citations of papers and other related metrics based on Western ideas and standards, to measure the quality of Chinese higher education and research, training, selection and evaluation of Chinese doctoral students, researchers and national projects of higher education and research, towards an emphasis on developing more diverse indicators and especially towards creating new standards based on the Chinese context and academic values and domestic demands.

Professor Futao Huang is based at the Research Institute for Higher Education, Hiroshima University, Japan, and is also co-investigator on the Centre for Global Higher Education’s global higher education engagement research programme.