How ‘publish or perish’ damages doctoral research

The newly published study on Research Assessment in the Transition to Open Science shows some remarkable results that directly relate to doctoral education. In the framework of the study, the European University Association or EUA asked institutions how they evaluate researchers and their academic activities.

According to the report, quantitative indicators still play a key role in research assessment, also for the new generation of researchers. For 82% of the responding institutions, metrics measuring research output based on number of publications and citations are considered either very important (53%) or important (29%).

Qualitative peer assessment of publications comes in second with 74% of institutions considering it either important or very important. EUA also asked institutions which indicators they use. Here the journal impact factor and the h-index are the most widely used indicators for research assessment.

The relevance of this data is in line with results of other EUA surveys in the area of doctoral education, including the ARDE survey on quality assurance and the 2019 Europe-wide survey on doctoral education, in which responding institutions declared that publications are the most widely used to evaluate doctoral education.

Impact factor

The key role of publications is not surprising. Publications are written traces of research results and document them. However, as previously mentioned, indicators like the journal impact factor or the h-index measure the popularity of a journal and look at the number of citations a researcher is able to generate. Neither look at the content and quality of the publication itself; rather they are used as a proxy.

The dominant role of these indicators entails several risks. First, early career researchers are forced to focus on quantity in order to increase their chances for an academic career. The hunt for impact factors becomes a part of academic socialisation.

As a consequence, there is a lack of time and space to seriously dedicate oneself to a research topic, which should be the heart of the doctorate. This can also lead to unnecessary stress and the kind of anxiety and other mental health issues which are increasingly being seen in the area of doctoral education.

Interdisciplinary research can become a liability as disciplinary publication patterns often differ and interdisciplinary journals in general have lower impact factors.

Doctoral candidates and other early career researchers may also be discouraged from spending time gaining teaching experience or from communicating their results through alternative channels like the media, neither of which normally increase impact factors.

A holistic approach to academic publishing

On an institutional level, the focus on quantitative proxies can also be counter-productive to open access and open science. Serious reforms leading to open access are only possible if the way researchers publish changes and a more holistic approach to academic publication is developed.

As long as researchers are forced to focus on a few high-level journals, they will not be able to broaden their focus or even take alternative elements of research assessment into account. Early career researchers who opt for alternative forms of publication or publish in an open access journal may be penalised for their choice.

In addition, the ‘publish or perish’ approach can have damaging effects on research integrity. If funding and career opportunities are tied to producing as many outstanding results as possible, then the pressure to use questionable research practices or even fraud increases.

Some researchers may be tempted to divide research results into as many partial publications (salami slicing) as possible or to publish non-reproducible results prematurely.

Reflection

In this regard, doctoral schools can play an important role. While they are not in a position to change the assessment system as such, they can serve as a place of reflection where knowledge and views can be exchanged.

No one can say with certainty what kind of research and publication system we can expect 20 years from now.

So far only a few institutions, when evaluating proposals, take into account whether an early career researcher has published in open access forums. However, this may change, making it important that early career researchers are able to make informed decisions and navigate through the complex area of research assessment.

It is worth discussing how doctoral schools can best address this issue. Reflections on research assessment can be included in training related to research integrity or open access, as a cross-cutting topic or separately. In addition, it is important that supervisors are sensitive to a world of research that may differ significantly from the one they are used to.

Doctoral education can also be important for making sure the voice of doctoral candidates is heard. No one starts a doctorate with the aim of collecting citations.

To have an evolved assessment system, we must work with the new generation of researchers – not only to provide them with support for making career decisions, but to listen to them and include their input in the development of strategies and policies.

Alexander Hasgall is head of the European University Association Council for Doctoral Education.