National peer review weighs up quality of education journals

An evaluation by the Academy of Science of South Africa (ASSAf) of South African educational journals has recommended that three out of 17 of those assessed be removed from the Department of Higher Education and Training’s (DHET) official list of accredited journals. Four others were ‘conditionally’ accepted for the list.

The Committee on Scholarly Publishing in South Africa Report on Grouped Peer Review of Scholarly Journals in Education, made public on 12 June, is the 10th in a series of discipline-grouped evaluations of South African scholarly journals by ASSAf.

The assurance process, based on independent multiple peer-review, is a precursor to the identification of journal titles to be accepted onto the open access platform, Scientific Electronic Library Online (SciELO) South Africa.

While five of the 17 journals assessed were already on the SciELO platform, three were recommended, three were invited to join subject to the implementation by the journal concerned of an open access model, and six were not recommended for inclusion in the SciELO SA platform.

Improvement in quality

According to Susan Veldsman, director of the ASSAf Scholarly Publishing Unit, the main purpose of the ASSAf review process for journals is to improve the quality of scholarly publications in the country in a manner that ensures high standards of scholarly practice – primarily in relation to rigorous peer review.

“It is not an attempt to regulate these publications in any way. ASSAf respects the independence of the editors and freedom of researchers and of the research process itself as important preconditions for the critical and innovative production of new knowledge,” she told University World News.

South Africa publishes approximately 272 accredited scholarly journals in different subject groups, but ASSAf divides these journals into broad subject groups that share a particular broad disciplinary focus for evaluation.

The 17 journals form the ‘education cluster’ and were evaluated as a group for the first time, and in the following categories: education in general (seven journals); education review (two); higher education and distance learning (three); language education (two); and mathematics and natural sciences (three).

Eight were specialist journals such as the Southern African Journal of Environmental Education and the South African Journal of Childhood Education and nine were more general, such as the Journal of Education and the South African Journal of Education and the Journal of Educational Studies.

The peer reviews were carried out by a series of purpose-appointed peer-review panels drawn from the ranks of researchers and other experienced scholars in and around the fields concerned in each case, as well as persons with practical (technical) publishing experience. Overall, the work was undertaken by three members of the ASSAf staff, six expert panel members and 53 expert scholars.

The panels examined a range of criteria which included editorial-process related criteria, business-related criteria and bibliometric assessments.

Among the editorial-process related criteria, consideration was, inter alia, given to longevity of the journal; number of original peer-reviewed papers per year in the last five years; manuscripts submitted; average length of published papers; nature of peer reviewers used per manuscript; delays before publication; experience of the editor; experience and selection of editorial board members; editorial policy review; and number of errata published annually.

Transparency in appointments

The panel reviewers wanted transparency in appointments of editors-in-chief and senior editorial staff through advertisements, with their terms of office clearly spelt out. The panel was critical of ‘in-house’ appointments by universities. The practice of editors who continue to publish in their own journals was criticised; and a “surprising number” of journals were found to have deficient editorial policies or were failing to publicly display such policies on their websites.

“Limited, or a lack of, international diversity was noted in a number of areas: in the case of board members, content, reviewers and readership,” the report said.

The review also identified “three loosely related matters” as common concerns: the absence of ‘additional’ or ‘enriching’ material (editorials, commentaries, book reviews, news and views items); variation in the quality of published articles (even within journals); and the need to develop ways of encouraging and supporting young scholars and novice researchers to read local journals and publish in them.

The panel also encountered a number of instances where its recommendations in relation to removal from the DHET list conflicted with the inclusion of the journals in other lists. “This source of conflict between recognised lists needs to be resolved with the DHET in the interest of clarity for authors and journals,” it said.

The review noted that the process had revealed a “variety of page and-or article charges, ranging from close to zero to over ZAR3,000 [US$173] per article”, which, it suggested, would “best be considered by the National Scholarly Editors’ Forum”.

DHET list

Among those journals the review said should not appear on the DHET accreditation list are: Perspectives in Education, Journal of Educational Studies and Per Linguam – A Journal of Language Learning.

The review found that the editor and editorial board of Perspectives in Education had been in office permanently since 2016 without competitive appointment. The quality of the articles was identified as “occasionally good but generally weak or extremely poor” with one reviewer commenting that it was more like a magazine than a scientific journal.

The panel was of the view that the journal should not continue to be listed on the DHET accredited list, irrespective of its inclusion in the IBSS and Scopus indices and should not be invited to the SciELO SA platform.

The Journal of Educational Studies was asked to reconsider the automatic appointment of the dean of the school of education as its editor-in-chief, while one reviewer expressed concerns about the “emerging researcher” standing of the editor-in-chief who did not appear to be well-published or an experienced researcher on the university’s website.

In the case of Per Linguam – A Journal of Language Learning, it was suggested that the editor-in-chief be appointed for a limited period and the appointment should follow a competitive process. Among other assessments, it was considered that the scope of the journal was too wide.

While noting the quality of its articles, the panel nonetheless suggested that Education as Change “should address the bad practice of associate editors publishing in their own journals”.

A periodical of the South African Education Research Association, the Journal of Education was criticised for having an editorial committee “almost exclusively” comprising academics from the University of KwaZulu-Natal and the Durban University of Technology. “This is concerning as it raises a conflict-of-interest risk and it seems that the journal does not have a policy regarding this,” the report states.

The South African Journal of Education, the official publication of the Education Association of South Africa, was commended for its articles “of a high standard and representing the best scholarly work in the fields of education and teacher education in South Africa”. However, it was recommended that the journal include a conflict-of-interest policy and a policy statement regarding errata.

The panel found that the Southern African Review of Education, which focuses on educational issues that have specific reference to educational policy, comparative education, sociology of education and history of education, could do more to harness articles from researchers outside South Africa and from the SADC (Southern African Development Community) region. There was also a need to recruit local young scholars onto the editorial board so as to transfer skills and ensure sustainability.

How much weight does the evaluation have?

Veldsman said evaluation reports are openly accessible and shared with all the relevant stakeholders, particularly with the DHET.

“Although it is the prerogative of the DHET to accept (or not) our recommendations, it does imply that a journal can either lose or retain its accreditation. Loss of accreditation means that the universities that employ authors of papers published in such a journal would receive no subsidy for the article,” she said.

But ASSAf’s process still has a bearing on the overall quality assurance of journals in the system. Editors are concerned about our recommendations, as they could have an impact on their accreditation, said Veldsman.

“Generally, we have found that these recommendations are welcomed by the editors as it assists them in determining the strategic direction of their journal. “In many cases, for the first time, they have an objective, external [peer] review of their journal,” she said.