First research ‘scorecard’ for G20 nations

The Web of Science released a global research report during the 2019 G20 summit held in Osaka, Japan, in late June. It is the first in a projected annual collection of research health ‘scorecards’ for G20 nations that produce 65% of world publications, spend 92% of research funding, employ 87% of the world’s researchers and publish 70% of global research papers.

The Annual G20 Scorecard – Research Performance 2019 was produced by the Institute for Scientific Information or ISI of the Web of Science Group, and features comparable country ‘scorecards’ that provide trend data on “matters salient to a healthy research enterprise”.

It is not a ranking. Rather the study looks at the research workforce and research collaboration, output, quality and competitiveness to produce profiles for G20 countries – 19 nations, with the study excluding G20 member the European Union.

Along with trade, it says, “an essential determinant of prosperity and advancement is a country’s research base and the innovation it fosters”. The 19 countries in the G20 accounted for more than five million articles and reviews indexed in the Web of Science research publication and citation index for the last three years – more than 70% of the global total.

The G20

The 2019 G20 Summit took place in Osaka, Japan, on 28-29 June. The G20 is a group of 19 leading economies spread around the globe that represents more than 80% of the world’s gross domestic product or GDP and two-thirds of global population.

“What happens in the G20 affects the world and the G20 group is undoubtedly a driver in the global research system,” writes Professor Jonathan Adams, director of the ISI and a visiting professor at King’s College London in the United Kingdom. He co-authored the report with Dr Martin Szomszor, head of research analytics at the ISI, and Gordon Rogers, an ISI senior data scientist.

The G20 meeting in Japan, the report points out, was held “against a backdrop of increasing global trade tensions. Expectations are high that the G20 will continue to play a positive role in fostering economic opportunity and addressing challenges in the global landscape.

“The world’s most prosperous economies are also among the most innovative ones and innovation is driven first and foremost by research.”

Country scorecards

In constructing country research profiles, the ISI says it chose topics of policy interest that are good signals of the health of the research base.

Key factors that contribute to impactful research include: relative research funding within the wider economy, gender balance in the research workforce, international engagement – “international collaboration accounts for about half of G20 output and produces some of the most impactful research” – open access to research, publication output and citation impact.

ISI argues that ranking countries would not be meaningful. The research profiles cover some large and mature economies and others that are smaller or still building a research base, and time trend graphs show the progress of research development.

“These scorecards will help policy-makers, observers and reporters to track, applaud and critique that progress, including the diversification of the research workforce, the allocation of strong research funding and the shift to open access,” says the report.

Each country scorecard contains concise data visualisations across two pages, offering a uniform national scorecard for the G20 members – along with a summary section discussing additional details – covering a 10-year window. The report provides the following summary of each country’s scorecard:

Argentina
Gross expenditure on research and development, or GERD, remains low at 0.53% of GDP, and patenting is also relatively low, but while the research workforce is relatively small it is gender-balanced (53% female). Output per researcher is half the G20 average. Publication via open access models is relatively high, especially in social sciences and humanities. The citation impact of publications is boosted by international collaboration: 11% of these papers are in the global top 10%.

Australia
Output has doubled in a decade – and is relatively high per researcher and per GERD – but it is levelling-out and purely domestic output is dropping. Overall citation impact is above the G20 average, boosted by international collaboration where 17.5% of papers are in the global top 10%. Open access is around the G20 average in science but well below this in other areas.

Brazil
Open access has been a strong feature of output across all disciplines. The absence of key OECD data for Brazil means some other indicators are absent. Output growth has been steady, driven as much by domestic activity as by collaboration and is strongest in life sciences. As elsewhere, collaboration is a significant aspect of citation impact, which is otherwise below the G20 average.

Canada
Output has risen by around 30% in a decade, and is strong in social sciences and health but, despite above average productivity, domestic output has been flat and is dropping. Citation impact is relatively good with 38% of publications above the world average citation impact compared to 31% for the G20 as a whole. It is strongest in health and natural sciences.

China
There is an enormous research workforce (two million researchers), a strong level of investment (GERD as 2% of GDP) and relatively high patenting rates. Its output productivity is domestically driven, although below the G20 average, and is particularly strong in science and technology. Average citation impact is rising and has reached the G20 average, and the Impact Profile shows that 10% of China’s papers are in the global top 10%.

France
Only 27% of researchers are female, open access is below the G20 average in most fields, and output both per researcher and per GERD funding is below average. Citation impact is relatively good in science, boosted by a high level of international collaboration, now accounting for over 60% of all output while domestic output has been in decline: output per researcher has been static.

Germany
Investment is high with GERD at 3% of GDP, and output per GERD is below the G20 average whereas output per researcher is around average. The citation impact of this research is relatively good, across both international and domestic activity, and secures a 12.6% share of the world’s top 10% of papers. Only 28% of researchers are female. Open access has been below the G20 average but is rising.

India
There is no readily available recent data on GERD for researchers, so the ISI could not index productivity, but volume output appears to be relatively low for such a large economy. International collaboration remains relatively low level and open access has been adopted only in bio-medicine. These factors all contribute to a relatively weak Impact Profile and low citation impact across all areas.

Indonesia
Research output is small but has trebled in a decade, across all disciplines. More than 75% of this output is internationally collaborative, so that also influences the Impact Profile. Average citation impact is relatively good in medicine and in social science and humanities, but domestic impact remains relatively low and over 40% of domestic papers are uncited. Open access take up is good, but this is driven by very high open access in medicine.

Italy
Citation impact is relatively good in all areas, but output is only just above the G20 median, which is surprising for a G7 research economy. The Impact Profile shows that average performance is boosted by international collaboration, which accounts for 55% of total output. Productivity figures (per GERD and per researcher) are well above average, so output is not constrained by consistently low investment.

Japan
Citation impact is relatively low for a well-established research economy with a high level of GERD/GDP (3.2%). The Impact Profile shows that performance is lifted above the G20 average through international collaboration, although this is relatively low, at 30% of a total output that has remained very flat over the decade. Productivity is well below average and only 16% of the researcher population is female.

Mexico
Research investment remains relatively low – GERD/GDP is 0.49% – but rising output is boosted by high researcher productivity. However, citation impact has been flat over the decade and the Impact Profile shows that the performance of the domestic base remains below the world average and overall impact has recently fallen compared to that benchmark. Only in medicine and health does impact match the world average.

Russia
Output has been recovering slowly from post-Soviet disruption and is close to the G20 median in physical sciences but still low in engineering. Citation impact is boosted by rising international collaboration (38% of total) but share of global top 10% papers is low (4.2%) and the Impact Profile shows that domestic research is generally cited less than the world average. Productivity by GERD and researcher is relatively low, and open access is exceptionally low.

Saudi Arabia
Average citation impact appears very high as only 20% of papers are domestic, and the bulk of the Impact Profile reflects a policy orientation towards international collaboration, or affiliation, that has grown steeply while domestic capacity has remained relatively static. Impact by discipline is consequently relatively high in all areas, compared both to the G20 average and to the country’s substantive activity.

South Africa
The overall Impact Profile is close to the G20 average, boosted by international collaboration (60% of output) that underpins a strong performance in medicine and health research. Productivity is high and output has been rising, although it remains relatively low in the G20. Open access publication rates across disciplines are consistently high and female researchers make up an impressive 45% of the workforce.

South Korea
GERD/GDP (4.5%) is exceptionally high. This has not yet been translated into output, where productivity is well below G20 averages, or citation impact. The latter may partly be due to lower international collaboration than the G20 average. Female researchers are a relatively small part (20%) of the workforce. Open access is rising and is already above G20 benchmarks.

Turkey
Output has recently declined and productivity by GERD and researcher has fallen throughout the decade. Citation impact has remained well below the world average in all areas, boosted by international collaboration which remains low at 25% total output. Output is also relatively low across all areas, given the size of the economy and open access is declining relative to the G20 generally.

United Kingdom
The share of papers in the global top 10% (14.5%) is the highest in the G20 and the Impact Profile shows that the domestic research base is also well above group average. By discipline, average citation impact is relatively high (above 1.18) in most areas and only low in art and design. International collaboration is exceptionally high (63%) for such a large economy. The number of female researchers (39%) is above the G20 average.

United States
Citation impact is consistent, though less than the UK, across all disciplines. The Impact Profile shows that the domestic research base is boosted less by international collaboration than for other countries. However, although GERD/GDP (2.8%) remains high, output and impact are in decline and output per researcher has fallen below the G20 average. Publishing via open access models is on the rise but is below the G20 average.

Overview

In future reports, the ISI expects to add information that tracks the evolving state of each country, benchmarking its activity against its historical position.

In sum, writes Jonathan Adams, the study probes key questions relevant to the health of a research enterprise: “The answers hold clues to which nations will likely be the winners and losers in the crucial process of translating research into innovation and, subsequently, into national prosperity and security.”