How innovative is the UK? Depends on how you count it

The UK government may be teetering, but to the world’s statisticians there’s an even more consequential problem: is the UK measuring its R&D correctly? The resulting row, while uniquely British, highlights the growing policy importance of R&D statistics globally.

On September 29, the UK Office for National Statistics stunned the number-crunching world by announcing that for some years it had been inadvertently undercounting the value of private sector research and development in the economy. It said a better count, to be finalised next month, would put that value at £43 billion for 2020 – up by £16.1 billion, or 62%, from its prior estimates.

Presto! As a result, in the click of a spreadsheet, the UK economy suddenly looks a lot more research-intensive than previously thought. It also hits the government’s target for investment in R&D as a percentage of GDP, and potentially pushes the UK several notches up in the world rankings for innovation.

Despite much eye-rolling, nobody in the rarified world of statisticians is suggesting the agency is fudging the numbers. ONS has a sterling reputation globally for conscientiousness and political independence, and it made extra effort to explain its rationale in multiple online posts foreshadowing the change.

But the timing has alarmed many in the policy world, coming just as the new chancellor, Jeremy Hunt, is searching for places to cut the state budget and stabilise the pound  - a search that will continue after the resignation today of Liz Truss as Prime Minister. Prominent targets for cutting could be business R&D tax credits and university funding.

“The concern is that – with more austerity in the cards – the government will say they’ve already hit the (R&D) target, so no more is needed,” says Diane Coyle, a public policy professor at Cambridge University. “This would be a mistake: the absolute number isn’t what matters, it’s the direction.”

Statistical ‘whataboutery’?

Some members of Parliament are also paying heed. “After a disastrous few weeks in British politics,… cutting incentives to spend on R&D would be shortsighted and counter-productive,” said Liberal Democrat MP Layla Moran, whose constituency is in Oxford. She says she trusts the ONS data, but not so the government’s possible response. “If the Treasury had any sense, they’d double down on R&D spend by increasing the target, not use statistical ‘whataboutery’ as an excuse to invest less.”

But the odd circumstances highlight a global phenomenon: statistics on R&D and innovation, formerly a recondite topic for economists and statisticians only, are now being sucked into political arguments in many countries. How much businesses invest in R&D, and the value of their resulting innovative products and services, feed into national measures of productivity and gross domestic product. “In the knowledge society, this number, R&D intensity, is relevant for a politician,” says Charles Edquist, a Lund University professor of innovation studies. “There’s a minister of research in every country, and that minister is happy when the R&D intensity goes up.”

There’s also real money attached to the numbers, at least indirectly. Edquist was involved in 2018 in a controversy over the way the European Commission calculates the innovativeness of its member states. In a Science|Business essay and elsewhere, he and University of Deusto innovation lecturer Jon Mikel Zabala argued that the Commission’s methodology overstates the performance of rich countries like his native Sweden, and understates the efficiency of smaller countries like Bulgaria in converting innovation to value. This came in the midst of a prolonged budget fight between rich and poor countries over how the Commission allocates R&D funding around the EU. (Edquist maintains the Commission still has its methodology wrong.)

Similar quarrels arise elsewhere. In Canada, universities have long used the country’s unimpressive R&D numbers to argue for more research funding. And in the US, as paranoia mounts about China’s growing technology might, R&D statistics often get pulled into Congressional debate to support both sides, pro and con, on budget, immigration, security and other hot-button issues.

‘Difficult to measure’

As R&D spending soars – globally, it accounts for more than $2 trillion a year – the whole system of measuring and comparing R&D and innovation has become an industry in its own right. “R&D has steadily become more economically important as a driver of productivity and growth, so the stakes are higher,” says Coyle. “But it is difficult to measure.”

The gatekeeper for these statistics is the Paris-based Organisation for Economic Cooperation and Development. With its 38 member-states, the OECD maintains a Bible – called the Frascati Manual – that details how a statistician should define research, innovation, the people who do it, and the money they spend on it. Once a year, the statistical agencies in most of the OECD member-states send their latest numbers to Paris, where they are analysed and incorporated into a global database. And periodically, the OECD updates the Frascati Manual to reflect changes in the way research and innovation happens around the world; the latest update was in 2015.

Revisions do happen in the OECD numbers, but nothing on the scale of the UK’s latest estimate; in recent memory, the biggest was on the order of 20%. The reasons vary: sometimes countries miscategorise EU R&D funding as their own, or undercount the number of small businesses in their country. In each case, the OECD can send the numbers back to the reporting statistical agency for another look. It is not obliged to incorporate a country’s numbers in its own global rankings.

Based on the old UK numbers, the OECD summaries show it was lagging far behind every other G7 economy except Canada in R&D spending – at just 1.76% of GDP in 2019, compared to the OECD average of 2.5% that year and 3.19% for Germany. The new UK numbers would push its standing up to 2.4%, by coincidence the very number the government set as a goal. That may sound impressive, but as a Bank of England adviser, Josh Martin, put it in UK news service The Wonkhe, “meeting the 2.4% target because we are now measuring it better doesn’t feel like a win: it feels like the target was wrong” and the government should be aiming higher still.

A methodology problem

While the scale of the change is huge, the nature of the UK’s methodological problem didn’t particularly surprise other statisticians. To get an estimate of how much the private sector spends on research – called Business Enterprise Research and Development or BERD - ONS works from two main sources. And for at least a few years they have been flashing contradictory signals.

One signal is from annual surveys of businesses that ONS aggregates. They ask a sample of 5,400 businesses throughout the UK how much R&D they are doing, of what type, and at what cost. The agency extrapolates from the survey what the economy-wide total might be, and for 2020 it had estimated BERD at £26.9 billion.

The second signals comes from the tax agency, His Majesty’s Revenue & Customs. The UK maintains an elaborate system of R&D tax breaks for corporate R&D – growing in recent years, as successive Tory governments bet on business to make the country an innovation powerhouse. Every year HMRC tallies how much tax relief companies claim against their investment in R&D. For the year ended 31 March 2021, the tax relief totaled £6.6 billion – suggesting an underlying, total R&D spend of £38.1 billion.

The gap between the two sources, 42%, was huge. There are lots of possible reasons. The HMRC numbers come from actual tax returns, where R&D claims may have been inflated; the government’s National Audit Office has dinged the tax agency in the past for not policing R&D claims more closely, citing “error and fraud” rates of as much as 7.3% for small businesses claiming the tax break. By contrast, the ONS survey samples firms from different databases than HMRC, and those databases appear to have been missing a lot of small business that do research.

In short, the HMRC figures may be too high, and the ONS numbers too low. For its part, ONS began studying the problem at the start of this year, according to Heather Bovill, deputy director for surveys and economic indicators at ONS. The work began “through looking at the data, and seeing that our data series are divergent: ‘what’s going on there?’” What’s more, the survey methods date back to the 1980s. “The way that we do the sampling for the survey for R&D is not as we would have done if we were setting up the survey now,” said Bovill.

As a result, ONS decided to revise its BERD estimate – first publishing the reason, and next month the final numbers. Then next February, it plans to send out new surveys with an updated methodology, to reflect the fact that more SMEs do R&D today than in the past. The results are due to be published in November 2023.

Open questions

Still unanswered is what HMRC will do about its own numbers. Zabala notes that the BERD numbers started looking unusual from 2015, around the time the UK government sweetened its business R&D tax credits. It’s the generous tax scheme that pushed up the HMRC numbers, he says, “and not because British companies are more inclined towards R&D investments.”

An HMRC spokeswoman, responding to Science|Business questions, said, the tax authority “welcomes” the ONS revision, as it “changes our understanding of the relative position of R&D expenditure in the UK compared to other countries.” On fraud, she added, “We have prevented significant organised criminal attacks and fraudulent abuse of the R&D tax reliefs and are continuing to step-up our monitoring of claims for fraud. Claims are checked for suspected fraud and since April 2022, more than 1,600 claimants have been asked for more information to validate their claim. More than 80% of these claims have not been paid out as a result of our checks. R&D tax reliefs are being reformed and these changes will help further reduce abuse.”

From an OECD spokesman, there was “no comment” about the UK situation. Generally, he said, "member countries and partner economies occasionally make revisions to the R&D data they submit to the OECD. When that happens, the OECD team in charge reviews the revised data, and may request additional details, before the data is incorporated in the OECD’s R&D databases." The first hint of an answer may come next March, when the OECD is due to release its next annual numbers.