League table comparisons drive ten years of increasing patient satisfaction with healthcare

In spite of financial crisis-induced austerity cutbacks, Europe’s healthcare systems keep producing better results, and patient choice and involvement are improving.

That is the - perhaps surprising - conclusion of the tenth edition of the European Health Consumer Index, published today (31 January).

However, it remains the case that too many countries stick to inefficient ways of funding and delivering care services. Copying the most successful European healthcare systems would save money that could be invested in saving lives and improving performance across the board, Arne Björnberg, the man behind the European Health Consumer Index, told Healthy Measures, as he reflected on ten years of comparing Europe’s national health systems.

Publishing outcomes has been good for patients and helped governments steal one another’s best policy ideas – but has drawn plenty of expert criticism.

“There was a big cultural problem in European public healthcare when it came to measuring what healthcare does, how much it does and how well it does it,” Björnberg said. “We wanted to change this weakness.”

This weakness prompted Björnberg to put together a pilot study comparing 12 countries’ health systems in 2005. Taking an unabashedly consumerist approach, his team compiled a league table, ranking countries according to waiting times, patient satisfaction, prevention, pharmaceuticals and, of course, outcomes.

“Instead of obsessing about inputs we have focused on outcomes so that patients can make an informed choice whenever possible,” he said. “Open benchmarking improves outcomes more than any guidelines ever have.”

After a pilot that successfully caught the attention of the media, ministers and academics, the first annual mega-survey was launched in 2006. It now covers 35 countries, ranking them from the Netherlands at number one, to Romania, 35th.

This year, Serbia and Macedonia come in for special mention thanks to their use of e-health to slash waiting lists. Despite good outcomes, Sweden, UK, Ireland and Poland are criticised for long waiting times.

Public criticism

The Index turns the performance of health systems into a horse race which can inspire policymakers to scrutinise areas for improvement. 

But, since its launch, there has been plenty of blowback against the transparency drive, which also features reports on specific fields, such as heart health, cancer and diabetes. “Doctors are like any professionals – they really hate if somebody publishes results showing they are doing worse than their peers,” Björnberg said.

Hospitals were not too thrilled either. Their fear was that a blunt comparison of survival rates or other indicators would give a false impression: the best hospitals and doctors often get the most complex cases where outcomes are generally worse. A middling doctor in a small regional clinic could have stellar survival rates by bumping complex cases to the nearest university hospital.

Governments and health service leaders have also been sceptical. Björnberg refers in the latest report to strong reactions from the Ministry of Health in Belgrade, which claimed that Serbia’s 2015 ranking was unfair. Ironically, feedback from Serbian health professionals suggested some scores were too generous.

The EHCI has also been in dispute with Irish health authorities for three years, rejecting official waiting time data because it contradicted less favourable reviews from patients. The spat bumped Ireland down to 21^st place overall due to its worst-in-class waiting lists. 

While league tables are widely seen as a valuable tool – OECD health ministers this month backed a new global league table of patient experiences – some critics have dismissed Björnberg’s rating system.

Arbitrary scoring system

The most stinging criticism came from public health professionals in the UK who claimed that the scoring method is arbitrary and should be ignored.

Björnberg, who compiles the report with a panel of experts, said it was with “joy and astonishment” that he read a blog post in the British Medical Journal early in 2016 saying the index did not have enough treatment indicators in its analysis. “We listened and have changed the weighting a little,” he said. “But it’s a big shift in 10 years to go from having no recognition to hearing calls for even greater measurement and transparency.”

So does the acknowledgement in the 2017 report that its findings are not of “dissertation quality” mean the Index plays fast and loose with the data? 

“What we do – and our friends at OECD and WHO cannot do – is we unashamedly compile data from different sources into a single indicator to arrive at what our expert panel feels is the least erroneous indicator of reality,” Björnberg said.

In some instances, hard data is difficult to source, so proxy measures are used. For example, when the panel was trying to rate Albania’s antibiotic-resistant superbug infection rates it had no official information with which to work.

“In the absence of data, we asked whether Albanians could buy antibiotics in pharmacies without a prescription,” said Björnberg. “The answer was ‘yes’ so we gave them a negative score because no country that allows over-the-counter sales of antibiotics is doing well on antimicrobial resistance.”

The index, he adds, is imperfect but has consistently pushed Europe towards an outcome-centred approach, making people think about how they deliver services.

Too proud to change

Björnberg also suggests that some of the flak from the UK is inspired by that country’s routinely mediocre performance.

While the UK edged out Slovenia to claim 15^th place, the report labels the National Health Service – and its “autocratic top-down management culture” – as one of the least accessible in Europe. It also features a section counting the ways in which ‘Bismarck’ health systems, such as the Netherlands, where insurers are separate from providers, beat the Beverage system beloved of the British public.

One of the biggest barriers to learning from peers is national pride, according to Björnberg. “People want to believe that their system is the best,” he said. “Big countries like Germany, France and the UK all think they have the best conceivable health system so they are less prepared to take ideas from others.”

Management deficiency disorder

One of the reasons the Index rubs policymakers up the wrong way is its general preference for the private sector. From Björnberg’s perspective, public health systems are a victim of the worst stereotypes of civil service inertia. “It’s an attitude difference,” he said. “In the private sector if you find out a competitor does the same thing as you, but does it better or cheaper, you want to find out how they do it and copy them.”

His prescription for the “management deficiency disorder” he diagnoses in many European health systems is to take a more consumer-focused approach. The suggestion that some hospitals could do with the help of budget airline Ryanair’s Michael O’Leary is surely tongue-in-cheek but reveals Björnberg’s faith in market-based solutions and hiring top managers.

The plainest example, he says, is the pace at which other countries have aped the Dutch model, which has consistently delivered greater efficiency. “The Dutch have a neat solution to a problem so instead of reinventing the wheel we hear people in Vienna saying ‘Oh that’s good for the Netherlands but it would never work in Austria.”

Yet the example of Serbia taking inspiration from Macedonia’s world-beating appointment-booking system demonstrates the value of copying the best performers. Macedonia rocketed up the index in 2013 thanks to an e-health tool that allows GPs to make appointments for their patients with hospital consultants, or to book slots on diagnostic scanners such as MRI and X-ray machines.

Serbia, which was disgruntled by its performance, responded by hiring the team behind Macedonia’s IT system, which duly rolled it out across the country. While the impact is less dramatic – Serbia is a bigger country than Macedonia so implementation cannot happen in a big bang – it has already paid dividends for many patients, resulting in significant improvement in Serbia’s ECHI ranking.

This, said Björnberg, is an example of how national pride can be leveraged to positive effect: naked jealousy and healthy local rivalries spark a race to the top. “Serbia sees itself as the big brother among ex-Yugoslav states so they licensed the Macedonian system and hired their team – now [its] waiting times are way down.”

End of the Index as we know it

The ECHI has achieved much in a short time - yet its future is uncertain. The 2016 edition is only a partial update due to a lack of money. Indeed, the 2015 table was clearly the template for this year’s installment, with updates to various indicators, where available. It may still be unloved in some quarters but with health outcome measures very much in fashion, the team behind the Index hope to attract supporters for a more comprehensive version next year.

The other big problem is a good one: European healthcare has improved to the point where the simple traffic light system Björnberg uses has become less informative. “There are too many greens” he said. “Even through the [financial] crisis there were improvements in fundamental areas. Take infant mortality, for example. At the beginning of this project over a decade ago there were nine green countries – now there are 24.”

Other measures have been rendered obsolete by technological progress. Where once just two countries could boast of having their pharmacopaedia available online, now all but five get a green score in this category. “We’ll either have to change the cut-offs for some measures or introduce more challenging indicators,” said Björnberg.

Despite its imperfections and the need to update the scoring criteria, the ECHI can claim to put the performance of health systems under the spotlight every year, inspiring interest in outcomes.

And as even critics must acknowledge, Björnberg and his colleagues were attempting to measure and report on health systems outcomes long before it went mainstream.