While there may be a creeping acceptance that measuring outcomes and not activity is the way to improve the effectiveness of Europe’s healthcare systems, “the concept is all very new” and the definition of a good outcome and what measurements to make is up in the air, said Sudhesh Kumar, Dean of Warwick University Medical School.
“Setting health outcomes for disease prevention is particularly difficult,” he told delegates at the Science|Business event, ‘The Data Challenge: How and why do we measure patient outcomes?’
Although as yet we are in the foothills of applying outcome measurements to make healthcare systems sustainable, one obvious starting point from which to develop methodologies is to make comparisons across institutions and investigate the causes of variation, setting a path for the worst to catch up with better performing peers.
Annemarie Haverhals, Programme leader for value-based care at Santeon, a group of seven hospitals in the Netherlands that since 2010 has been collaborating to improve patient care, described work to assess the best cancer outcomes. The 3,000 key performance indicators used by the group were shown to be inadequate when a 2012 investigation of outcomes in lung cancer highlighted variations. As a result, “We stopped treating [lung cancer] patients in one hospital,” Haverhals said.
Buoyed by the success of the initial project, in 2015 Santeon stated looking at outcomes in general, setting up teams of ten nurses, doctors and patients for each specialism, with a brief to define outcomes and use them to drive improvements. The data on outcomes is cross-referenced to the amount of resources used and how long the treatment takes.
“Collecting data is a hassle, but we manage to do it. We can see where there are differences in outcomes, costs and treatments. We see a huge variation with a similar patient population,” said Haverhals. This is the spur to determine the causes, and that discussion is underway currently in stroke and cancer. Later in 2017 a similar exercise will start in five more conditions.
The improvement teams meet each month to review the latest data. This is in addition to their regular duties, but Haverhals said, “They want to improve care, so they are motivated.”
As yet, there is no shared technology platform between the Santeon hospitals for collecting and analysing data. However, there are thorough criteria for the selection of patients included in the analyses and each hospital has to validate its data.
Burden of measuring
The “hassle” of collecting outcomes data should not merely be added to the huge burden of measuring that exists already in the National Health Service (NHS) in the UK, said Kiran Patel, Medical Director, NHS England. At its most basic, patients want a better quality of life and to live longer. “It’s simple, so don’t over-complicate it,” he said.
As advocates for patients, clinicians are in favour of outcomes measurements. They are also the lynchpin in applying them to improve healthcare systems. “They are custodians of value and decision-making: 70 per cent of expenditure is directed by clinicians. What they want are measurable, real-time data,” Patel said. This should go down to the level of individual clinicians, to generate peer pressure to improve.
Managers and leaders in health systems who are responsible and accountable for quality, finance and performance require outcomes measures too. “[They] should have low tolerance of variation,” said Patel. He cited two programmes in the NHS using legacy data to drive improvements. ‘Getting it right first time’, targeting physicians, aims to address the fact that recovery time following an operation is four times longer in some areas of the country than others. Meanwhile, ‘Patient Market’ uses payer data to show which payers get the best return on investment.
Politicians also, should have more regard for patient outcomes, not only when thinking about healthcare funding, but also in relation to the contribution that health gains in the population as a whole makes to the health of the economy. In addition, outcomes data puts the spotlight on health inequalities and provides tools to address them.
“We need to measure, but we have to measure the right things so we don’t overburden ourselves,” said Patel. That points to the core difficulties – of stopping measuring things that are not important and ensuring people get feedback from the data they submit. Currently, healthcare systems are assessed in terms of activity. To make progress people and systems must be incentivised to focus on outcomes. “We need to transition from paying for activity to paying for outcomes, and we need to empower physicians and organisations to define what these outcomes are,” said Patel.
Process versus outcomes
For Richard Lilford, Pro-Dean for Research at Warwick University Medical School, it is not patient outcomes data per se that matter, but how this is converted into information. The difficulties lie in detecting the signal in all the distorting noise. Death rates for example, might be considered a good proxy for the performance of a hospital, yet only 3 – 4 per cent of people die in hospital, of which only 5 per cent die as a result of their care there. As an outcomes measure death rates have “no sensitivity” Lilford said.
Even with risk adjustment the signal may still be drowned out, especially if different hospitals measure things differently. Sometimes, the signal emerges despite the noise, with one case in point being a disparity in outcomes in cardiac surgery in the UK. “The differences were so wide, it had to be down to the treatment,” said Lilford. Patient-reported outcomes in prostate cancer and success rates in paediatric surgery are two other areas where outcomes are clear; in most other fields there is too much signal to noise.
Lilford believes the most valid outcome is patient perception of care – though even that is not perfect, he said. “Expectations vary, some are harder to please than others. Older people are easier to please – so you have to have your eyes open.”
Taken overall, the shortcomings - as he sees them - with outcomes, lead Lilford to propose the primary focus should be on process. Not that this is a straightforward way of improving care, especially in the case of multi-morbid patients, for whom it is impossible to get all processes right and set out a single guideline for every possible combination of treatments.
Focussing on process is not about “going data-free”, but conducting clinical research to validate processes. “If you have done clinical research to [prove] a certain process leads to a certain outcome [….] forget about the outcomes, just look at the process,” Lilford said.
Haverhals countered that outcomes data are the starting point for discussion. “If outcomes vary, then you go and look at processes to [define] what is best,” she said. Patel agreed outcomes must be coupled to processes and said that in addition it is important to set out how changes will be evaluated to see if they are effective. The NHS in England is in the process of drawing up Sustainability and Transformation Plans, which are multi-year plans built around the needs of local populations to help drive a genuine and sustainable transformation in health and care outcomes. For Patel the question is, “How will they be assessed?”
It is important to recognise that equality of outcomes will not result from treating everyone the same. Different ethnic groups have different susceptibilities to diabetes, for example. “South Asian populations need more screening to ensure equity of outcomes,” said Patel. “Equality is not sameness.”
Whether outcomes or processes are given primacy, neither is a reflection of the quality of healthcare, the measurement of which, “is not an easy business”, as Lilford noted. Longer life expectancy does not reflect quality of life, with elderly people often suffering a lot in their last years. From this perspective, healthcare systems are spending money on things that patients do not perceive as providing benefits.
How to value treatments
Faced with the imperative to reconcile growing needs with finite resources, European healthcare systems have turned to health technology assessment (HTA) as a means to put a value on specific treatments. In England, the National Institute for Health and Care Excellence (NICE) assesses the value of treatments and drugs on the basis of quality-adjusted life years (Qaly), a generic measure of disease burden, including both the quality and the quantity of life lived. One Qaly equates to one year in perfect health. In most cases, NICE will recommend that a drug or treatment is reimbursed by the NHS if the Qaly is £30,000 or less.
The arrival of personalised medicines and smaller patient groups in phase III clinical trials, most notably in cancer, means it is increasingly difficult for NICE to measure the value of drugs as they arrive on the market, said Mohit Misra, Chief Medical Officer and VP of Health Systems at Aetion, a company that provides analytics for assessing the value of drugs, devices and treatments, who is also a member of NICE’s HTA Assessment committee.
In addition, there is a divergence between what NICE considers to constitute value – putting the focus on overall survival - whereas for patients progression-free survival is preferable (in cancer treatment, drugs that prolong progression-free survival may not increase length of overall survival). In order to expedite access to new cancer drugs, NICE is moving to approving reimbursement but then reviewing after 18 months to assess if therapies are effective in the real world. “We have to use patient data more [….] A lot of the time, there is not enough evidence, so we have to measure outcomes data,” Misra said. “NICE needs to change the way it operates now there are no massive phase III trials.”
In summary, Kumar, said there are clearly different classes of stakeholders with an interest in patient outcomes, creating tensions in moving to deploy outcomes measures. Compounding this, there is a gap between the very best outcomes and what healthcare systems can afford, leading to “tough choices”. Even when it is obvious that one approach is better than another, effecting change is hard. Furthermore, there is a balance to be struck between outcomes -which for some is a reductionist and blunt instrument - and looking to the process as the source of improvements.
Receive the Funding Newswire each Tuesday, our Policy Bulletin each Thursday, and news about bridging Europe’s east-west innovation gap twice a month in The Widening. 
