This article explores how the intelligent application of information technology can help NHS Wales achieve better outcomes for patients and improve the experience of people receiving care. Well-structured IT systems that share information can reduce complexities in the system, speed up treatment processes and enable more people to receive treatment in non-hospital environments.
IT is essential in assessing quality, through measuring levels of harm, to tracking patient outcomes and thereby ascertaining whether treatments have delivered the desired outcomes and provided value for money. This should include patient input in a ‘co-productive’ approach. The article includes five key considerations for creating an IT infrastructure that delivers all these benefits.
We need to measure outcomes
Prudent healthcare places a greater value on patient outcomes; rather than the volume of activity or procedures delivered, it aims to rebalance the NHS and create a patient-centred system.
To deliver this vision, the NHS in Wales needs to re-orientate itself to monitor and act on the intelligence gathered through the measurement of patient outcomes.
Outcomes are the experiences of patients receiving treatments. They include survival, quality of life, ability to participate in society, symptoms, side effects, and future health and social care service needs.
Measuring outcomes requires a longitudinal perspective. We need to link information about treatments and initial health status, to a variety of outcomes. This information can be collected through routine data sources and patient experiences over the short, medium and long term.
Value-based healthcare needs us to consider improvement by assessing outcomes relative to the resource costs involved. Sometimes we have access to this information from existing scientific studies but more often than not, we know far too little. We must pay more attention to the measurement of outcomes.
Wales could become a world leader in this field. We could build on our acknowledged success in privacy protecting data linkage for research. We could adapt international best practice in measuring patient-reported outcomes and we could involve patients to a greater extent in the co-production of knowledge.
Information on outcomes will drive prudent healthcare
The NHS is a large and complex organisation, which has grown out of all recognition since its creation by Aneurin Bevan in 1948. We now face significant challenges in meeting the healthcare needs of a growing, ageing population in a financially constrained environment. We all welcome the improvements in our living standards, marked reduction in smoking and increasing access to a far wider range of treatments that have led to dramatic improvements in life expectancy over the last 30 years. But these achievements have come with a legacy and a large proportion of older people now living with multiple chronic conditions.[2,3]
The NHS is under pressure and needs to adapt to deliver services based on the principles of prudent healthcare. This will demand a shift in emphasis from activity and waiting time metrics to patient outcomes and experiences. We need to make sure both activity and outcomes are equally valued and we used them to inform our decisions, as patients, clinicians, health boards and NHS trusts.
The measurement and aggregation of individual-level outcomes will be key to undertaking high-quality evaluation of interventions and services, whether the focus is prevention, treatment or rehabilitation.
Which outcomes are important?
We have a wide variety of outcome measurements available to us. These measures reflect the interests of patients, health and social care providers, commissioners of services and the Welsh Government. We can still do more to collect and interpret these kinds of data. An essential component of a responsive 21st century service will be the ability to feed back meaningful information about the experiences of groups of patients and patterns of service and then use it to elevate the voice of patients and the wider public in the decision-making process.
There are many groups involved in defining and measuring outcomes around the world, including different parts of the NHS across the UK. We cannot review all these activities in this short chapter, but will point out a number of salient developments.
For example, in England, the NHS Outcomes Framework (2013–14) focuses on five domains:
- preventing people from dying prematurely
- enhancing quality of life for people with long-term conditions
- helping people to recover from episodes of ill health or injury
- ensuring people have a positive experience of care
- caring for people in a safe environment and protecting them from avoidable harm.
Patient-reported outcome measures (PROMs), such as general health or diseases specific quality of life measures, are used to measure health gain in four groups of patients – those undergoing hip replacement, knee replacement, varicose vein, and groin hernia surgery and are based on responses to questionnaires before and after surgery. The results reveal interesting variations in outcomes.
Use of Patient Reported Outcome Measures
In Victoria, Australia, PROMs have been used with health service and mortality data to evaluate the impact of changes in trauma services on population health. A key aspect of this research was a detailed investigation into the suitability and efficiency of using different quality of life measures in trauma populations. Trauma populations have a preponderance of young males, who we know are least likely to fill in surveys or questionnaires.
These early research findings informed the decision to adopt a low-cost telephone follow up system in trauma services using simple PROMs. An evaluation showed their introduction on an integrated trauma service was a prudent decision. Following reorganisation of the service, researchers found the improved survival of seriously-injured patients was also accompanied by substantial reductions in overall treatment costs.
This study highlights the benefits of using multiple short- and longer-term patient and service-orientated outcome measures to quantify the impact of service configurations and determine whether these represent good societal value.
Testing feasibility of outcome measures in Wales
Building on the reported experiences of Australian trauma services, the Welsh National Orthopaedic Innovation and Delivery Board commissioned a feasibility study of an Orthopaedic and Trauma Surveillance and Outcomes System in 2012.
The feasibility study was based around knee replacement surgery and was a collaboration between Cardiff and Vale University Health Board, Public Health Wales, and Cardiff and Swansea universities. The feasibility work showed that collecting PROMS using the Australian model was acceptable to patients. The research also found that even though they were not asked directly their opinions on the success of the surgery, patients often commented on this aspect.
The study findings suggest that adding a simple question to PROMS, such as: “How much would you recommend this procedure to somebody else considering surgery?” could be very informative. The study also showed how linking data already collected by the NHS across different systems could be used to provide useful information on both the patterns of health service use and patient outcomes after surgery, including complication and surgery revision rates.
One of the outputs of this work is shown below. It is a simple graph, showing variations in one outcome, the percentage of patients requiring revision surgery up to five years following a knee replacement by different age groups in Wales.
We can see from this graph that revision rates are substantially higher in younger age groups. Around 10 per cent of the younger age group had revision surgery within five years of the original knee replacement. There are a number of underlying reasons for this, including the use of different prostheses.
This type of information showing patterns of a variety of expected outcomes could be presented to individuals considering surgery. We could routinely produce this kind of data and use it to inform patients and service providers to assist with informed decision making.
International outcomes measurement
The International Consortium for Health Outcomes Measurement (ICHOM) is a non-profit organisation founded by three international institutions. Its aim is transforming healthcare systems worldwide by measuring and reporting patient outcomes in a standardised way.
Its philosophy and methods of working are outlined on its website, where it explains the importance of outcomes:
“We believe outcomes are the ultimate measure of success in healthcare. When seeking treatment, patients want to know what their life will be like after treatment: Will I return to work, will I be able to take care of myself, and will my symptoms improve? Helping patients answer these questions is why we formed ICHOM.”
ICHOM organises global teams of healthcare leaders, outcomes researchers and patient advocates to work together to define standard sets of outcomes for a medical condition. It then promotes adoption of the outcome measure to enable healthcare providers globally to compare, learn, and improve.
Whilst the approach of ICHOM has grown out of competitive private health insurance systems we believe there is much that NHS Wales could learn from their approach.
One issue which can be easily overlooked in attempts to define comprehensive sets of outcome measures is the amount of resource needed to collect this kind of data. It’s tempting to believe we need a large number of outcome measures that cover every eventuality. While this is possible is sometimes adopted in well funded research studies, in general it would not be prudent to take this kind of approach in routine NHS practice.
As is often stated, perfection should not be the enemy of the good. Some quality of life and other outcome measures lend themselves to be easily used, are accessible through a variety of different media, take little time to complete, and can be automatically analysed to provide timely data. These are the measures that are more suitable for large-scale, cost-efficient use in the NHS.
We need to remember that collecting more information is not our objective, driving improved patient outcomes and better decision making, through efficient 21st century informatics is.
Quality assurance of outcomes using routinely collected data
In the NHS in Wales we already collect vast amounts of information on individual patients and services, but have not yet used all the value this information can offer, in planning and evaluating our services. Extracting value will not be easy. It is generally very difficult to fix problems without properly understanding them or without the information we need to identify promising approaches and without evaluating the impact of approaches we have tested. Guess work and reliance on simple conceptual logic models to identify solutions usually fail in complex systems such as the NHS.
Until the development of the prudent healthcare concept, assessment of NHS performance was largely based on aggregate process metrics, such as numbers seen and waiting times. Prudent offers a new and welcome focus on developing an outcomes-orientated service with much greater patient and clinician involvement.
We know from experience it is hard to engage clinicians or have meaningful conversations with the public, on the need for changes to services, unless we can put improved patient outcomes at the heart of these discussions. Both clinical staff and the public need access to information on outcomes to be able to judge proposals for service reconfigurations. We know this data has been in short supply.
Hospital mortality rates
An approach used by the NHS in both England and Wales to assess hospital performance, has been to outsource the monitoring of hospital death rate analyses. There has been concern about some of the results, including the apparently high in-hospital death rates in a number of Welsh hospitals.
Professor Stephen Palmer, from Cardiff University, was asked to undertake an independent review by Health and Social Services Minister, Mark Drakeford AM, into the way NHS Wales collects and uses risk adjusted mortality data (RAMI) to describe mortality in Welsh hospitals.
In his review for the Minister, published in July 2014, he concluded RAMI is not a meaningful measure of hospital quality. In fact RAMI may be misleading and potentially diverts attention from more meaningful approaches to measuring and improving hospital care.
The report also highlighted there was no epidemiological centre, with overall responsibility for the development and evaluation of mortality and other outcome measures that could advise and support health boards and Welsh Government.
The report recommended that a centre of national epidemiological expertise for Wales be set up. This would:“draw on the expertise in universities in Wales, including the Farr Institute of Health Informatics Research. Such a facility would offer NHS Wales an opportunity to move towards international leadership in the area of population based clinical outcome measurement.”
We hope this recommendation is acted on, as such a centre could also offer further benefits to the NHS in Wales. It could review and share best practise from international experience in this field, particularly the QUEST (Quest for Quality and Improved Performance) collaborative in the US.
This collaborative has demonstrated the benefits of doing it differently – by using a clear strategy for the design and use of measures: 157 hospitals in the QUEST collaborative broke through national trends in cost, safety and quality over a five year period.[10,11]
A way forward
Many important outcomes can already be routinely obtained through linking data collected by the NHS or data made available to it by other bodies, for example mortality information from the Office of National Statistics.
Wales is already a world leader in the field of privacy protecting data linkage for research purposes through the Secure Anonymised Information Linkage (SAIL) system, created by the NISCHR-funded Health Information Research Unit at Swansea University.[12,13] Both privacy protection and public engagement were built into the heart of research conducted through SAIL and, as a result, Wales has avoided the public concerns around systems elsewhere.
Secure Anonymised Information Linkage – SAIL
From a research perspective, SAIL has been a tremendous success for Wales, resulting in three Research Council UK centres of excellence in informatics being based at Swansea University since 2012. Together these have brought in more than £20m of investment into the local economy.
The NHS is also starting to benefit directly from this expertise and investment. Some of the £5 million Medical Research Council’s investment in the Farr Institute of Health Informatics at Swansea has been used to create state-of-the-art privacy protecting data linkage tools for every health board in Wales.
These are due to be deployed in autumn 2014 and will enable health boards to anonymise and link multiple datasets within their organisations and this will give them the ability to produce local reports to assist with prudent healthcare decision making. The MRC has provided the capital investment for these but for them to be fully used, health boards will need to ensure that informatics staff are deployed.
Health boards currently do not have access to all of the datasets and tools needed to carry out analysis to inform the development and quality assurance of their services. Discussions at the health board clinical outcomes steering group revealed it was not possible to accurately measure 30-day or longer-term survival to understand the impact of new services.
The following analysis of data within the SAIL databank was undertaken at the request of the medical directors’ group to show how survival can easily be measured using linked data. The graph shows survival over one year following an admission to hospital with a stroke in Wales, by the two major categories of stroke.
In future, health boards should be able to produce analyses such as these and evaluate the impact of service developments designed to improve care.
Role of clinicians
Clinician involvement is crucial in the design and delivery of information for outcome-orientated services. Health boards will need to invest in training a cadre of clinicians in data science and outcome measurement, to support the work of Prudent Healthcare.
Each board should have a senior clinical information officer responsible for the implementation of outcomes monitoring, reporting directly to the board. All health professionals will need to become more literate in health informatics and how they can use data to make better decisions.
Skills shortage is an issue. There is a world shortage of expertise in manipulating and interrogating large and complex datasets. We are fortunate NHS Wales has a considerable number of informatics staff working in a variety of organisations, such as the NHS Wales Informatics Service and Public Health Wales, as well as the health boards. But although there are many informatics staff, there are still not enough.
Many of our informatics staff have high level skills in traditional informatics, statistics and epidemiology, but few have yet been trained in the analysis of complex linked datasets – the essential skill set for 21st century. Public Health Wales has seconded a number of staff to work with the Farr Institute to develop these skills. There will be scope to include up to 30 NHS staff in the new data science building at Swansea University in 2015 to expand capacity in this field.
In the immediate term, it will be difficult to meet the demand for high-quality informative analyses. It is imperative we use all available skills in supporting the considerable informatics needs of prudent healthcare. To sustain our effort, NHS Wales will also need to begin training a much larger cadre of individuals.
We have seen a notable change in the degree of public engagement in NHS research over recent years. The public is involved in multiple aspects of health informatics research, including:
- contributing to strategic groups deciding on research priorities
- assessment and approval of any proposed research studies
- active involvement in the conduct of research projects.
Involving people in informatics research in these ways has had tremendous benefits. It makes sure the research we undertake addresses relevant issues and is acceptable to the public. A similar approach should be adopted by the NHS in Wales to engage the public in all aspects of outcome measurement.
Five practical steps to move from debate to action:
- NHS Wales should make the measurement and publication of outcomes a Tier One requirement for health boards and NHS trusts, equivalent to activity and financial targets.
- Every health board and NHS trust should have a prominent section on its website where patient and population health outcomes are published in an easy-to-understand language, perhaps under a “How are we doing? Patient and population outcomes” heading. This should include the degree to which organisations participate in mandatory national audits about the quality of care and a commentary on the interpretation of results.
- NHS Wales should adopt and implement the efficient collection of standardised patient-reported outcomes and experiences.
- NHS Wales should build on its involvement with academia and use the expertise and facilities provided by the Secure Anonymous Information Linkage (SAIL) system and recent investment by the MRC in the Farr Institute of Health Informatics Research to design, implement and integrate standardised measurement of health outcomes.
- Clinician involvement is crucial in the design and delivery of an information driven and outcomes-orientated service. Health boards and NHS trusts should invest in training a cadre of clinicians in data science and outcome measurement. Each organisation should have a senior clinical information officer responsible for the implementation of outcomes monitoring, reporting directly to the board.
Angharad Walters and Daniel Thayer, Swansea University, for the production of graphs used in this article.
- Drakeford M. Ministerial statement: Prudent health care: essential new approach for NHS Wales, 16/01/2014 [Internet]. [cited 2014 September 3]; Available at: http://wales.gov.uk/newsroom/healthandsocialcare/2014/140116prudent1/?lang=en
- Murray CJ et al. United Kingdom performance on health: implications from the Global Burden of Diseases Study. The Lancet 2013;381:997-1020.
- Barnett K, Mercer SW, Norbury M, Watt G, Wyke S, Guthrie B. Epidemiology of multimorbidity and implications for health care, research, and medical education: a cross-sectional study. The Lancet 2012; 380:37-43. doi: 10.1016/S0140-6736(12)60240-2.
- NHS Outcomes Framework 2013–2014 [Internet]. [cited 2014 September 3]; Available at; https://www.gov.uk/government/publications/nhs-outcomes-framework-2013-to-2014
- Provisional Patient Reported Outcome Measures (PROMs) in England – April 2013 to February 2014 [Internet]. [cited 2014 September 3]; Available at: http://www.hscic.gov.uk/catalogue/PUB14314.
- Gabbe BJ, Lyons RA, Fitzgerald M, Judson R, Richardson J, Cameron Reduced population burden of road transport-related major trauma following introduction of an inclusive trauma system. Ann Surg 2014;00:1-8.
- Griffiths S et al. Orthopaedic and trauma surveillance and outcomes system. Update Report to the National Orthopaedic Innovation and Delivery Board, March 2013
- International Consortium for Health Outcomes Measurement (ICHOM) [Internet] 2014 [cited 2014 September 3]; Available at: https://www.ichom.org/.
- Palmer SR. An independent review looking into the way NHS Wales collects and uses data to describe mortality in Welsh hospitals [Internet]. 2014 July16. [cited 2014 September 3]; Available at: http://wales.gov.uk/topics/health/publications/health/reports/mortality-data/?lang=en
- Meyer GS, Nelson EC, Pryor DB, James B, Swensen SJ, Kaplan GS, Weisberrg JI, Bisognano M, Yates GR, Hunt GC. More quality measures versus what matters: a call for balance and parsimony. BMJ Quality and Safety Online First 2012 doi:10.1136/bmjqs-2012-001081.
- Citters AD and Nelson EC. Quality and performance improvement amongst QUEST charter member hospitals. A realist evaluation to understand factors that drive and retain improvement. Boston: Institute for Healthcare Improvement; 2011.
- Lyons RA, Jones KH, John G, Brooks CJ, Verplancke JP, Ford DV, Brown G, Leake K. The SAIL databank: linking multiple health and social care datasets. BMC Med Inform Decis Mak 2009 Jan 16;9:3.
- Ford DV, Jones KH, Verplancke JP, Lyons RA, John G, Brown G, Brooks CJ, Thompson S, Bodger O, Couch T, Leake K. The SAIL Databank: building a national architecture for e-health research and evaluation. BMC Health Services Research 2009;9:157; doi:10.1186/1472-6963-9-157.
- Lyons R, Ford DV, Jones, KH. The Secure Anonymised Information Linkage (SAIL) system in Wales has privacy protection at its heart.BMJ 2014; 348 doi: http://dx.doi.org/10.1136/bmj.g2384 (Published 3 April 2014).