Search tips
Search criteria 


Logo of jrsocmedLink to Publisher's site
J R Soc Med. 2006 May; 99(5): 226–231.
PMCID: PMC1457759

Payment by results or payment by outcome? The history of measuring medicine


Provision and funding of healthcare in Britain today is undergoing one of the most profound revolutions in the 58 year history of the National Health Service. Clinicians and managers are being presented with a series of organizational reforms that affect how care to their patients is delivered and funded. Choose and book, practice-based commissioning, and payment by results are new concepts that introduce a more business focused healthcare economy. The evidence base for these initiatives, and the benefits that they will deliver, has not yet been made explicit.

Healthcare outcome assessment is an essential aspect of reforming health provision successfully. Currently, the NHS records outcome based on measures of activity and process, such as waiting times and the number of patients treated. What really matters to patients is the outcome of their healthcare intervention, what effect it will have on their wellbeing, and the length of their life.1

Since 1991 there has been a series of changes to funding within the NHS.2 The latest reform to be introduced is PbR which, the Department of Health assures, will:

`... reward efficiency and quality in providing services; support greater patient choice and more responsive services; and enable PCTs [primary care trusts] to concentrate on quality and quantity rather than price'.3

These are laudable aims for any healthcare reimbursement structure. Nevertheless, it would be unwise to adopt too narrow a definition of quality. Too much concern with the technical management of illness, and too little attention to prevention, rehabilitation, coordination and continuity of care, will result in a poorer outcome for the overall health community that the reforms are meant to improve.4

Most observers recognize that biomedical measures such as clinical or laboratory indices do not provide a complete representation of the effect of a treatment on an individual.5 These measures, while important in their own right, are being supplemented by measures of constructs that focus on issues of importance to the patient such as functional status, health-related quality of life and emotional well being.6 Compared with concrete measures like blood pressure, constructs such as pain relief, walking ability or depression are complex to measure. They generally require the use of patient-based outcome measures, where the patient gives his or her opinion on the construct in question.7

Measuring outcome remains the ultimate validation of the effectiveness and quality of healthcare. Only by systematically recording the outcomes using methods that are appropriate to the patient group under consideration can a healthcare system promote quality in all activities.8 However, payment by results will fail to meet the objectives set out by the Department of Health if the wrong outcomes are measured. We aim to examine the introduction of the payment by results policy within the history of healthcare outcome measurement and suggest how the policy may actually match its purported outcome.


The outcome of medical therapy has always been a topic of interest to communities with medical practitioners. In ancient Egypt, it was recognized that certain conditions such as tetraplegia were incurable.9 Untoward outcomes were punished by fiscal and financial penalties depending on the severity of the mishap.1 By the 1700s the ability to obtain good quality medical care was related to social status—provision of medical attention depended on one's ability to agree a legal contract with a physician.10 Thus, healthcare was readily available for landowners, but difficult to obtain for women and children who were barred from entering into contracts.

The science of recording and learning from outcomes of treatment at this time was non-existent; it was not until 1754 that the first investigative trial was performed by James Lind (1716-1794) for the treatment of scurvy.11

Even with this step forward in medical treatment, it was nearly 50 years before dietary modification was used by the British Navy as an intervention for scurvy.

Public health, similarly, was poorly developed and only through the efforts of John Snow (1813-1858) was the cause of the Broad Street pump cholera outbreak elucidated. Whilst the outbreak was well under control by the time that Snow persuaded the authorities of the time to prevent people drinking the water, it was his work associating contaminated water from the pump with gastrointestinal symptoms that proved the link.12 This was the first example of what could be called contact tracing.

Outcome measurement by hospitals in the 1800s was largely limited to collecting mortality statistics, with no regard for the results of the operations and interventions that were performed. Florence Nightingale (1820-1910) wrote, in her important 1863 treatise Notes on Hospitals:

`... if the function of a hospital were to kill the sick, statistical comparison of this nature would be admissible'.13

After returning from the Crimean War—where she was responsible for a marked decrease in the death rate amongst wounded soldiers14—Nightingale highlighted the importance of proper hospital activity analysis in identifying the causes of inpatient mortality. Her later work with Henry Currey (1820-1900) revolutionized hospital design, leading to further decreases in inpatient mortality.15

At the start of the 20th Century, cleanliness had improved and hospital committees began to report on the throughput of their institutions. However, hospital statistics remained cursory with no record of the results of the interventions provided to patients. Ernest Codman (1869-1940) was the first clinician to systematically follow all patients to record the `end-result' of the surgical care they received in his hospital in Boston of the same name.16 He recorded diagnostic and treatment errors and linked these to the patient's eventual outcome, in order to make improvements in the care his hospital provided. It was a prestigious institution with Charles Mayo and Harvey Cushing also working as surgeons at the End Result Hospital and participating in the systematic recall of their patients. Unfortunately, the hospital failed after the First World War and Codman turned his energies to other issues—most notably setting up the first tumour register for sarcomas. Nevertheless, Codman's work laid the basis for future work studying the outcome of medical care.

The next step-change in evaluating the quality of health outcomes was delivered by Avedis Donabedian (1919-1990). Professor of Public Health at the University of Michigan from 1961 until his death, he spent his career studying the relationships between quality and healthcare systems. Donabedian was the first researcher to assess healthcare quality using the concepts of structure, process and outcomes, with which we are familiar with today. He felt that politicians made decisions on health policy unsupported by evidence: a situation not unlike current practice!

Outcome evaluation became established in the 1970s as medical interventions became commonplace, yet more technical; it was no longer such an ordeal to have even quite major surgery. Indeed, it became the case that some surgery was done routinely; for example, tonsillectomies were performed often for minor indications. The ethics and monetary consequences of these practices came into question and forced clinicians to base their treatments on the results of research and not anecdote.17


Quality is influenced by the importance society places on certain values such as compassion, equity and value for money. Complexity in measuring quality in healthcare is compounded by deciding who should set the criteria for quality, how these criteria are set, and whether the criteria are empirical or normative (i.e. whether quality is based on existing healthcare provision or whether a standard is established). Irrespective of whichever quality criteria are set, it is important not to use too narrow a definition of quality of care. Too much concern with the technical management of illness will result in a diminution in attention to prevention, rehabilitation, and coordination and continuity of care, and the consequent effects on the clinician-patient relationship.4

Once the definition of the quality of the delivered healthcare has been agreed, it is necessary to examine the outcomes that need to be measured. Often survival data are presented for conditions where survival is not at risk. Survival is also frequently presented in combination with other outcomes, such as recurrence of the index event or other morbidity.18 In some instances, prolonging survival may not actually be in a patient's best interests.19 Therefore, it is essential to choose a scientifically robust measure that is mapped on to the definition of quality in use. There are numerous texts that provide examples of suitable measures and their measurement properties.6,20,21

Of course, it is not only healthcare interventions that affect how quality is perceived. A patient's experience of the healthcare environment can be affected by other aspects outside the control of the clinical staff, such as food or accommodation. The impact of primary or secondary prevention can be undermined by poor concordance with medication or exercise programmes22,23 leading to a poorer experience of healthcare overall. Secondary prevention is a particularly difficult element of healthcare to measure, as it is substantially different from procedural interventions, which, by their definition, can have a near immediate effect.

The length of time from a discrete intervention to an eventual outcome can have a large bearing on whether healthcare delivery is regarded as effective. Endoprostheses for large joint arthroplasties, such as the hip and knee, are, in short-term follow-up studies, some of the most effective procedures known.24 However, with the lifespan of modern endoprostheses in excess of 20 years, it will be some time before we can fully evaluate the quality of these procedures.

A further consideration formerly under-recognized by clinicians is that health status measures—such as walking ability or level of pain—may not actually be the relevant outcome for many people; as improving walking or reducing pain may only be an antecedent to returning to work or education. There are not many healthcare providers in the UK today who regularly measure wider, participation outcomes; yet they are increasingly relevant to society, particularly in reducing the cost of incapacity benefits.


Within the NHS at present there is an increasing emphasis on audit; not just the end-results of medical care, but also the process by which it has been achieved. Audit is defined as `... the systematic critical analysis of the quality of medical care, including diagnosis, treatment, outcome and quality of life for the patient'.25 All departments within hospitals and in the community are expected to audit their practice regularly.26 Nevertheless, systematic introduction of audit is not a panacea to healthcare quality issues.27,28 Whilst data is gathered about the process of medical care, little is often done to alter the deficiencies found. The essential part of using outcomes to improve care is to complete the `audit cycle' by examining the deficiencies noted, implementing changes and then reassessing the changes to see if they have resulted in the anticipated result.29-31 Furthermore, audit should measure the enhancement in patients' quality of life and functioning that is brought about by the medical care using the appropriate measurement tools.32


Payment by results first entered the NHS lexicon in 2002.3 The concept included a nationally agreed set of prices for healthcare activities known as tariffs, and the introduction of healthcare resource groups (HRG)—treatment episodes that are similar in resource use and in clinical response. It was intended to focus initially on the commissioning of elective care between PCTs and secondary care (principally NHS hospital trusts) but was envisioned to encompass all clinical activity in the NHS by 2006.

However, information released in January 200533 revealed the complexity of introducing such changes in financing healthcare. This resulted in a marked reduction in the number of services that would be included in the tariffs (including most transplant operations, burn services and rehabilitation), leaving only elective care to be reimbursed by payment by results.34 This was, in part, due to the large increases in the numbers of short-stay emergency patients in hospitals—possibly related to targets set around 4 hour waiting times in accident and emergency departments.


Payment by results does not deliver on the promises made by the Department of Health as there is no mechanism to collect data on individual relevant patient outcomes and link these to the payment received by the treating institution.35 This would promote good quality care by encouraging clinicians to adopt better medical treatments to minimize adverse events. It would also reimburse healthcare organizations adequately for complex patients and those who require further treatment because of unavoidable complications.36 Hospital trusts are already finding it difficult to manage the budgetary constraints imposed by new funding arrangements and, for the first time in several years, the NHS will be in deficit.37

Instead, with payment by results, volume of activity and speed of delivery has become a surrogate for high-quality healthcare provision.3 Ensuring consistent quality in healthcare provision requires constant attention to outcome: very few centres are able to achieve this because of the limited resources devoted to recording the outcome of interventions. There are a number of examples of services with extensive experience of systematically recording and analysing their outcomes, using this information to continually improve the service they deliver; these need to be emulated across the UK.38 Even if this could be achieved within the time allocated to the introduction of payment by results, there is a failure to link improvements that individual departments achieve through audit programmes prospectively and inadequate recognition of those changes through increased funding. The potential to encourage quality improvement in the NHS through payment by results is present but without proper outcome collection and analysis this will not occur.

Payment by results is also promoted as an agent to deliver choice to patients. However, routine healthcare for long-term conditions may not attract as much of the available resources as elective surgery; district general hospitals which bear the burden of chronic disease management in the UK may close resulting, paradoxically, in less choice for patients.39 Furthermore, many private providers might take the opportunity to select less complicated elective surgical patients; thus further draining the resources of NHS Trusts left to deal with less surgically fit patients and those prone to developing complications.40

What has largely been ignored throughout the introduction of payment by results is the increased cost of administering the system. The internal market reforms of the 1990s saw a marked increase in administration charges as trusts had to cope with multiple service level agreements with commissioners.41 With payment by results, trusts will struggle to manage what will, in effect, be millions of service level agreements each year.42

A further issue is the method of calculating the tariffs that will be paid to trusts for healthcare. There is a lack of transparency to the origin of these figures and how the `average' across trusts was calculated. Various publications have described the tariff as either the mean43 or the median39 of the cost of each procedure within the NHS: they do not, however, recognize the important difference between the two calculations when describing skewed data.

Finally, there is the disturbing comment in payment by results literature that trusts which are able to provide a service for less than the tariff will be able to generate what amounts to profit.34 This can only result in perverse incentives to reduce the quality of care that is provided to patients.44 This was anticipated prior to the introduction of payment by results, where, by focusing on activity, which is easily measurable, the overall quality of care may diminish.45 An allied development could be `HRG-drift', where trusts spuriously code for more complex interventions than patients actually receive;46 more worryingly, patients may actually receive interventions that attract a higher tariff with the intention of enhancing income.47


None of the concerns outlined above include the effects that payment by results may have on a range of other activities that hospitals in the UK perform. Much undergraduate and postgraduate clinical teaching occurs in district general hospitals in the UK. This is often unrecognized and goes unrewarded. Departments struggling to keep up with the demands placed upon them by payment by results may find it impossible to deliver this training; thus jeopardizing the significant changes that are occurring in medical education in the UK at present, such as the introduction of the Foundation Years. Similarly, many hospital departments perform an important research role, either alone or as part of multi-centre trials, which could also be affected by the proposed changes in funding.

By concentrating on services that are included in the list of tariffs, there will be no incentive for trusts to develop new services for which there is currently an (un)recognized need. This will be particularly evident in areas such as provision of equipment, physiotherapy and wheelchairs, many of which are already inadequate.48

Lastly, there is the role that hospitals play within the community as a source of employment for local populations. The longer-term political effects of hospital closures may alter the effectiveness with which payment by results can deliver the promises made of it because of interference by external agencies.49


In order to make sense of the impact of payment by results on the health economy in the UK, considerable work needs to be done on modelling the effects of such a marked change in funding on the NHS. This can only be achieved by ensuring that there is a robust method of capturing data on outcomes throughout the health service. Adequate funding of health services research is required to deliver the outcomes that are essential to the introduction and monitoring of payment by results.

There are a number of practical elements that need to be put into place to ensure the success of the payment by results project:

  • data concerning healthcare interventions must be recorded accurately
  • measures must be appropriately chosen for the outcome under consideration
  • outcome measures must be recorded accurately at the appropriate times
  • healthcare costs should be realistic. Complex multidisciplinary care is expensive. Trying to substitute simplified versions of best practice through fragmenting multi-professional teams will only result in inferior care for patients
  • adequate numbers of expertly trained staff are needed to collect and analyse these data. These requirements must be balanced against the policy of reducing administrative staff within the NHS.

One method of correctly aligning healthcare with properly measured outcomes is to introduce integrated care pathways to collect and manage the data that are generated.50 An integrated care pathway encompasses the development and implementation of evidence-based guidelines, with continuous evaluation of the clinical process and outcomes to improve the quality of care. As discussed above, this has not been achieved by audit alone. Several studies of NHS services have examined the use of integrated care pathways and the contribution that they can have on enhancing patients' care.51-53 Properly designed statistical packages could collect and analyse these data in real time, providing trusts with timely financial information. This could also offer an unprecedented opportunity to research a complex nationwide healthcare system.36

Once structures are in place to properly measure and manage scientifically sound outcomes of healthcare interventions, it is crucial to align those outcomes correctly with income provided by payment by results. It is not sufficient in a modern healthcare system to reimburse trusts for activity that is not adequately monitored.54


To use the analogy of paying for a train journey, you would expect to arrive on time and in safety at your destination in exchange for your fare. Train companies are rewarded or penalized on the basis of the number of delayed journeys and issues affecting passengers' safety. Yet, we are expecting hospitals to be paid simply for activity, not for satisfactory and scientifically measured outcomes. Outcome remains the ultimate validation of the effectiveness and quality of healthcare. Measuring outcome, not activity, will enhance the quality of clinical services facilitating a proper choice to patients. Whilst clinicians strive to deliver evidence-based medicine to their patients, it is worrying that the Department of Health is introducing evidence-free financial policies.


Competing interests None.


1. Schwartz JS, Lurie N. Assessment of medical outcomes. New opportunities for achieving a long sought-after objective. Int J Technol Assess Health Care 1990;6: 333-9 [PubMed]
2. Pollock AP. NHS plc. London: Verso, 2004
3. Department of Health. Reforming NHS Financial Flows: Introducing Payment By Results. London: DoH, 2002
4. Donabedian A. Evaluating the quality of medical care. Milbank Mem Fund Q 1966;44(Suppl): 166-206 [PubMed]
5. Thyberg I, Skogh T, Hass UA, Gerdle B. Recent-onset rheumatoid arthritis: a 1-year observational study of correlations between health-related quality of life and clinical/laboratory data. J Rehab Med 2005;37: 159-65 [PubMed]
6. Fitzpatrick R, Davey C, Buxton MJ, Jones DR. Evaluating patient-based outcome measures for use in clinical trials. Health Technol Assess 1998;2: 1-74 [PubMed]
7. Bergner M, Rothman ML. Health status measures: an overview and guide for selection. Ann Rev Public Health 1987;8: 191-210 [PubMed]
8. Turner-Stokes L, Williams H, Abraham R, Duckett S. Clinical standards for inpatient specialist rehabilitation services in the UK. Clin Rehab 2000;14: 468-80 [PubMed]
9. Inman C. Effectiveness of spinal cord injury rehabilitation. Clin Rehab 1999;13(Suppl 1): 25-31 [PubMed]
10. Crawford C. Patients' rights and the law of contract in eighteenth-century England. Soc Hist Med 2000;13: 381-410 [PubMed]
11. Bloom BS. Does it work? The outcomes of medical interventions. Int J Technol Assess Health Care 1990;6: 326-32 [PubMed]
12. Snow J. On The Mode of Communication of Cholera, 2nd edn. London: John Churchill, 1855
13. Iezzoni LI. 100 apples divided by 15 red herrings: a cautionary tale from the mid-19th century on comparing hospital mortality rates. Ann Internal Med 1996;124: 1079-85 [PubMed]
14. Neuhauser D. Florence Nightingale gets no respect: as a statistician that is. Qual Safety Health Care 2003;12: 317 [PMC free article] [PubMed]
15. Cook GC, Webb AJ. Reactions from the medical and nursing professions to Nightingale's “reform(s)” of nurse training in the late 19th century. Postgrad Med J 2002;78: 118-23 [PMC free article] [PubMed]
16. Neuhauser D. Ernest Amory Codman, M.D., and end results of medical care. Int J Technol Assess Health Care 1990;6: 307-25 [PubMed]
17. Wennberg JE, Bunker JP, Barnes B. The need for assessing the outcome of common medical practices. Ann Rev Public Health 1980;1: 277-95 [PubMed]
18. Stroke Unit Trialists' Collaboration. Collaborative systematic review of the randomised trials of organised inpatient (stroke unit) care after stroke. Stroke Unit Trialists' Collaboration. BMJ 1997;314: 1151-9 [PMC free article] [PubMed]
19. Voltz R. Palliative therapy in the terminal stage of neurological disease. J Neurol 1997;244: S2-10 [PubMed]
20. Bowling A. Measuring Disease: A Review of Disease-Specific Quality of Life Measurement Scales, 2nd edn. Buckingham: Open University Press, 2001
21. Staquet MJ, Hays RD, Fayers PM. Quality of Life Assessment In Clinical Trials: Methods And Practice, 1st edn. New York: Oxford University Press, 1998
22. Javors JR, Bramble JE. Uncontrolled chronic disease: patient non-compliance or clinical mismanagement? Dis Mgmt 2003;6: 169-78 [PubMed]
23. Campbell R, Evans M, Tucker M, Quilty B, Dieppe P, Donovan JL. Why don't patients do their exercises? Understanding non-compliance with physiotherapy in patients with osteoarthritis of the knee. J Epidemiol Commun Health 2001;55: 132-8 [PMC free article] [PubMed]
24. O'Boyle CA, McGee H, Hickey A, O'Malley K, Joyce CR. Individual quality of life in patients undergoing hip replacement. Lancet 1992;339: 1088-91 [PubMed]
25. Department of Health. Working For Patients. London: HMSO, 1989
26. Wainwright JR, Sullivan FM, Morrison JM, MacNaughton RJ, McConnachie A. Audit encourages an evidence-based approach to medical practice. Med Educ 1999;33: 907-14 [PubMed]
27. Brook RH. Managed care is not the problem, quality is. JAMA 1997;278: 1612-14 [PubMed]
28. Davies HT. Exploring the pathology of quality failings: measuring quality is not the problem—changing it is. J Evaluation Clin Practice 2001;7: 243-51 [PubMed]
29. Gnanalingham J, Gnanalingham MG, Gnanalingham KK. An audit of audits: are we completing the cycle? J R Soc Med 2001;94: 288-9 [PMC free article] [PubMed]
30. National Institute for Clinical Evidence. Principles For Best Practice In Clinical Audit, 1st edn. Oxford: Radcliffe Medical, 2002
31. Turner-Stokes L. The development of clinical governance in the UK: its implications for rehabilitation medicine. Clin Med 2003;3: 135-41 [PubMed]
32. National Health Service Review. Working Paper No 6. Medical Audit. London: HMSO, 1989
33. Douglas R. Scope of Payment By Results In 2005/6. London: Department of Health, 2005
34. Department of Health. Implementing Payment By Results—Technical Guidance 2005/06. London: DoH, 2005
35. Department of Health. Payment by Results Consultation: Preparing for 2005. London: DoH 2003: 1-60
36. Neuhauser D, Harper DL. Too good to last: did Cleveland Health Quality Choice leave a legacy and lessons to be learned? Qual Safety Health Care 2002;11: 202-3 [PMC free article] [PubMed]
37. National Audit Office. Financial Management in the NHS. Summarised Accounts 2003-04. London: NAO, 2005
38. Freeman JA, Hobart JC, Playford ED, Undy B, Thompson AJ. Evaluating neurorehabilitation: lessons from routine data collection. J Neurol Neurosurg Psych 2005;76: 723-8 [PMC free article] [PubMed]
39. Dixon J. Payment by results—new financial flows in the NHS. BMJ 2004;328: 969-70 [PMC free article] [PubMed]
40. The Public Administration Select Committee. Choice, Voice and Public Services. London: The Stationery Office, 2005: 78
41. Webster C. Overthrowing the market in health care: the achievements of the early National Health Service. J R Coll Physicians Lon 1995;29: 502-7 [PubMed]
42. Maynard A, Bloor K. Reforming the consultant contract again? BMJ 2004;329: 929-30 [PMC free article] [PubMed]
43. Department of Health. Annex 4. Reforming NHS Financial Flows: Introducing Payment By Results. London: DoH, 2002
44. Bevan G, Hood C. Have targets improved performance in the English NHS? BMJ 2006;332: 419-22 [PMC free article] [PubMed]
45. Sheldon T. Promoting health care quality: what role performance indicators? Qual Health Care 1998;7(Suppl): S45-50 [PubMed]
46. Rogers R, Williams S, Jarman B, Aylin P. “HRG drift” and payment by results. BMJ 2005;330: 563. [PMC free article] [PubMed]
47. Wilson T, Buck D, Ham C. Rising to the challenge: will the NHS support people with long term conditions? BMJ 2005;330: 657-61 [PMC free article] [PubMed]
48. Kersten P, McLellan L, George S, Smith JA, Mullee MA. To what extent are the needs of severely disabled people met by community rehabilitation services? A follow-up study. Disability Rehab 2000;22: 855-61 [PubMed]
49. Smith R. The private sector in the English NHS: from pariah to saviour in under a decade. Can Med Assoc J 2005;173: 273-4 [PMC free article] [PubMed]
50. Kitchiner D, Davidson C, Bundred P. Integrated care pathways: effective tools for continuous evaluation of clinical practice. J Evaluation Clin Pract 1996;2: 65-9 [PubMed]
51. Nathwani D, Williams F, Winter J, Winter J, Ogston S, Davey P. Use of indicators to evaluate the quality of community-acquired pneumonia management. Clin Infect Dis 2002;34: 318-23 [PubMed]
52. Newton M. Integrated care pathway: the prevention and management of pressure ulcers. J Tissue Viability. 2003;13: 126-9 [PubMed]
53. Sulch D, Perez I, Melbourn A, Kalra L. Randomized controlled trial of integrated (managed) care pathway for stroke rehabilitation. Stroke 2000;31: 1929-34 [PubMed]
54. O'Connor RJ, Neumann VC, Tennant A. Payment needs to be by end-results, not just results. BMJ 2004;328: 969-70 [PubMed]

Articles from Journal of the Royal Society of Medicine are provided here courtesy of Royal Society of Medicine Press