PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of jrsocmedLink to Publisher's site
 
J R Soc Med. 2001 December; 94(12): 617–619.
PMCID: PMC1282292

Aiming at averages

E J Will, BM FRCP

This much, then, is clear—that the mean state is in every case to be praised, but that sometimes we must incline towards the excess, sometimes towards the deficiency, because in this way we shall most easily hit the mean, namely what is good—Aristotle, Nicomachean Ethics

All the business of war, and indeed all the business of life, is to endeavour to find out what you don't know by what you do; that's what I called ‘guessing what was at the other side of the hill’—Duke of Wellington, The Croker Papers

If anything has characterized the professions in the past decade it is the proliferation of guidelines, targets and standards. These are beginning to act as a kind of scaffold—in both senses, some fear—for medical practice. In this modern setting it is increasingly important, for clinicians, patients and payers, to know the consequences of clinical intervention. However, guessing what is on the ‘other side of the hill’ is arguably not good enough, and ad hoc clinical encounters are not the best basis for predictable clinical outcomes1. In renal medicine, my own specialty, the quantitative components of clinical practice are particularly suitable for audit purposes, in regard to both treatment processes and outcomes. It is not surprising that some of the issues in predicting clinical outcomes are being explored first in a renal context.

AIMING POINTS

Current documents recommend a variety of renal standards, for blood pressure, biochemistry, dialysis dose and so on2,3. A minimum or maximum value is declared typically as a limit for desirable clinical results. When outcome data are presented—for example, in the UK Renal Registry (UKRR) report—they are given as distributions for any given renal unit patient cohort. Depending on where the standard limits have been pitched for the variable there will be an overlap, since the outcome distributions have an inevitable dispersion (standard deviation, SD) (Figure 1). Minimizing the dispersion of results is desirable for clinical and economic reasons, but, contrary to intuition, the mere declaration of a ‘target’ value does not necessarily narrow the eventual distribution. Much effort may be required to do so, even in areas of technical control such as dialysis4. One may of course choose to aggregate the mean values of repeated measurements in individual patients, so as to narrow any given range—a tactic that has been little debated. For a majority of values to fall on the desired side of the declared minimum/maximum, the mean/median of the outcome distribution must to a varying degree exceed the guideline value. The ‘target’ aiming point for management therefore also needs to exceed the standard limit and the necessary mean by some uncertain amount. This ‘something in excess’ of desirable results does not have a name, nor do we have a simple phrase to convey the necessity (except perhaps Robert Browning's ‘a man's reach should exceed his grasp’). The excess allows for the inevitable under-achievements of practice, whether due to problems of patient ascertainment or inadequacy of delivered treatment. Perhaps because we are always opposing a pathological ‘pressure’, it is usual for the factors that impede therapy to far outweigh those that facilitate it. In other words, our processes do not lead to failure or success at random, but are biased to under-performance5,6,7. Both under-aspiration and miscellaneous practical factors underlie this phenomenon, as demonstrated in studies of dialysis dosing in the US8,9.

Figure 1
Any set of results will be aligned in relation to a declared standard minimum or maximum, overlapping to a variable extent because of the inevitable dispersion of results. In this example the distribution overlaps a desired minimum value in a particular ...

DEFINING OUTCOME DISTRIBUTIONS

The more sophisticated ‘standards’ may take this into account by specifying that physician compliance need only involve, say, 85% of the patient group. Such an allowance still implies that the distribution of results must assume a certain position in relation to the limit. As shown in Figure 1, when results are Gaussian in distribution 85% will be above a given minimum if that is one SD below the mean, a property of classical statistics. This gives a lead to achieving the standard, as demonstrated in Box 1 (Nos 1-3). Such specific positioning of the distribution of results is difficult to achieve by design. The unthinking use of ‘target’ values seems to lead to distributions that straddle the limit, as illustrated by the ESAM study of renal anaemia, where the minimum haemoglobin standard of 11 g/dL is also the outcome mean value10. Perhaps clinical effort falls away once the value is achieved, or perhaps pathological ‘pressure’ causes an undesirable drift in the population under stable therapy. It remains the case that the ‘target’ aiming point towards which effort must be applied is uncertain in current systems—to what pressure below 140/80 mmHg should one pursue values in order to achieve a high rate of correspondence with a 140/80 maximum? Moreover, even in the best studies of treatment efficacy the declared treatment aims may prove unachievable11.

Studies from the UKRR suggest another way to assess the ‘over-achievement’ necessary for complete correspondence with ‘standards’. The outcome distributions for haemoglobin and dialysis dose, measured as urea reduction ratio, are Gaussian, with rather uniform dispersion of data (SDs). This allows the use of data from several renal units to explore the relation of mean/median and per cent satisfaction with a guideline minimum/maximum. A plot of the mean/median of each unit against the per cent compliance with any standard min/max indicates the mean/median of the necessary distribution (Figure 2)12. In this case a median unit haemoglobin of about 11.5 g/dL would be necessary to comply with 85% above 10 g/dL. An essential caveat is that this reflects current procedures, since a systematic narrowing of outcome ranges would give different necessary values.

Figure 2
Plot of median renal unit haemoglobin (Hb) against per cent values greater than 10 g/dL (the Renal Association ‘standard’). A median of 11.5 g/dL will be necessary, under current clinical conditions, to achieve the ‘standard’ ...

Box 1 Strategy for achieving desired minimum haemoglobin in dialysis patients

  1. For haemoglobin in haemodialysis patients the Renal Association ‘standards document’ recommended a minimum value of 10 g/dL in >85% of patients after 3 months of treatment (Ref. 2)
  2. The UK Renal Registry 2000 records that the average standard deviation of haemoglobin values (single point data) in data-sets from several renal units is 1.7
  3. The mean of the distribution of haemoglobin in any current cohort needs to be desired minimum plus SD (i.e. 10+1.7=11.7 g/dL) to give 85% above 10 g/dL (Ref. 17)
  4. This was achieved and sustained in one study by using 11 g/dL as the lower intervention value and 12 g/dL as the ceiling, for treatment with intravenous iron and subcutaneous erythropoietin. This intervention model was based on local clinical research in a particular dialysis patient cohort that examined paired intervention values of 10.5/11.5 and 12.0/13.0 g/dL (Ref. 13).

ACHIEVING PARTICULAR DISTRIBUTIONS

Having decided what specific distribution, how should clinicians then proceed? Can we manage a patient cohort so as to produce a predictable distribution of results? In other words, can we aim at averages? The usual technique of declaring progressively more extreme ‘aiming-points’ (say lower blood pressure or higher haemoglobin) may drive outcome distributions in the desired direction, but this is scarcely a predictable methodology. There could be other approaches, but it begs a method that is more explicit than an ad hoc approach to individual patients (the usual gold standard of practice7). Sociology exists partly because people tend to behave differently in groups than as individuals, so we might reasonably ask whether patient groups might be handled in the whole, rather than as simply an aggregate ‘sum of parts’. Since we manage individual patients by shifting treatment doses to adjust towards desirable results, what would be the effect of doing the same systematically to a large population? This implies fixed intervention points—in the case of renal anaemia, say, one ‘threshold’ value below the desired mean/median and one ‘ceiling’ value above. This has been attempted in a large unselected dialysis cohort over several years for the management of renal anaemia with erythropoietin and iron and seems to produce reliable distributions that can be made to comply with ‘standard’ recommendations13,14,15 (see Box 1, No. 4). As it happens, UKRR data show that in practice we can always know what is ‘on the other side of the hill’, month on month, year on year. The outcome of clinical management is very stable when reflected in large groups, and shifts only with major changes of procedure or case-mix. What are required are treatment technologies to allow the determination of distributions in response to best practice guidelines16.

IMPLICATIONS FOR GUIDELINES

We do not have the means of predicting the distribution of results unless we adopt some new approaches, where the ‘aiming-point’ is likely to be less important than the threshold/ceiling values for intervention. These need to be defined through clinical research in each case. The further implication is that recommendations should in future not only contain the desirable limits but also attempt to define the features of the anticipated outcome distributions in mean/median and range. They should also, for best, indicate the costs and safety of achieving them, in case of hazard at the extremes of predictable outcome ranges and futile expenditure in the course of over-compensation for under-achievement17. This reflects the fact that guidelines and standards are the basis of treatment policies that should be subject to explicit risk analysis before implementation. Although the fanfares of the guideline culture were not entirely without justification, it appears that we know better where to go than how to get there. This is partly because efficacy studies (can it be done?) greatly exceed effectiveness studies (does it work?)18. Declarations of ideal intent imply the need for research into calibrated clinical interventions, to put the achievement of clinical outcomes into a predictable, safe and cost-effective mould19. The fusion of clinical aspiration, basic medical science and statistics in this exercise represents a novel response to the recent call for integration of these elements of medicine20.

References

1. Wyatt JC. Management of explicit and tacit knowledge. J R Soc Med 2000;94: 6-9 [PMC free article] [PubMed]
2. Renal Association. Treatment of Adult Patients with Renal Failure. Recommended Standards and Audit Measures. London: Royal College of Physicians of London, 1997 [PubMed]
3. NKF-DOQI clinical practice guidelines. Am J Kidney Dis 2001; 37(suppl 1): S1-S238 [PubMed]
4. Depner T, Beck G, Daugirdas J, Kusek J, Eknoyan G. Lessons from the hemodialysis (HEMO) study: an improved measure of the actual hemodialysis dose. Am J Kidney Dis 1999;33: 142-9 [PubMed]
5. Smith WCS, Lee AJ, Crombie IK, Tunstall-Pedoe H. Control of blood pressure in Scotland: the rule of halves. BMJ 1990;300: 981-3 [PMC free article] [PubMed]
6. Mead A, Burnett S, Davey C. Diabetic retinal screening in the UK. J R Soc Med 2001;94: 127-9 [PMC free article] [PubMed]
7. Will EJ. Target practice. Int J Artif Organs 1998;21: 433-6 [PubMed]
8. Seghal AR, Snow RJ, Singer ME, et al. Barriers to adequate delivery of hemodialysis. Am J Kidney Dis 1998;31: 593-601 [PubMed]
9. Palevsky PM, Washington MS, Stevenson JA, et al. Barriers to the delivery of adequate hemodialysis in ESRD network 4. Adv Renal Replace Ther 2000;7(suppl 1): S11-S20 [PubMed]
10. Jacobs C, Horl WH, Macdougall IC, et al. European best practice guidelines 5: target haemoglobin. Nephrol Dial Transplant 2000; 15(suppl 4): 15-19 [PubMed]
11. Hansson L, Zanchetti A, Carruthers SG, et al. Effects of intensive blood-pressure lowering and low-dose aspirin in patients with hypertension: principal results of the Hypertension Optimal Treatment (HOT) randomised trial. Lancet 1998;351: 1755-62 [PubMed]
12. Rose G, Day S. The population mean predicts the number of deviant individuals. BMJ 1990;301: 1031-4 [PMC free article] [PubMed]
13. Richardson D, Bartlett C, Will EJ. Intervention thresholds and ceilings can determine the haemoglobin outcome distribution in a haemodialysis population. Nephrol Dial Transplant 2000;15: 2007-13 [PubMed]
14. Richardson D, Bartlett C, Jolly H, Will EJ. Intravenous iron for CAPD populations: proactive or reactive strategies? Nephrol Dial Transplant 2001;16: 115-19 [PubMed]
15. Richardson D, Bartlett C, Will EJ. Optimizing erythropoietin therapy in hemodialysis patients. Am J Kidney Dis 2001;38: 109-17 [PubMed]
16. Will EJ. Targets and targeting. Am J Kidney Dis 2001;38: 411-14 [PubMed]
17. Will EJ, Cameron JS. European guidelines for renal anaemia— predicting 85% compliance. Nephrol Dial Transplant 2000;15: 439-40 [PubMed]
18. Haynes B. Can it work? Does it work? Is it worth it? BMJ 1999;319: 652-3 [PMC free article] [PubMed]
19. Chalmers I. It's official: evaluative research must become part of routine care in the NHS. J R Soc Med 2000;93: 555-6 [PMC free article] [PubMed]
20. Swales J. The troublesome search for evidence: three cultures in need of integration. J R Soc Med 2000;93: 402-7 [PMC free article] [PubMed]

Articles from Journal of the Royal Society of Medicine are provided here courtesy of Royal Society of Medicine Press