|Home | About | Journals | Submit | Contact Us | Français|
This much, then, is clear—that the mean state is in every case to be praised, but that sometimes we must incline towards the excess, sometimes towards the deficiency, because in this way we shall most easily hit the mean, namely what is good—Aristotle, Nicomachean Ethics
All the business of war, and indeed all the business of life, is to endeavour to find out what you don't know by what you do; that's what I called ‘guessing what was at the other side of the hill’—Duke of Wellington, The Croker Papers
If anything has characterized the professions in the past decade it is the proliferation of guidelines, targets and standards. These are beginning to act as a kind of scaffold—in both senses, some fear—for medical practice. In this modern setting it is increasingly important, for clinicians, patients and payers, to know the consequences of clinical intervention. However, guessing what is on the ‘other side of the hill’ is arguably not good enough, and ad hoc clinical encounters are not the best basis for predictable clinical outcomes1. In renal medicine, my own specialty, the quantitative components of clinical practice are particularly suitable for audit purposes, in regard to both treatment processes and outcomes. It is not surprising that some of the issues in predicting clinical outcomes are being explored first in a renal context.
Current documents recommend a variety of renal standards, for blood pressure, biochemistry, dialysis dose and so on2,3. A minimum or maximum value is declared typically as a limit for desirable clinical results. When outcome data are presented—for example, in the UK Renal Registry (UKRR) report—they are given as distributions for any given renal unit patient cohort. Depending on where the standard limits have been pitched for the variable there will be an overlap, since the outcome distributions have an inevitable dispersion (standard deviation, SD) (Figure 1). Minimizing the dispersion of results is desirable for clinical and economic reasons, but, contrary to intuition, the mere declaration of a ‘target’ value does not necessarily narrow the eventual distribution. Much effort may be required to do so, even in areas of technical control such as dialysis4. One may of course choose to aggregate the mean values of repeated measurements in individual patients, so as to narrow any given range—a tactic that has been little debated. For a majority of values to fall on the desired side of the declared minimum/maximum, the mean/median of the outcome distribution must to a varying degree exceed the guideline value. The ‘target’ aiming point for management therefore also needs to exceed the standard limit and the necessary mean by some uncertain amount. This ‘something in excess’ of desirable results does not have a name, nor do we have a simple phrase to convey the necessity (except perhaps Robert Browning's ‘a man's reach should exceed his grasp’). The excess allows for the inevitable under-achievements of practice, whether due to problems of patient ascertainment or inadequacy of delivered treatment. Perhaps because we are always opposing a pathological ‘pressure’, it is usual for the factors that impede therapy to far outweigh those that facilitate it. In other words, our processes do not lead to failure or success at random, but are biased to under-performance5,6,7. Both under-aspiration and miscellaneous practical factors underlie this phenomenon, as demonstrated in studies of dialysis dosing in the US8,9.
The more sophisticated ‘standards’ may take this into account by specifying that physician compliance need only involve, say, 85% of the patient group. Such an allowance still implies that the distribution of results must assume a certain position in relation to the limit. As shown in Figure 1, when results are Gaussian in distribution 85% will be above a given minimum if that is one SD below the mean, a property of classical statistics. This gives a lead to achieving the standard, as demonstrated in Box 1 (Nos 1-3). Such specific positioning of the distribution of results is difficult to achieve by design. The unthinking use of ‘target’ values seems to lead to distributions that straddle the limit, as illustrated by the ESAM study of renal anaemia, where the minimum haemoglobin standard of 11 g/dL is also the outcome mean value10. Perhaps clinical effort falls away once the value is achieved, or perhaps pathological ‘pressure’ causes an undesirable drift in the population under stable therapy. It remains the case that the ‘target’ aiming point towards which effort must be applied is uncertain in current systems—to what pressure below 140/80 mmHg should one pursue values in order to achieve a high rate of correspondence with a 140/80 maximum? Moreover, even in the best studies of treatment efficacy the declared treatment aims may prove unachievable11.
Studies from the UKRR suggest another way to assess the ‘over-achievement’ necessary for complete correspondence with ‘standards’. The outcome distributions for haemoglobin and dialysis dose, measured as urea reduction ratio, are Gaussian, with rather uniform dispersion of data (SDs). This allows the use of data from several renal units to explore the relation of mean/median and per cent satisfaction with a guideline minimum/maximum. A plot of the mean/median of each unit against the per cent compliance with any standard min/max indicates the mean/median of the necessary distribution (Figure 2)12. In this case a median unit haemoglobin of about 11.5 g/dL would be necessary to comply with 85% above 10 g/dL. An essential caveat is that this reflects current procedures, since a systematic narrowing of outcome ranges would give different necessary values.
Box 1 Strategy for achieving desired minimum haemoglobin in dialysis patients
Having decided what specific distribution, how should clinicians then proceed? Can we manage a patient cohort so as to produce a predictable distribution of results? In other words, can we aim at averages? The usual technique of declaring progressively more extreme ‘aiming-points’ (say lower blood pressure or higher haemoglobin) may drive outcome distributions in the desired direction, but this is scarcely a predictable methodology. There could be other approaches, but it begs a method that is more explicit than an ad hoc approach to individual patients (the usual gold standard of practice7). Sociology exists partly because people tend to behave differently in groups than as individuals, so we might reasonably ask whether patient groups might be handled in the whole, rather than as simply an aggregate ‘sum of parts’. Since we manage individual patients by shifting treatment doses to adjust towards desirable results, what would be the effect of doing the same systematically to a large population? This implies fixed intervention points—in the case of renal anaemia, say, one ‘threshold’ value below the desired mean/median and one ‘ceiling’ value above. This has been attempted in a large unselected dialysis cohort over several years for the management of renal anaemia with erythropoietin and iron and seems to produce reliable distributions that can be made to comply with ‘standard’ recommendations13,14,15 (see Box 1, No. 4). As it happens, UKRR data show that in practice we can always know what is ‘on the other side of the hill’, month on month, year on year. The outcome of clinical management is very stable when reflected in large groups, and shifts only with major changes of procedure or case-mix. What are required are treatment technologies to allow the determination of distributions in response to best practice guidelines16.
We do not have the means of predicting the distribution of results unless we adopt some new approaches, where the ‘aiming-point’ is likely to be less important than the threshold/ceiling values for intervention. These need to be defined through clinical research in each case. The further implication is that recommendations should in future not only contain the desirable limits but also attempt to define the features of the anticipated outcome distributions in mean/median and range. They should also, for best, indicate the costs and safety of achieving them, in case of hazard at the extremes of predictable outcome ranges and futile expenditure in the course of over-compensation for under-achievement17. This reflects the fact that guidelines and standards are the basis of treatment policies that should be subject to explicit risk analysis before implementation. Although the fanfares of the guideline culture were not entirely without justification, it appears that we know better where to go than how to get there. This is partly because efficacy studies (can it be done?) greatly exceed effectiveness studies (does it work?)18. Declarations of ideal intent imply the need for research into calibrated clinical interventions, to put the achievement of clinical outcomes into a predictable, safe and cost-effective mould19. The fusion of clinical aspiration, basic medical science and statistics in this exercise represents a novel response to the recent call for integration of these elements of medicine20.