Models are increasingly used for economic and policy decisions, but where data do not exist they are often based on expert opinion. To establish estimates for key parameters in decision models of malaria case management, a Delphi survey was conducted with malaria experts on the consequences of untreated malaria and non-malarial febrile illnesses. Some consensus was reached about the probability that patients over five years of age from medium/high transmission settings with untreated malaria would progress to severe disease, and the probability that a non-malaria illness could benefit from antibiotics in patients from areas of low HIV prevalence. The ranges described here are reasonable to use for decision models. However, the survey results indicated a wide dispersion of opinion on most key parameters which drive model outputs, with responses to several questions ranging from 5% to 100%. Introducing the range of estimates into a typical model evaluating malaria RDTs demonstrated the impact that diversity of opinion and variation in parameter values could have on such model outputs and on the policy conclusions drawn, varying from very positive to negative. This survey has highlighted the lack of agreement of acknowledged experts on the central parameters required to model the management of febrile illness suggestive of malaria, and the risks of relying on a single expert opinion to establish model parameters in the absence of evidence. These findings are equally important in the context of epidemiological models that often rely on similar parameters to the ones explored in this survey.
An understanding of the consequences of not treating, or inadequately treating, malaria and bacterial infections is essential for evaluating the benefits and risks of managing febrile illnesses. In 1954, Bruce-Chwatt commented that …'morbidity due to malaria is so imperfectly known that even an approximate estimate of it would be merely a guess' 
. The results of this survey suggest that little has changed in 50 years. Although more is known about the epidemiology, pathophysiology, and treatment of malaria, we still know little about the serious consequences of the illness. In malaria-endemic areas, probably only a small proportion of children with uncomplicated illness (1–2%) progress to severe disease 
. However, the case fatality rate of children hospitalized with severe malaria ranges from 10–50%, and it has been suggested that the mortality of untreated severe falciparum malaria may reach 100% 
. Even less is known about the consequences of non-malarial illnesses common in Africa and Asia. In malaria-endemic areas, at least as many children die of non-malarial causes as die of malaria 
. Many non-malarial febrile illnesses in African children are likely to be bacterial, and would benefit from appropriate antibiotic treatment 
To date, Delphi surveys have been used infrequently in the context of malaria treatment and diagnostics. When chloroquine resistance was emerging, Sudre and colleagues used a Delphi survey to estimate mortality rates due to treatment failure amongst children of different age groups 
. This survey reported a 5% mortality rate in children with highly chloroquine-resistant infections, although the distribution of expert opinions was not reported. By comparison, the case fatality rates in our study (calculated by multiplying the probability of developing severe illness by the mortality rate for severe illness) for untreated malaria was 15% in low/epidemic prone transmission areas and 9% in medium/high areas. In 2004, a Delphi survey was again used to assess the contribution of ACT usage to the reduction in malaria transmission in KwaZulu Natal 
The primary aim of this study was to gather estimates to be applied in decision modelling. The simplest application of these results would be to enter the median values into decision models. However, given the wide dispersion of values for most estimates, at a minimum sensitivity analyses should be conducted to explore the impact of more extreme estimates on the model output. As demonstrated by the experience with the RDT model reported here, varying the estimates from the mean value to the extremes can have a significant impact on model output, reversing possible policy conclusions. The uncertainty demonstrated by the divergence in expert opinion can be described by fitting appropriate probability distributions for probabilistic sensitivity analyses.
Some of the variability in responses is likely to be due to genuine heterogeneity of malaria across settings, therefore the stratification of malaria transmission in the models is likely to impact the assessments obtained from experts and the accuracy of the results. Models could, for instance, broaden the stratification beyond two or three transmission levels to allow for greater geographical specificity. The need for more geographically focused estimates is perhaps even greater with non-malarial illnesses, where not only HIV prevalence is variable, but also that of other zoonoses such as rickettsial illnesses and leptospirosis, and endemic and epidemic pathogens such as S. typhi
and N. meniningitidis
. This variation will, of course, lead to variation in the progression to severe disease and death outcomes as well. Another approach to handling the diverse range of opinions would be to create interactive models, which would allow stakeholders to enter their own estimates for some of the more contentious parameters and produce results relevant to local aetiology and their own circumstances 
. Regardless of the approach, the output of any model that utilizes parameters with such a wide range of estimates should be interpreted with caution.
As with all Delphi studies, this one had potential limitations. The Delphi survey approach, including the development of the questions, selection of panellists, processing of feedback, and determination of the number of rounds of questioning, tends to be subjective, although less subjective than the usual way experts are chosen for modelling studies (the reason this study was undertaken). For this survey, 27 panellists were invited. Our panellists were selected for their expertise in malaria in particular, but also their wider clinical and research experience with non-malarial illnesses, including in bacterial diseases. Although only 21 completed the second round of questioning, it is unlikely that including the additional 6 would have changed our results. No panel will be exactly representative of world opinion, but whilst it is likely that a different sample of experts would produce different results, it is unlikely that the opinions of a different group would converge, particularly if a large, diverse panel was selected. For the malaria-related estimates, the diversity of opinions seems to reflect genuine uncertainty and strikingly diverse views on what could ultimately be better defined estimates of health outcomes of untreated malaria (notwithstanding some variation due to factors not accounted for in the questionnaire, such as co-morbidities). For the non-malarial illnesses, it is likely that the diversity of opinions partly reflects actual epidemiological heterogeneity.
There is no standard number of experts required for Delphi surveys, although the number recruited for this study is at the high end of most clinical Delphi surveys. The selection of panellists may determine the range and nature of views expressed in the surveys 
. Bias may be introduced by the initial selection of the panellists, or by non-completion. Some participants expressed discomfort with providing answers without supporting evidence, and in some instances did not provide an estimate for a particular parameter- although models are constructed with estimates from experts who are prepared to venture an opinion. The variability in opinions could partly result from patient level heterogeneity in factors that were not included in the survey, such as nutritional status and concomitant illness. Decision models however will seldom be able to capture such detail in their own structure. The stratification by age, malaria transmission intensity and HIV included in this survey were as detailed or more so than currently found in decision and mathematical models.
The questionnaire revision process, unique to Delphi surveys, may be subject to bias. In this survey, decisions to revise questions were driven by the frequency of comments, relevance to policy decisions, and practical considerations. This Delphi survey was halted after two rounds due to lack of convergence on most questions, with no indication that this would occur in further rounds. In the past, four rounds were considered ideal for Delphi surveys, although in more recent studies two or three rounds are accepted as sufficient. The decision on the number of rounds employed tends to be pragmatic, but may impact on the final results 
. There is usually a trade-off between a higher number of rounds (which potentially increases convergence, improving the data) and falloff in response rate (which reduces its usefulness). Delphi surveys aim to reach consensus; however our results suggested that convergence was unlikely to be achieved for most parameters included in this survey. The limitations of this study should not be overstated however; the results reflect the variety of opinion there is in reality, and demonstrate the shortcomings of models which tend to be based on one or at most a small number of experts, who generally are drawn from a far narrower pool for any given model than in this survey, and are only sampled once. In this survey, the wide variation of opinions likely resulted at least in part from the variability of malaria epidemiology at the sites where the expert panel members work, and is a testimony to the complexity of this infection.
Decision and economic models are widely used in policy decisions, but depend on the key parameters used being right, or at least in roughly the right place. This study provides an expert panel view of key data points, but demonstrates a striking dispersion in expert opinion on the parameters which have a significant impact on most existing malaria treatment decision models and models of diagnosis of febrile disease in malaria-endemic countries. Wherever possible real data rather than expert opinion should be used in models. Where this is not possible, the lack of clear consensus on most of the parameters and the wide range of estimates suggests that expert opinion should be used cautiously in decision models, and should always be supported by appropriate sensitivity analyses including the range of opinions shown in this study.