The Delphi technique is an approach used to gain consensus among a panel of experts in order to address a lack of agreement or incomplete state of knowledge [9
]. The technique was adopted here to develop consensus on probabilities of different causes of death occurring at the population level and probabilities of specific signs and symptoms presenting themselves at the population level and in specific causes of death. The technique was also utilised to develop consensus on key conceptual issues of cause of death classification and VA usage.
An expert group convened over four consecutive days. The group was comprised of five physicians (YB, TC, KK, LM, DDV) with extensive clinical experience in resource-poor settings. They represented a range of important disciplines of medicine: surgery; maternal and reproductive health; paediatrics; and internal medicine. The experts came from a range of settings in developing or transitional countries where routine death registration is often absent (South Africa, Ethiopia, The Gambia and Vietnam). It was felt that the range of backgrounds and geographical spread of the expert group would lead to a generalised consensus not specific to any one region or medical discipline. Each member of the expert group was either experienced in or very familiar with the process, importance and limitations of VAs and all were briefed on the probabilistic approach to VA interpretation.
The researchers facilitated discussions in which the experts were requested to consider the inclusion of indicators and causes of death in the model, bearing in mind that friends or relatives of the deceased person must be able to notice and report indicators to the lay fieldworkers [5
]. A list of 34 possible causes of death and 104 indicators was developed (). Probabilities were agreed upon and assigned to each indicator and cause of death at the population level and for each specific cause of death using on a semi-qualitative scale following work by Kong et al (1986) [11
] (). A higher degree of precision was not sought since previous work suggests that this is not essential in order to build a workable model [12
Verbal autopsy indicators and causes of death used in the refined model.
The semi-qualitative scale used for assigning probabilities of indicators and causes in the refined model.
There was strong consensus among the physicians that probabilities of causes of death with large variations in prevalence at the population level between regions, such as HIV/AIDS and malaria, should have the possibility of being adjusted in the model to reflect the local burden of these diseases at the population level. To warrant adjustment of the database it was felt that regional variations of disease prevalence should be at least ten-fold. It was not felt necessary to adjust the database to reflect regional variations in causes of death with very specific indicators, such as meningitis or transport accidents. The revised model therefore included a facility to reflect either high or low prevalence for HIV/AIDS and malaria.
The model was updated using Visual FoxPro database software to make adjustments to probabilities and removal or insertion of various causes and indicators. The revised model's output was modified to only show more than one cause of death if the probability of the additional cause(s) was within 20% of the most likely cause. This is in contrast to the preliminary model, which always gave the three most likely causes irrespective of probabilities. The model was also adjusted so that certain causes of death were extremely unlikely to be diagnosed without the presence of specific indicators. For example, it is thus highly unlikely that the model will conclude that death resulted from diarrhoeal disease without the symptom of diarrhoea being reported. Each member of the expert group was provided with a working prototype of the model and given the opportunity to test it on hypothetical cases to highlight any inconsistencies and anomalies.
The updated probabilistic model was applied to the VA data from the same 189 Vietnamese cases used to validate the preliminary model. Indicators were gathered from the original VA questionnaires and included open-ended, free- text information. These data and the underlying VA process used in Vietnam are described in detail elsewhere [3
]. Comparisons were made with the cause of death as previously agreed by the two local physicians in Vietnam and with the results from the preliminary model.
Many studies aiming to validate VA interpretation methodologies against hospital records or physician review describe sensitivities, specificities and positive predictive values (PPV)[1
]. However, the calculation of such statistics assumes that the referent diagnosis gives the right answer and is an absolute gold standard. This assumption is flawed due to the inconsistencies of physician review and studies describing sensitivity, specificity and PPV of VA methods often discuss the possibility that in certain cases the VA diagnosis may be more accurate than the diagnosis provided by physician review or hospital records [13
]. As such, it was considered inappropriate to calculate the sensitivities, specificities or PPV for the probabilistic model in this validation study. Instead, kappa (κ) values are calculated since they simply reflect the level of agreement between the two methods and do not imply superiority of one method over the other.