We illustrate the various procedures that Meta-DiSc implements in a case-study of ultrasound test in the diagnosis of uterine pathology [21
]. Ultrasound measurement of the lining of the uterus (endometrium) can predict pathology such as endometrial hyperplasia (a precancerous condition) or cancer. The greater the thickness of endometrium, the more likely that the target condition is present. Various thresholds (such as 3, 4 or 5 mm etc) have been used to define a positive ultrasound result.
A systematic review of test accuracy studies identified 57 studies. Figure shows a datasheet in Meta-DiSc which has been loaded with information from these 57 studies. The information includes study identifiers, accuracy data, thresholds, and some study level co-variates (such as hormone replacement therapy use).
Meta-Disc datasheet. Meta-DiSc data set with details of test accuracy studies of ultrasound in the prediction of endometrial cancer.
As the first step in the analysis, we have used Meta-DiSc to present accuracy measures from each individual study in forest plots for sensitivities (figure ), specificities (figure ), LRs (figures and ) and dOR (figure ). All these indices can also be represented in tabular form as shown in table . Although the forest plots and the tables contain a pooled summary at the bottom, at this early stage in the analysis, it is recommended that the plots are used to obtain a general overview of the accuracy estimates from each study, and the interpretation of the pooled summary is left to later stages of analysis.
Forest plot. Forrest plot of sensitivities (3a) and specificities (3b) from test accuracy studies of ultrasound in the prediction of endometrial cancer.
Forest plot. Forrest plot of likelihood ratios for positive (4a) and negative (4b) test results from studies of ultrasound in the prediction of endometrial cancer.
Forrest plot. Forest plot of diagnostic odds ratios (dOR) from test accuracy studies of ultrasound in the prediction of endometrial cancer.
Tabulation of Likelihood ratio for positive test result (LR+) with respective 95% confidence intervals from all test accuracy studies included in systematic review of ultrasound for prediction of endometrial cancer.
The next step is the representation of sensitivity against 1-specificity from each study in a ROC space (figure ), which can be used for exploration for threshold effect. The pattern of the points in this plot suggest a "shoulder-arm" shape, indicating the possibility of threshold effect. We, therefore, performed a Spearman rank correlation as a further test for threshold effect, and found that there was further indication of threshold effect (Table , Spearman correlation coefficient = 0.394; p = 0.006). Having found some clues about the presence of threshold effect, we now focus on a subgroup of 21 studies that used a singular threshold of >5 mm to define test positivity. Although an explicit
threshold of 5 mm was used in these studies, there can still be an implicit
threshold effect due to, for example, variation in the interpretation of the test results. Therefore, within this subgroup with an explicit threshold of 5 mm, it is still recommended that the above explorations for threshold effect are undertaken. We performed such analyses for this subgroup in Meta-DiSc, and found no evidence of further threshold effect (data not shown). There are a number of other more advanced methods not implemented in Meta-DiSc that allow to incorporate explicitly information about tests thresholds defined between or within studies [17
ROC Space. Representation of sensitivity against (1-specificity) in Receiver Operating Characteristics space for each study of ultrasound in the prediction of endometrial cancer.
Results of Spearman rank correlation of sensitivity against (1 – specificity) to assess the threshold effect in all test accuracy studies included in systematic review of ultrasound for prediction of endometrial cancer.
As the next step, heterogeneity arising from factors other than threshold effect is explored. We performed a visual exploration of the forest plots of accuracy measures for these 21 studies as well as statistical tests for heterogeneity (Meta-DiSc output not shown). In addition, possible sources of heterogeneity across the studies were explored using meta-regression analysis with the following co-variates as predictor variables: use or non-use of hormone replacement therapy (HRT); technique of ultrasound measurement (single or double layer); and population enrolment (consecutive or other). Results are shown in Table , which suggest that the number of layers is strongly associated with accuracy. The double layer technique is associated with two times higher accuracy compared to single layer measurement (rdOR = 2.04; 95% CI: 1.01–4.13; p = 0.048)
Table 4 Results of meta-regression analysis for predicting the presence or absence of endometrial carcinoma with variables: use or non-use of hormone replacement therapy (HRT); technique of ultrasound measurement (single or double layer); and population enrolment (more ...)
The final step in the analysis is pooling if this is considered appropriate. We illustrate pooling of the LRs for negative test results in one homogenous subgroup of studies of non-HRT users, with a test threshold of ≤ 5 mm, and using a single layer technique (Figure ). Finally, we demonstrate sROC curve fitting in the presence of threshold effect for the whole data-set in Figure .
Forrest plot. Forrest plots of Likelihood ratios for positive (7a) and negative (7b) test results in one homogenous subgroup of studies of non-HRT users, with a test threshold of ≤ 5 mm, and using a single layer technique.
sROC curve. Receiver operating characteristics curve for all studies included in systematic review of ultrasound for prediction of endometrial cancer.