Search tips
Search criteria 


Logo of jnciLink to Publisher's site
J Natl Cancer Inst. 2011 January 19; 103(2): 129–142.
PMCID: PMC3022618

Contemporary Diagnostic Imaging Modalities for the Staging and Surveillance of Melanoma Patients: a Meta-analysis



Meta-analyses were performed to examine the utility of ultrasonography, computed tomography (CT), positron emission tomography (PET), and a combination of both (PET-CT) for the staging and surveillance of melanoma patients.


Patient-level data from 74 studies containing 10 528 patients (between January 1, 1990, and June, 30, 2009) were used to derive characteristics of the diagnostic tests used. Meta-analyses were conducted by use of Bayesian bivariate binomial models to estimate sensitivity and specificity. Diagnostic odds ratios [ie, true-positive results/false-negative results)/(false-positive results/true-negative results)] and their 95% credible intervals (CrIs) and positive predictive values were used as indicators of test performance.


Among the four imaging methods examined for the staging of regional lymph nodes, ultrasonography had the highest sensitivity (60%, 95% CrI = 33% to 83%), specificity (97%, 95% CrI = 88% to 99%), and diagnostic odds ratio (42, 95% CrI = 8.08 to 249.8). For staging of distant metastases, PET-CT had the highest sensitivity (80%, 95% CrI = 53% to 93%), specificity (87%, 95% CrI = 54% to 97%), and diagnostic odds ratio (25, 95% CrI = 3.58 to 198.7). Similar trends were observed for melanoma surveillance of lymph node involvement, with ultrasonography having the highest sensitivity (96%, 95% CrI = 85% to 99%), specificity (99%, 95% CrI = 95% to 100%), and diagnostic odds ratio (1675, 95% CrI = 226.6 to 15,920). For distant metastases, PET-CT had the highest sensitivity (86%, 95% CrI = 76% to 93%), specificity (91%, 95% CrI = 79% to 97%), and diagnostic odds ratio (67, 95% CrI = 20.42 to 229.7). Positive predictive values were likewise highest for ultrasonography in lymph node staging and for PET-CT in detecting distant metastases.


Among the compared modalities, ultrasonography was superior for detecting lymph node metastases, and PET-CT was superior for the detection of distant metastases in both the staging and surveillance of melanoma patients.


Prior knowledge

Melanoma may recur in up to 50% of melanoma survivors, especially during the first years after diagnosis. Positron emission tomography (PET) and a combination of PET and computed tomography (CT) (PET-CT) have gained acceptance as the imaging modalities to identify recurrence in survivors and to stage lymph nodes and metastatic melanoma but evidence-based data on their risks and benefits are scarce.

Study design

Meta-analysis of patient-level data from published studies was used to derive characteristics (sensitivity, specificity, diagnostic odds ratio, and positive predictive value) of the four diagnostic imaging modalities.


Among the four imaging methods examined, for regional lymph node staging, ultrasonography had the best performance. For staging of distant metastases, PET-CT had the best performance. Similar patterns were observed for surveillance of melanoma survivors for lymph node involvement and for distant metastases.


The superior modality for lymph node staging and detecting lymph node involvement was ultrasonography. The superior modality for staging and detecting distant metastases was PET-CT.


Diagnostic criteria and the quality of the imaging equipment for each modality varied during the period studied. Most studies included in this meta-analysis had a retrospective design.

From the Editors

Staging guidelines for distant metastatic melanoma published by the National Comprehensive Cancer Network indicate that melanoma patients with regional lymph node involvement (ie, American Joint Committee on Cancer stage III) (1) should undergo diagnostic imaging at the time of diagnosis (2), although the detection rate is low particularly in subsets of asymptomatic patients with microscopically detected disease in the lymph nodes (stage IIIA) (3). The guidelines further state that those with less advanced disease (ie, stage IA or II) with clinical indications (ie, suspected or palpable lymph nodes) should be considered for imaging. Although sentinel lymph node biopsy is the acknowledged gold standard for pathological staging of clinically lymph node–negative patients (4), in some clinical settings, ultrasound has also been used for preoperative lymph node assessment and postoperative surveillance (5,6). Positron emission tomography (PET) and a combination of PET and computed tomography (CT) (PET-CT) have rapidly gained acceptance as the imaging modalities of choice for identifying metastatic melanoma but have often been applied without regard to tumor-specific risk strata or known benefits (710). Given the limited nature of health-care resources, it is critical to examine these and other new technologies as they emerge.

In 2006, the number of melanoma survivors in the United States was estimated to be more than four million (11), largely as a result of the successful treatment of most patients with newly diagnosed early-stage melanoma (12). In up to 50% of these patients, however, the tumor may recur (1315), with the risk of first recurrence being greatest in the initial years after diagnosis (1618). It has been estimated that 20% of all first recurrences occur locally, 50% occur in the regional lymph nodes, and 30% arise at distant sites (1922). Although surgical resection continues to be the standard treatment for local and regional recurrences, reports of surgical resection or metastasectomy for distant recurrence in select patients have also been associated with improved survival (2327). These optimistic reports of survival after salvage surgical resection of melanoma recurrences offer a rationale for defining optimal follow-up strategies. Despite the benefit of early detection of locoregional (19,28,29) or distant (2327) recurrences in these patients, there are no evidence-based guidelines for their surveillance, and clinical practice patterns vary widely.

Currently, the most commonly used imaging modalities for melanoma patients include ultrasonography, CT, PET, and PET-CT. The proposed advantage of PET-CT is that differences in metabolism and function can be detected that complement anatomical imaging techniques (30). Several studies (710,31) have reported characteristics of the individual diagnostic imaging tests for the evaluation of melanoma recurrences. However, the utility of each modality as applied to various clinical scenarios has not been examined, and the modalities have not been directly compared. The objective of this meta-analysis was to analyze the contemporary literature related to diagnostic imaging in melanoma patients and to compare the test characteristics of various imaging modalities, such as ultrasonography, CT, PET, and PET-CT, for the staging and surveillance of patients with melanoma.

Patients, Studies, and Methods

Studies and Patients Included in the Meta-Analysis

A comprehensive literature search of MEDLINE (from January 1, 1990, through June 30, 2009), EMBASE (from January 1, 2001, through June 30, 2009), Cancerlit (from January 1, 1990, through October 31, 2002), and the Controlled Trials Register from the Cochrane Library (from January 1, 1990, through June 30, 2009) was performed with the following keywords: “melanoma”; “lymph node metastasis”; “ultrasound”; “computed tomography”; “positron-emission tomography”; and “positron emission tomography with computerized tomography.” Articles identified from the search were reviewed in detail and included in the analysis if they met following criteria: 1) included more than 10 patients with melanoma and 2) included comparisons of single or multiple imaging modalities (ie, ultrasonography, CT, PET, and/or PET-CT) to a gold standard. For primary staging of regional lymph nodes, sentinel lymph node biopsy with pathological confirmation is the gold standard for clinically lymph node–negative patients (2,5,32). For surveillance studies, a minimum of 6 months of follow-up was required for clinical confirmation. No language restrictions were applied, and additional references in identified articles were also reviewed for inclusion.

The literature search yielded 1096 unique citations. In total, 1020 (93%) citations were excluded. The two most common reasons for exclusion were inadequate reporting of patient-level data that were required to calculate test characteristics and/or lack of a reported gold standard. Two reports that met inclusion criteria were excluded because of overlapping study populations (33,34). Thus, 74 studies containing 10 528 patients were included in this meta-analysis.

Patient-level data were extracted and used to construct two-by-two tables. Each melanoma patient who was included in this study had undergone ultrasonography, CT, PET, or PET/CT. Their test results had been classified as true positive, true negative, false negative, or false positive by the histological analysis of lymph node specimens or distant metastasis specimens or by the outcome after long-term follow-up (ie, >6 months) as the gold standards. Diagnostic test characteristics were analyzed according to standard definitions for each individual study: sensitivity [TP/(TP + FN)], specificity [TN/(FP + TN)], false-negative rate [FN/(FN + TP), or 1 − sensitivity], and the positive predictive value [TP/(TP + FP)] (where TP is the number of patients with a true-positive result, TN is the number of patients with a true-negative result, FP is the number of patients with a false-positive result, and FN is the number of patients with a false-negative result). Accuracy was calculated as [(TP + TN)/number of patients in the study. Several studies reported true-positive, false-positive, false-negative, and true-negative results at the patient level, whereas others reported these values at the lesion or assessment level. In these instances, independence between lesions on the same image and different assessments for the same patient were assumed.

After data abstraction, two raters (Y. Xing and J. N. Cormier) independently assessed the quality of the included studies by use of the Quality Assessment of Diagnostic Accuracy Studies (QUADAS) scale (35). Discrepancies were resolved by consensus or third-party review (by R. L. Askew). This scale contains 14 items that examine potential sources of bias in diagnostic studies, with one point assigned for each criterion satisfied. The questions related to the representativeness of the sample, selection criteria, and the appropriateness of the reference standard test. Higher scores reflect higher quality, and no articles were excluded because of the assigned quality score.

Statistical Analysis

Bayesian Bivariate Binomial Model

To provide overall summary estimates for sensitivity and specificity for each imaging modality, Bayesian bivariate binomial models were applied that were similar to those proposed by Chu and Cole (36). This model assumed a binomial distribution for the number of patients with true-positive and true-negative results and allowed the inclusion of covariates and random effects. The inherent association between sensitivity and specificity was modeled in the bivariate normal distribution by assuming random effects. The full model can be expressed as:

An external file that holds a picture, illustration, etc.
Object name is jncidjq455fx1_ht.jpg

where i represent the individual diagnostic studies nl,i = TPi + FNi (ie, the number of subjects with the disease), n0,i = TNi + FPi (ie, the number of subjects without the disease), Xi and Zi were vectors of covariates related to specificity and sensitivity, respectively, α and β were the corresponding regression coefficients, and μ and ν are the variables representing random effects, N is normal distribution, σ2 is the between-study variance, and ρ is correlation coefficient.

To fully specify the model, the following vague previous probability distribution were assigned to the parameters:

An external file that holds a picture, illustration, etc.
Object name is jncidjq455fx2_ht.jpg

Separate analyses were conducted for detecting lymph node and distant metastases, but ultrasound imaging was not included in the model for detecting distant metastasis. Both models included the specific surveillance tests used to study melanoma patients (eg, ultrasonography, CT, PET, and PET/CT) as covariates along with other clinically important covariates to account for between-study heterogeneity, including study design (eg, prospective vs retrospective), reason for diagnostic imaging (eg, primary staging only, restaging, or both), and the level of tumor assessment (eg, patient or lesion). Sensitivity, specificity, and diagnostic odds ratios were calculated as conditional probabilities that were identified as specific values for the covariates in the model. The diagnostic odds ratio, defined as [(TP/FN)/(FP/TN)], was used as an indicator of test performance because it combines information from all four statistical cells. Its value ranges from zero to infinity, with a higher value indicating better discriminatory power. A value of 1.0 is expected for tests with no difference detected between disease and nondisease groups (37). The 95% credible intervals (CrIs), or Bayesian confidence intervals, which are equivalent to the frequentist’s confidence interval, were calculated for sensitivity, specificity, and diagnostic odds ratios. Positive predictive values were also calculated for each of the diagnostic modalities by use of the following formula: (sensitivity × prevalence)/{(sensitivity × prevalence) + [(1 − specificity) × (1 − prevalence)]}, with 5-year recurrence serving as prevalence estimates (38). The three prevalence risk categories for lymph node and distant metastasis were defined as follows for the calculation of the positive predictive value: low = 5%, intermediate = 15%; and high = 30%.

Model Implementation

Bayesian bivariate binomial models were constructed with Markov chain Monte Carlo methods by use of WinBUGS, version 1.4.2 (39). Each covariate was centered about its mean to ensure approximate previous independence between the regression coefficients and good convergence of the three Markov chains (39). The first 10 000 draws were discarded, and only the second 10 000 draws were used to obtain posterior estimates that were based on three separate chains with overdispersed starting values. The Brooks, Gelman, and Rubin convergence statistics (40) were used to assess model convergence, and only properly converged models were further considered. The results were based on 30 000 draws, and R-hat (40) for all parameters was equal to 1.0 in two models, indicating model convergence.

Funnel plots were created to assess publication bias by examining the relationship between the effect measure (log diagnostic odds ratio) and its standard error. Standard error serves as a good proxy for sample size because of their inversely proportional relationship; a small value reflects high precision for an effect size estimate. Conversely, smaller studies are more likely to exhibit a larger spread around the summary estimate of effect size (41). Egger tests were used to assess asymmetry of the funnel plot and to quantitatively assess bias (42). For this analysis, P values were two-sided and statistical significance was defined as a P value of less than or equal to .05. These analyses were performed with Stata, version 10.0 (Stata Corporation, College Station, TX).


Quality assessment scores for the diagnostic studies were calculated as the number of individual criteria satisfied from the 14-point QUADAS scale (35). Figure 1 shows the distribution of the quality rankings for the 74 studies included in this analysis, with an overall mean score of 5.8 (standard deviation = 2.5). Approximately 90% of the studies had a total quality score of less than 9.0, and the majority of articles satisfied items pertaining to details of the reference standard and index test. The most commonly unmet quality criteria related to insufficient detail when reporting patient withdrawals, intermediate results, and the selection and training of raters.

Figure 1
Histogram of the Distribution of Quality Assessment of Diagnostic Accuracy Studies (QUADAS) scores for diagnostic studies (n = 74).

Characteristics of the 74 studies as stratified by diagnostic imaging modality for the detection of lymph node or distant metastases are shown in Tables 1 and and2.2. Twenty-one studies met the inclusion criteria for ultrasonography (28,4362), 13 for CT (6375), 45 for PET (51,66109), and 13 for PET-CT (68,69,71,72,74,75, 94,110115). The median patient age for these studies was 55 years (range = 14–93 years); the percentage of men was 55 (range = 37%–79%), and the mean number of participants per study was 140 (range = 10–2008 participants per study). Patients were enrolled exclusively for the purposes of primary staging in 30 studies (48,51,52,5456,5862,70,83,8587,91,93,98106,108,109,114) or surveillance in 34 studies (28,4447,49,53,57,6365,6769,71,72,73,7577,8082,8789,92,9496,110,111,113,115).

Table 1
Published reports on diagnostic test characteristics of ultrasonography (US) and computed-tomography (CT) for staging and surveillance testing in patients with melanoma*
Table 2
Published reports on diagnostic test characteristics of positron emission tomography (PET) and PET–computed tomography (CT) for staging and surveillance testing of patients with melanoma*

Results of these 74 studies were also assessed according to the clinical intent of the tests (primary staging or follow-up surveillance) and the anatomical site of evaluation (lymph node or distant metastases) (Table 3 and Figure 2). For the staging of regional lymph nodes, ultrasonography had the highest sensitivity (60%, 95% CrI = 33% to 83%), specificity (97%, 95% CrI = 88% to 99%), and diagnostic odds ratio (42, 95% CrI = 8.08 to 249.8) compared with the respective values for CT (9%, 95% CrI = 1% to 52%; 92%, 95% CrI = 50% to 99%; and 1.13, 95% CrI = 0.04 to 33.25), PET (30%, 95% CrI = 12% to 55%; 96%, 95% CrI = 87% to 99%; and 9.45, 95% CrI = 1.89 to 48.12), and PET-CT (11%, 95% CrI = 1% to 50%; 97%, 95% CrI = 78% to 100%; and 4.39, 95% CrI = 0.21 to 94.2). For staging of distant metastases, PET-CT had the highest sensitivity (80%, 95% CrI = 53% to 93%), specificity (87%, 95% CrI = 54% to 97%), and diagnostic odds ratio (25, 95% CrI = 3.58 to 198.7) compared with the respective values for CT (51%, 95% CrI = 24% to 76%; 69%, 95% CrI = 30% to 92%; and 2.29, 95% CrI = 0 .34 to 14.98) and PET (74%, 95% CrI = 51% to 88%; 75%, 95% CrI = 45% to 91%; and 8.14, 95% CrI = 1.76 to 38.45).

Table 3
Estimates of sensitivity, specificity, and diagnostic odds ratio for the staging and surveillance of metastatic sites for ultrasonography (US), computed-tomography (CT), positron emission tomography (PET), and PET-CT*
Figure 2
Forest plots of the test characteristics of four diagnostic modalities. A) Forest plots for sensitivity and specificity for ultrasonography (US), computed tomography (CT), positron emission tomography (PET), and PET-CT for the clinical scenarios indicated. ...

Similar trends were observed for melanoma surveillance of lymph node involvement. Ultrasonography had the highest sensitivity (96%, 95% CrI = 85% to 99%), specificity (99%, 95% CrI = 95% to 100%), and diagnostic odds ratio (1675, 95% CrI = 226.6 to 15920) compared with the respective values for CT (61%, 95% CrI = 15% to 93%; 97%, 95% CrI = 70% to 100%; and 46.25, 95% CrI = 2 .27 to 1354), PET (87%, 95% CrI = 67% to 96%; 98%, 95% CrI = 93% to 100%; and 391, 95% CrI = 68 to 2737), and PET-CT (65%, 95% CrI = 20% to 93%; 99%, 95% CrI = 92% to 100%; and 196, 95% CrI = 10.77 to 4675) (Table 3). Likewise, positive predictive values for the surveillance of lymph nodes were consistently higher for ultrasonography among low-risk patients (positive predictive value = 83%, 95% confidence interval [CI] = 36% to 100%), intermediate-risk patients (positive predictive value = 94%, 95% CI = 68% to 100%), and high-risk patients (positive predictive value = 98%, 95% CI = 83% to 100%) (Table 4). For the surveillance of distant metastases, PET-CT had the highest sensitivity (86%, 95% CrI = 76% to 93%), specificity (91%, 95% CrI = 79% to 97%), and diagnostic odds ratio (67, 95% CrI = 20.42 to 229.7) compared with the respective values for CT (63%, 95% CrI = 46% to 77%; 78%, 95% CrI = 58% to 90%; and 6, 95% CrI = 2 to 17.83) and PET alone (82%, 95% CrI = 72% to 88%; 83%, 95% CrI = 70% to 91%; and 22, 95% CrI = 9.31 to 51) (Table 3). Positive predictive values for the surveillance of distant metastasis were consistently higher for PET-CT among patients at low risk (positive predictive value = 33%, 95% CI = 9% to 61%), intermediate risk (positive predictive value = 63%, 95% CI = 38% to 82%), and high risk (positive predictive value = 80%, 95% CI = 64% to 93%) (Table 4). Because of the higher number of false-positive results (ie, the lower specificity), the positive predictive values were lower for distant metastasis than for regional lymph nodes.

Table 4
Positive predictive value (PPV) estimates for surveillance for ultrasonography (US), computed tomography (CT), positron emission tomography (PET), and PET-CT, stratified by level of risk and metastatic site*

Funnel plots that demonstrate the effects of small study size for each diagnostic imaging modality are presented in Figure 3. Using diagnostic odds ratio as the effect measure, potential publication bias was identified for the studies examining ultrasonography because estimates from 11 of the 16 studies fell outside of the funnel. Results of the Egger test for small study effects were not statistically significant (P = .44), indicating that no trend toward higher levels of test accuracy was observed among studies with smaller sample sizes.

Figure 3
Funnel plot for diagnostic odds ratio (dOR) with standard error. A) Ultrasonography (US). B) Computed tomography (CT). C) Positron emission tomography (PET). D) PET-CT. Data from each modality are plotted against their standard error (SE). Solid line ...


The results of this meta-analysis indicate that when selecting among the four diagnostic imaging modalities examined, the anatomical site to be evaluated was more important than the clinical scenario (ie, staging or surveillance). Among the four diagnostic imaging modalities for the assessments of lymph node metastasis, ultrasonography was superior to CT, PET, and PET-CT. PET-CT had the highest positive predictive value for the surveillance of distant metastasis; however, the higher number of false-positive results (ie, lower specificity) from PET-CT lead to the loss of precision. Furthermore, for patients at low risk of metastasis, the positive predictive value of PET-CT (ie, 33%, 95% CI = 9% to 61%) indicated that use of PET-CT is not warranted without additional clinical indications.

Practice guidelines are becoming an increasingly important element in disseminating treatment algorithms to physicians who treat patients in a community setting (116123), and investigators have suggested that these guidelines can be used as a means of measuring the quality of care delivered (124). However, evidence-based surveillance strategies for survivors of most cancers including melanoma do not exist. A recent report (125) that was based on Surveillance, Epidemiology, and End Results–Medicare data acknowledges geographic and patient variation in the receipt of surveillance after treatment of primary melanoma. With the increasing number of melanoma survivors and rapid advances in health-care technology, the costs of caring for these survivors are rising (126, 127). In 1997, Mooney et al. (128) reported that screening for melanoma recurrence (in this report for asymptomatic pulmonary metastasis) accounted for approximately 80% of program costs, totaling between $27 and $32 million for a 20-year program. As technological advances permit us to more precisely determine metastatic tumor spread, physicians and patients alike are faced with making clinical decisions on the basis of contemporary risk assessment. Nevertheless, controversy continues to surround the optimal imaging modality and interval of patient surveillance.

Sentinel lymph node biopsy is the acknowledged gold standard for the pathological staging of clinically lymph node–negative melanoma (3,5). A recent study by Sanki et al. (5) comparing ultrasonography with sentinel lymph node biopsy found that the sensitivity of targeted high-resolution ultrasound was only 24.3% (95% CI = 19.5% to 28.7%) compared with that of the sentinel lymph node biopsy. The combination of preoperative ultrasound and fine needle biopsy in select high-risk patients can, however, eliminate the need for sentinel lymph node biopsy by preoperatively identifying lymph node metastases, which indicate the need for therapeutic lymph node dissection (52,129,130). The primary utility of ultrasonography for the assessment of metastases in regional lymph nodes is for lymph node surveillance (31,46,131). PET-CT was superior for detection of distant metastases. Given the low positive-predictive value of CT, PET, and PET-CT in the surveillance of patients at low risk of lymph node metastasis, ultrasonography is the only justifiable imaging choice for lymph node surveillance.

The overall point estimates for the diagnostic test characteristics in this study are lower than those reported in two recently published prospective studies (6,132) that evaluated the utility of ultrasonography, CT, and PET in primary staging. Voit et al. (6) reported that ultrasonography combined with fine needle aspiration cytology had a sensitivity of 65% and a specificity of 99% in a cohort of 400 consecutive melanoma patients. Another study (132) reported the sensitivities of PET and CT for 251 patients with clinically palpable (stage III) lymph nodes as 86% and 78%, respectively, with a specificity of 94% for both tests. These discrepancies likely relate to heterogeneity among patient populations in these studies.

The purpose of staging and surveillance is to detect treatable tumors, monitor success of therapy, and provide reassurance and support to patients (133,134). However, these benefits must be balanced with the risks of testing to patients and their associated costs. Costs for CT, PET, and PET-CT can often be more than twice that of ultrasonography, with differences in charges of up to four times more. Although sufficient evidence regarding clinical effectiveness is not yet available to justify the use of new technologies, such as PET-CT, instead of the best existing alternatives, they are already widely used in oncology. Imaging is one of the fastest growing health-care services (135) and is a prime example of technology that must be examined in the context of comparative effectiveness to “improve the quality and affordability of US health care” (136). Quality medical care has been summarized by Earle et al. (137) as the “delivery of optimal health services” (138), with “technical proficiency” (139); “avoiding overuse, underuse, or misuse of technologies”(140); and “incorporating patient centered preferences in shared decision making” (141).

Inappropriate imaging, which adds to health-care costs without improving the quality of care, has been attributed to both physician and patient factors (142). Lack of knowledge (143) and fear of liability for missed diagnoses (144) attributed to physicians have commonly resulted in the inappropriate use of imaging. In addition, patients with a newly diagnosed cancer often expect certain examinations (145), particularly whole-body imaging. A negative imaging result, even when unnecessary, is often reassuring for the patient and physician and is often perceived to come with few if any negative consequences. However, levels of radiation exposure are known to vary widely even with the same imaging modality, potentially leading to health consequences, including increased lifetime risk of cancer (146,147). Furthermore, incidental abnormalities that can be identified on by imaging that do not affect health but require additional evaluation (eg, further imaging or interventional procedures) can result in additional associated costs, complications, and patient anxiety (142).

Compared with previous meta-analyses (710,31) that examined test characteristics of diagnostic imaging modalities in patients with melanoma, this analysis has a number of strengths. First, all eligible studies from January 1, 1990 through June 30, 2009 with sufficient data on four widely used contemporary imaging modalities (ie, ultrasonography, CT, PET, and PET-CT) were examined. Patient-level data from these studies were extracted and analyzed according to specific clinical scenarios (eg, initial staging vs surveillance); these data have been reported by few studies, despite the large potential impact of diagnostic imaging on both the quality and cost of medical care (148). Second, Bayesian bivariate binomial models were used for the meta-analysis of diagnostic test characteristics to capture the variability in both sensitivity and specificity simultaneously, as well as their intercorrelation. Such models are applicable to both large and small studies without ad hoc correction (36). Because of the methodological advantages of bivariate models, Harbord et al. (149) have recommended that such models be considered standard methods for meta-analysis of diagnostic accuracy.

This study has several limitations that must also be considered. First, technology has advanced over the last two decades, and the diagnostic criteria for each modality have varied during the period studied. Second, selection bias and work-up bias inherent to each individual study could be considerable in this pooled analysis because most of the studies of patients undergoing the index test were retrospective in design. Third, partial verification bias may exist when only those patients undergoing a reference test are included in a sample, and no data were reported on the remaining patients who only underwent the index test. Another well-described drawback of meta-analyses is publication bias because studies with favorable results have a higher likelihood of being published than those with unfavorable results. The studies examining the diagnostic accuracy of ultrasonography reported widely varying estimates of sensitivity and specificity ranging from 5% to 100% with similar variations observed for PET imaging. There are several potential explanations for such variation including small sample sizes in some studies, differing study designs, varying quality of imaging equipment, and differing imaging criteria for diagnosis. An inherent strength of a meta-analysis in evaluating a large body of literature is that it can overcome limitations of small sample sizes and heterogeneous designs of individual trials by pooling the data and obtaining summary sensitivities and specificities.

With the ever-increasing number of melanoma survivors and limited health-care resources, the need to tailor current consensus-based National Comprehensive Cancer Network guidelines toward an evidence-based cost-effective surveillance program is becoming increasingly critical. Test characteristics and performance are considered the first two levels of the evidence hierarchy for all diagnostic technologies (150). The objective of this analysis was to use contemporary techniques of meta-analysis to summarize the existing evidence for four common diagnostic imaging modalities that are used in the staging and surveillance of regional and distant metastasis for patients with melanoma. Future comparative effectiveness analyses should use decision-analytic modeling to simulate the effectiveness and cost-effectiveness of various surveillance strategies with respect to imaging modality and frequency on stage-specific patient outcomes.

In summary, when diagnostic imaging is indicated for staging or surveillance, we found that ultrasonography was the best diagnostic imaging test to detect lymph node metastases and that PET-CT was more suitable for the detection of distant metastases in patients at intermediate or high risk or when distant metastases are clinically indicated. Results of this meta-analysis should provide information for clinical decisions on the staging and surveillance of patients with melanoma.


National Cancer Institute, National Institutes of Health (1 R01 CA127328-01 to J.N.C., PI).


Conflict of interest statement: None declared.

The contents of this manuscript are the sole responsibility of the authors and do not necessarily represent the official views of the National Institutes of Health. The authors had full responsibility for the design of the study, the collection of the data, the analysis and interpretation of the data, the decision to submit the manuscript for publication, and the writing of the manuscript.

We would like to acknowledge Greg Pratt, MS, for assistance with the systematic review and document retrieval and Debbie Dunaway for her assistance with document retrieval and manuscript submission. Abstract was presented at the 46th Annual Meeting of the American Society of Clinical Oncology, June 2010, Chicago, IL. Y. Xing, Y. Bronstein, M. I. Ross, R. L. Askew, J. E. Lee, J. E. Gershenwald, R. Royal, J. N. Cormier. Diagnostic imaging modalities for the surveillance of melanoma patients: a meta-analysis.


1. American Joint Committee on Cancer. New York, NY: Springer; 2009. Melanoma of the Skin; pp. 325–344. In: Edge SB, Byrd DR, Compton CC, et al. eds. AJCC Cancer Staging Manual. 7th ed.
2. National Comprehensive Cancer Network: Clinical Practice Guidelines in Oncology.
3. Aloia TA, Gershenwald JE, Andtbacka RH, et al. Utility of computed tomography and magnetic resonance imaging staging before completion lymphadenectomy in patients with sentinel lymph node-positive melanoma. J Clin Oncol. 2006;24(18):2858–2865. [PubMed]
4. Balch CM, Cascinelli N. Sentinel-node biopsy in melanoma. N Engl J Med. 2006;355(13):1370–1371. [PubMed]
5. Sanki A, Uren RF, Moncrieff M, et al. Targeted high-resolution ultrasound is not an effective substitute for sentinel lymph node biopsy in patients with primary cutaneous melanoma. J Clin Oncol. 2009;27(33):5614–5619. [PubMed]
6. Voit CA, van Akkooi AC, Schafer-Hesterberg G, et al. Rotterdam criteria for sentinel node (SN) tumor burden and the accuracy of ultrasound (US)-guided fine-needle aspiration cytology (FNAC): can US-guided FNAC replace SN staging in patients with melanoma? J Clin Oncol. 2009;27(30):4994–5000. [PubMed]
7. Mijnhout GS, Hoekstra OS, van Tulder MW, Teule GJ, Deville WL. Systematic review of the diagnostic accuracy of (18)F-fluorodeoxyglucose positron emission tomography in melanoma patients. Cancer. 2001;91(8):1530–1542. [PubMed]
8. Schwimmer J, Essner R, Patel A, et al. A review of the literature for whole-body FDG PET in the management of patients with melanoma. Q J Nucl Med. 2000;44(2):153–167. [PubMed]
9. Krug B, Crott R, Lonneux M, et al. Role of PET in the initial staging of cutaneous malignant melanoma: systematic review. Radiology. 2008;249(3):836–844. [PubMed]
10. El-Maraghi RH, Kielar AZ. PET vs sentinel lymph node biopsy for staging melanoma: a patient intervention, comparison, outcome analysis. J Am Coll Radiol. 2008;5(8):924–931. [PubMed]
11. Horner MJ, Ries LAG, Krapcho M, et al. SEER Cancer Statistics Review, 1975-2006. Accessed November 1, 2008.
12. Jemal A, Murray T, Samuels A, et al. Cancer statistics, 2003. CA Cancer J Clin. 2003;53(1):5–26. [PubMed]
13. Leiter U, Meier F, Schittek B, Garbe C. The natural course of cutaneous melanoma. J Surg Oncol. 2004;86(4):172–178. [PubMed]
14. Poo-Hwu WJ, Ariyan S, Lamb L, et al. Follow-up recommendations for patients with American Joint Committee on cancer stages I-III malignant melanoma. Cancer. 1999;86(11):2252–2258. [PubMed]
15. MacCormack MA, Cohen LM, Rogers GS. Local melanoma recurrence: a clarification of terminology. Dermatol Surg. 2004;30(12, pt 2):1533–1538. [PubMed]
16. McEwan L, Smith JG, Matthews JP. Late recurrence of localized cutaneous melanoma: its influence on follow-up policy. Plast Reconstr Surg. 1990;86:527. [PubMed]
17. Slingluff CL, Jr., Dodge RK, Stanley WE, Seigler HF. The annual risk of melanoma progression. Implications for the concept of cure. Cancer. 1992;70(7):1917–1927. [PubMed]
18. Romero JB, Stefanato CM, Kopf AW, Bart RS. Follow-up recommendations for patients with stage I malignant melanoma. J Dermatol Surg Oncol. 1994;20(3):175–178. [PubMed]
19. Benvenuto-Andrade C, Oseitutu A, Agero AL, Marghoob AA. Cutaneous melanoma: surveillance of patients for recurrence and new primary melanomas. Dermatol Ther. 2005;18(6):423–435. [PubMed]
20. Soong SJ, Harrison RA, McCarthy WH, Urist MM, Balch CM. Factors affecting survival following local, regional, or distant recurrence from localized melanoma. J Surg Oncol. 1998;67(4):228–233. [PubMed]
21. Meier F, Will S, Ellwanger U, et al. Metastatic pathways and time courses in the orderly progression of cutaneous melanoma. Br J Dermatol. 2002;147(1):62–70. [PubMed]
22. Dicker TJ, Kavanagh GM, Herd RM, et al. A rational approach to melanoma follow-up in patients with primary cutaneous melanoma. Br J Dermatol. 1999;140:249–254. [PubMed]
23. Petersen RP, Hanish SI, Haney JC, et al. Improved survival with pulmonary metastasectomy: an analysis of 1720 patients with pulmonary metastatic melanoma. J Thorac Cardiovasc Surg. 2007;133(1):104–110. [PubMed]
24. Andrews S, Robinson L, Cantor A, DeConti RC. Survival after surgical resection of isolated pulmonary metastases from malignant melanoma. Cancer Control. 2006;13(3):218–223. [PubMed]
25. DiLuna ML, King JT, Jr., Knisely JP, Chiang VL. Prognostic factors for survival after stereotactic radiosurgery vary with the number of cerebral metastases. Cancer. 2007;109(1):135–145. [PubMed]
26. Young SE, Martinez SR, Essner R. The role of surgery in treatment of stage IV melanoma. J Surg Oncol. 2006;94(4):344–351. [PubMed]
27. Riker AI, Kirksey L, Thompson L, Morris A, Cruse CW. Current surgical management of melanoma. Expert Rev Anticancer Ther. 2006;6(11):1569–1583. [PubMed]
28. Garbe C, Paul A, Kohler-Spath H, et al. Prospective evaluation of a follow-up schedule in cutaneous melanoma patients: recommendations for an effective follow-up strategy. J Clin Oncol. 2003;21(3):520–529. [PubMed]
29. Rhodes AR. Cutaneous melanoma and intervention strategies to reduce tumor-related mortality: what we know, what we don’t know, and what we think we know that isn’t so. Dermatol Ther. 2006;19(1):50–69. [PubMed]
30. Popescu RA, Patel PM, Spencer J. Imaging and investigation of melanoma patients. In: Newton Bishop JA, Gore M, editors. Melanoma: Critical Debates. Oxford, UK: Blackwell Science Ltd; 2002. pp. 133–149.
31. Bafounta ML, Beauchet A, Chagnon S, Saiag P. Ultrasonography or palpation for detection of melanoma nodal invasion: a meta-analysis. Lancet Oncol. 2004;5(11):673–680. [PubMed]
32. Choi EA, Gershenwald JE. Imaging studies in patients with melanoma. Surg Oncol Clin N Am. 2007;16(2):403–430. [PubMed]
33. Saiag P, Bernard M, Beauchet A, et al. Ultrasonography using simple diagnostic criteria vs palpation for the detection of regional lymph node metastases of melanoma. Arch Dermatol. 2005;141(2):183–189. [PubMed]
34. Steinert HC, Voellmy DR, Trachsel C, et al. Planar coincidence scintigraphy and PET in staging malignant melanoma. J Nucl Med. 1998;39(11):1892–1897. [PubMed]
35. Whiting P, Rutjes AW, Reitsma JB, Bossuyt PM, Kleijnen J. The development of QUADAS: a tool for the quality assessment of studies of diagnostic accuracy included in systematic reviews. BMC Med Res Methodol. 2003;3:25. [PMC free article] [PubMed]
36. Chu H, Cole SR. Bivariate meta-analysis of sensitivity and specificity with sparse data: a generalized linear mixed model approach. J Clin Epidemiol. 2006;59(12):1331–1332. author reply 1332–1333. [PubMed]
37. Glas AS, Lijmer JG, Prins MH, Bonsel GJ, Bossuyt PM. The diagnostic odds ratio: a single indicator of test performance. J Clin Epidemiol. 2003;56(11):1129–1135. [PubMed]
38. Dawson B, Trapp RG. Basic and Clinical Biostatistics. Norwalk, CT: Appleton & Lange; 1990.
39. Spiegelhalter D, Thomas A, Best NG. WinBUGS Version 1.4 User Manual. Cambridge, UK: MRC Biostatistics Unit; 2003.
40. Gelman A, Rubin DB. Inference from iterative simulation using multiple sequences. Stat Sci. 1992;7:457–511.
41. Petitti DB. Meta-analysis, Decision Analysis, and Cost-effectiveness Analysis. 2nd ed. New York, NY: Oxford University Press; 2000.
42. Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ. 1997;315(7109):629–634. [PMC free article] [PubMed]
43. Prayer L, Winkelbauer H, Gritzmann N, et al. Sonography versus palpation in the detection of regional lymph-node metastases in patients with malignant melanoma. Eur J Cancer. 1990;26(7):827–830. [PubMed]
44. Tregnaghi A, De Candia A, Calderone M, et al. Ultrasonographic evaluation of superficial lymph node metastases in melanoma. Eur J Radiol. 1997;24(3):216–221. [PubMed]
45. Binder M, Kittler H, Steiner A, et al. Lymph node sonography versus palpation for detecting recurrent disease in patients with malignant melanoma. Eur J Cancer. 1997;33(11):1805–1808. [PubMed]
46. Uren RF, Howman-Giles R, Thompson JF, et al. High-resolution ultrasound to diagnose melanoma metastases in patients with clinically palpable lymph nodes. Australas Radiol. 1999;43(2):148–152. [PubMed]
47. Voit C, Mayer T, Kron M, et al. Efficacy of ultrasound B-scan compared with physical examination in follow-up of melanoma patients. Cancer. 2001;91(12):2409–2416. [PubMed]
48. Rossi CR, Scagnet B, Vecchiato A, et al. Sentinel node biopsy and ultrasound scanning in cutaneous melanoma: clinical and technical considerations. Eur J Cancer. 2000;36(7):895–900. [PubMed]
49. Brountzos EN, Panagiotou IE, Bafaloukos DI, Kelekis DA. Ultrasonographic detection of regional lymph node metastases in patients with intermediate or thick malignant melanoma. Oncol Rep. 2003;10(2):505–510. [PubMed]
50. Schmid-Wendtner MH, Paerschke G, Baumert J, Plewig G, Volkenandt M. Value of ultrasonography compared with physical examination for the detection of locoregional metastases in patients with cutaneous melanoma. Melanoma Res. 2003;13(2):183–188. [PubMed]
51. Hafner J, Schmid MH, Kempf W, et al. Baseline staging in cutaneous malignant melanoma. Br J Dermatol. 2004;150(4):677–686. [PubMed]
52. Hocevar M, Bracko M, Pogacnik A, et al. The role of preoperative ultrasonography in reducing the number of sentinel lymph node procedures in melanoma. Melanoma Res. 2004;14(6):533–536. [PubMed]
53. Machet L, Nemeth-Normand F, Giraudeau B, et al. Is ultrasound lymph node examination superior to clinical examination in melanoma follow-up? A monocentre cohort study of 373 patients. Br J Dermatol. 2005;152(1):66–70. [PubMed]
54. Starritt EC, Uren RF, Scolyer RA, Quinn MJ, Thompson JF. Ultrasound examination of sentinel nodes in the initial assessment of patients with primary cutaneous melanoma. Ann Surg Oncol. 2005;12(1):18–23. [PubMed]
55. Testori A, Lazzaro G, Baldini F, et al. The role of ultrasound of sentinel nodes in the pre- and post-operative evaluation of stage I melanoma patients. Melanoma Res. 2005;15(3):191–198. [PubMed]
56. Voit C, Kron M, Schafer G, et al. Ultrasound-guided fine needle aspiration cytology prior to sentinel lymph node biopsy in melanoma patients. Ann Surg Oncol. 2006;13(12):1682–1689. [PubMed]
57. Dalle S, Paulin C, Lapras V, et al. Fine-needle aspiration biopsy with ultrasound guidance in patients with malignant melanoma and palpable lymph nodes. Br J Dermatol. 2006;155(3):552–556. [PubMed]
58. Sibon C, Chagnon S, Tchakerian A, et al. The contribution of high-resolution ultrasonography in preoperatively detecting sentinel-node metastases in melanoma patients. Melanoma Res. 2007;17(4):233–237. [PubMed]
59. Schmid-Wendtner MH, Dill-Muller D, Baumert J, et al. Lymph node metastases in patients with cutaneous melanoma: improvements in diagnosis by signal-enhanced color Doppler sonography. Melanoma Res. 2004;14(4):269–276. [PubMed]
60. Rossi CR, Mocellin S, Scagnet B, et al. The role of preoperative ultrasound scan in detecting lymph node metastasis before sentinel node biopsy in melanoma patients. J Surg Oncol. 2003;83(2):80–84. [PubMed]
61. van Rijk MC, Teertstra HJ, Peterse JL, et al. Ultrasonography and fine-needle aspiration cytology in the preoperative evaluation of melanoma patients eligible for sentinel node biopsy. Ann Surg Oncol. 2006;13(11):1511–1516. [PubMed]
62. Schafer-Hesterberg G, Schoengen A, Sterry W, Voit C. Use of ultrasound to early identify, diagnose and localize metastases in melanoma patients. Expert Rev Anticancer Ther. 2007;7(12):1707–1716. [PubMed]
63. Buzaid AC, Sandler AB, Mani S, et al. Role of computed tomography in the staging of primary melanoma. J Clin Oncol. 1993;11(4):638–643. [PubMed]
64. Buzaid AC, Tinoco L, Ross MI, Legha SS, Benjamin RS. Role of computed tomography in the staging of patients with local-regional metastases of melanoma. J Clin Oncol. 1995;13(8):2104–2108. [PubMed]
65. Kuvshinoff BW, Kurtz C, Coit DG, et al. Computed tomography in evaluation of patients with stage III melanoma. Ann Surg Oncol. 1997;4(3):252–258. [PubMed]
66. Holder WD, Jr., White RL, Jr., Zuger JH, Easton EJ, Jr., Greene FL. Effectiveness of positron emission tomography for the detection of melanoma metastases. Ann Surg. 1998;227(5):764–769. discussion 769–771. [PubMed]
67. Swetter SM, Carroll LA, Johnson DL, et al. Positron emission tomography is superior to computed tomography for metastatic detection in melanoma patients. Ann Surg Oncol. 2002;9(7):646–653. [PubMed]
68. Finkelstein SE, Carrasquillo JA, Hoffman JM, et al. A prospective analysis of positron emission tomography and conventional imaging for detection of stage IV metastatic melanoma in patients undergoing metastasectomy. Ann Surg Oncol. 2004;11(8):731–738. [PMC free article] [PubMed]
69. Romer W, Nomayr A, Greess H, et al. Retrospective interactive rigid fusion of (18)F-FDG-PET and CT. Additional diagnostic information in melanoma patients. Nuklearmedizin. 2006;45(2):88–95. [PubMed]
70. Brady MS, Akhurst T, Spanknebel K, et al. Utility of preoperative [(18)]f fluorodeoxyglucose-positron emission tomography scanning in high-risk melanoma patients. Ann Surg Oncol. 2006;13(4):525–532. [PubMed]
71. Pfannenberg C, Aschoff P, Schanz S, et al. Prospective comparison of 18F-fluorodeoxyglucose positron emission tomography/computed tomography and whole-body magnetic resonance imaging in staging of advanced malignant melanoma. Eur J Cancer. 2007;43(3):557–564. [PubMed]
72. Iagaru A, Quon A, Johnson D, Gambhir SS, McDougall IR. 2-Deoxy-2-[F-18]fluoro-D-glucose positron emission tomography/computed tomography in the management of melanoma. Mol Imaging Biol. 2007;9(1):50–57. [PubMed]
73. Fuster D, Chiang S, Johnson G, et al. Is 18F-FDG PET more accurate than standard diagnostic procedures in the detection of suspected recurrent melanoma? J Nucl Med. 2004;45(8):1323–1327. [PubMed]
74. Reinhardt MJ, Joe AY, Jaeger U, et al. Diagnostic performance of whole body dual modality 18F-FDG PET/CT imaging for N- and M-staging of malignant melanoma: experience with 250 consecutive patients. J Clin Oncol. 2006;24(7):1178–1187. [PubMed]
75. Veit-Haibach P, Vogt FM, Jablonka R, et al. Diagnostic accuracy of contrast-enhanced FDG-PET/CT in primary staging of cutaneous malignant melanoma. Eur J Nucl Med Mol Imaging. 2009;36(6):910–918. [PubMed]
76. Gritters LS, Francis IR, Zasadny KR, et al. Initial assessment of positron emission tomography using 2-fluorine-18-fluoro-2-deoxy-D-glucose in the imaging of malignant melanoma. J Nucl Med. 1993;34(9):1420–1427. [PubMed]
77. Boni R, Boni RA, Steinert H, et al. Staging of metastatic melanoma by whole-body positron emission tomography using 2-fluorine-18-fluoro-2-deoxy-D-glucose. Br J Dermatol. 1995;132(4):556–562. [PubMed]
78. Blessing C, Feine U, Geiger L, et al. Positron emission tomography and ultrasonography. A comparative retrospective study assessing the diagnostic validity in lymph node metastases of malignant melanoma. Arch Dermatol. 1995;131(12):1394–1398. [PubMed]
79. Steinert HC, Huch Boni RA, Buck A, et al. Malignant melanoma: staging with whole-body positron emission tomography and 2-[F-18]-fluoro-2-deoxy-D-glucose. Radiology. 1995;195(3):705–709. [PubMed]
80. Damian DL, Fulham MJ, Thompson E, et al. Positron emission tomography in the detection and management of metastatic melanoma. Melanoma Res. 1996;6(4):325–329. [PubMed]
81. Nguyen AT, Akhurst T, Larson SM, Coit DG, Brady MS. PET Scanning with (18)F 2-fluoro-2-deoxy-D-glucose (FDG) in patients with melanoma. Benefits and limitations. Clin Positron Imaging. 1999;2(2):93–98. [PubMed]
82. Jadvar H, Johnson DL, Segall GM, et al. The effect of fluorine-18 fluorodeoxyglucose positron emission tomography on the management of cutaneous malignant melanoma. Clin Nucl Med. 2000;25(1):48–51. [PubMed]
83. Eigtved A, Andersson AP, Dahlstrom K, et al. Use of fluorine-18 fluorodeoxyglucose positron emission tomography in the detection of silent metastases from malignant melanoma. Eur J Nucl Med. 2000;27(1):70–75. [PubMed]
84. Paquet P, Henry F, Belhocine T, et al. An appraisal of 18-fluorodeoxyglucose positron emission tomography for melanoma staging. Dermatology. 2000;200(2):167–169. [PubMed]
85. Acland KM, O’Doherty MJ, Russell-Jones R. The value of positron emission tomography scanning in the detection of subclinical metastatic melanoma. J Am Acad Dermatol. 2000;42(4):606–611. [PubMed]
86. Tyler DS, Onaitis M, Kherani A, et al. Positron emission tomography scanning in malignant melanoma. Cancer. 2000;89(5):1019–1025. [PubMed]
87. Klein M, Freedman N, Lotem M, et al. Contribution of whole body F-18-FDG-PET and lymphoscintigraphy to the assessment of regional and distant metastases in cutaneous malignant melanoma. A pilot study. Nucl Med (Stuttg). 2000;39(3):56–61. [PubMed]
88. Stas M, Stroobants S, Dupont P, et al. 18-FDG PET scan in the staging of recurrent melanoma: additional value and therapeutic impact. Melanoma Res. 2002;12(5):479–490. [PubMed]
89. Cobben DC, Jager PL, Elsinga PH, et al. 3’-18F-fluoro-3’-deoxy-L-thymidine: a new tracer for staging metastatic melanoma? J Nucl Med. 2003;44(12):1927–1932. [PubMed]
90. Gulec SA, Faries MB, Lee CC, et al. The role of fluorine-18 deoxyglucose positron emission tomography in the management of patients with metastatic melanoma: impact on surgical decision making. Clin Nucl Med. 2003;28(12):961–965. [PubMed]
91. Reinhardt MJ, Kensy J, Frohmann JP, et al. Value of tumour marker S-100B in melanoma patients: a comparison to 18F-FDG PET and clinical data. Nuklearmedizin. 2002;41(3):143–147. [PubMed]
92. Harris MT, Berlangieri SU, Cebon JS, Davis ID, Scott AM. Impact of 2-deoxy-2[F-18]fluoro-D-glucose positron emission tomography on the management of patients with advanced melanoma. Mol Imaging Biol. 2005;7(4):304–308. [PubMed]
93. Horn J, Lock-Andersen J, Sjostrand H, Loft A. Routine use of FDG-PET scans in melanoma patients with positive sentinel node biopsy. Eur J Nucl Med Mol Imaging. 2006;33(8):887–892. [PubMed]
94. Mottaghy FM, Sunderkotter C, Schubert R, et al. Direct comparison of [18F]FDG PET/CT with PET alone and with side-by-side PET and CT in patients with malignant melanoma. Eur J Nucl Med Mol Imaging. 2007;34(9):1355–1364. [PubMed]
95. Koskivuo IO, Seppanen MP, Suominen EA, et al. Whole body positron emission tomography in follow-up of high risk melanoma. Acta Oncol. 2007;46(5):685–690. [PubMed]
96. Rinne D, Baum RP, Hor G, Kaufmann R. Primary staging and follow-up of high risk melanoma patients with whole-body 18F-fluorodeoxyglucose positron emission tomography: results of a prospective study of 100 patients. Cancer. 1998;82(9):1664–1671. [PubMed]
97. Macfarlane DJ, Sondak V, Johnson T, Wahl RL. Prospective evaluation of 2-[18F]-2-deoxy-D-glucose positron emission tomography in staging of regional lymph nodes in patients with cutaneous malignant melanoma. J Clin Oncol. 1998;16(5):1770–1776. [PubMed]
98. Crippa F, Leutner M, Belli F, et al. Which kinds of lymph node metastases can FDG PET detect? A clinical study in melanoma. J Nucl Med. 2000;41(9):1491–1494. [PubMed]
99. Acland KM, Healy C, Calonje E, et al. Comparison of positron emission tomography scanning and sentinel node biopsy in the detection of micrometastases of primary cutaneous malignant melanoma. J Clin Oncol. 2001;19(10):2674–2678. [PubMed]
100. Kokoska MS, Olson G, Kelemen PR, et al. The use of lymphoscintigraphy and PET in the management of head and neck melanoma. Otolaryngol Head Neck Surg. 2001;125(3):213–220. [PubMed]
101. Belhocine T, Pierard G, De Labrassinne M, Lahaye T, Rigo P. Staging of regional nodes in AJCC stage I and II melanoma: 18FDG PET imaging versus sentinel node detection. Oncologist. 2002;7(4):271–278. [PubMed]
102. Longo MI, Lazaro P, Bueno C, et al. Fluorodeoxyglucose-positron emission tomography imaging versus sentinel node biopsy in the primary staging of melanoma patients. Dermatol Surg. 2003;29(3):245–248. [PubMed]
103. Havenga K, Cobben DC, Oyen WJ, et al. Fluorodeoxyglucose-positron emission tomography and sentinel lymph node biopsy in staging primary cutaneous melanoma. Eur J Surg Oncol. 2003;29(8):662–664. [PubMed]
104. Fink AM, Holle-Robatsch S, Herzog N, et al. Positron emission tomography is not useful in detecting metastasis in the sentinel lymph node in patients with primary malignant melanoma stage I and II. Melanoma Res. 2004;14(2):141–145. [PubMed]
105. Wagner JD, Schauwecker D, Davidson D, et al. Inefficacy of F-18 fluorodeoxy-D-glucose-positron emission tomography scans for initial evaluation in early-stage cutaneous melanoma. Cancer. 2005;104(3):570–579. [PubMed]
106. Vereecken P, Laporte M, Petein M, Steels E, Heenen M. Evaluation of extensive initial staging procedure in intermediate/high-risk melanoma patients. J Eur Acad Dermatol Venereol. 2005;19(1):66–73. [PubMed]
107. Bastiaannet E, Oyen WJ, Meijer S, et al. Impact of [18F]fluorodeoxyglucose positron emission tomography on surgical management of melanoma patients. Br J Surg. 2006;93(2):243–249. [PubMed]
108. Clark PB, Soo V, Kraas J, Shen P, Levine EA. Futility of fluorodeoxyglucose F 18 positron emission tomography in initial evaluation of patients with T2 to T4 melanoma. Arch Surg. 2006;141(3):284–288. [PubMed]
109. Maubec E, Lumbroso J, Masson F, et al. F-18 fluorodeoxy-D-glucose positron emission tomography scan in the initial evaluation of patients with a primary melanoma thicker than 4 mm. Melanoma Res. 2007;17(3):147–154. [PubMed]
110. Akcali C, Zincirkeser S, Erbagcy Z, et al. Detection of metastases in patients with cutaneous melanoma using FDG-PET/CT. J Int Med Res. 2007;35(4):547–553. [PubMed]
111. Strobel K, Skalsky J, Kalff V, et al. Tumour assessment in advanced melanoma: value of FDG-PET/CT in patients with elevated serum S-100B. Eur J Nucl Med Mol Imaging. 2007;34(9):1366–1375. [PubMed]
112. Falk MS, Truitt AK, Coakley FV, et al. Interpretation, accuracy and management implications of FDG PET/CT in cutaneous malignant melanoma. Nucl Med Commun. 2007;28(4):273–280. [PubMed]
113. Kell MR, Ridge JA, Joseph N, et al. PET CT imaging in patients undergoing sentinel node biopsy for melanoma. Eur J Surg Oncol. 2007;33(7):911–913. [PubMed]
114. Yancovitz M, Finelt N, Warycha MA, et al. Role of radiologic imaging at the time of initial diagnosis of stage T1b-T3b melanoma. Cancer. 2007;110(5):1107–1114. [PubMed]
115. Singh B, Ezziddin S, Palmedo H, et al. Preoperative 18F-FDG-PET/CT imaging and sentinel node biopsy in the detection of regional lymph node metastases in malignant melanoma. Melanoma Res. 2008;18(5):346–352. [PubMed]
116. Tsao MN, Lloyd NS, Wong RK. Clinical practice guideline on the optimal radiotherapeutic management of brain metastases. BMC Cancer. 2005;5(1):34. [PMC free article] [PubMed]
117. Blay JY, Bonvalot S, Casali P, et al. Consensus meeting for the management of gastrointestinal stromal tumors. Report of the GIST Consensus Conference of 20-21. March 2004, under the auspices of ESMO. Ann Oncol. 2005;16(4):566–578. [PubMed]
118. Loblaw DA, Perry J, Chambers A, Laperriere NJ. Systematic review of the diagnosis and management of malignant extradural spinal cord compression: the Cancer Care Ontario Practice Guidelines Initiative's Neuro-Oncology Disease Site Group. J Clin Oncol. 2005;23(9):2028–2037. [PubMed]
119. Fervers B, Burgers JS, Haugh MC, et al. Predictors of high quality clinical practice guidelines: examples in oncology. Int J Qual Health Care. 2005;17(2):123–132. [PubMed]
120. Im SS, Gordon AN, Buttin BM, et al. Validation of referral guidelines for women with pelvic masses. Obstet Gynecol. 2005;105(1):35–41. [PubMed]
121. Landier W, Bhatia S, Eshelman DA, et al. Development of risk-based guidelines for pediatric cancer survivors: the Children's Oncology Group Long-Term Follow-Up Guidelines from the Children's Oncology Group Late Effects Committee and Nursing Discipline. J Clin Oncol. 2004;22(24):4979–4990. [PubMed]
122. Kuehn T, Bembenek A, Decker T, et al. A concept for the clinical implementation of sentinel lymph node biopsy in patients with breast carcinoma with special regard to quality assurance. Cancer. 2005;103(3):451–461. [PubMed]
123. Schulz KD, Koller M, Lorenz W, et al. [Short version of the guideline “Early Detection of Breast Cancer in Germany”] Z Arztl Fortbild Qualitatssich. 2004;98(5):361–373. [PubMed]
124. Du XL, Key CR, Osborne C, Mahnken JD, Goodwin JS. Discrepancy between consensus recommendations and actual community use of adjuvant chemotherapy in women with breast cancer. Ann Intern Med. 2003;138(2):90–97. [PMC free article] [PubMed]
125. Barzilai DA, Cooper KD, Neuhauser D, Rimm AA, Cooper GS. Geographic and patient variation in receipt of surveillance procedures after local excision of cutaneous melanoma. J Invest Dermatol. 2004;122(2):246–255. [PubMed]
126. Francken AB, Bastiaannet E, Hoekstra HJ. Follow-up in patients with localised primary cutaneous melanoma. Lancet Oncol. 2005;6(8):608–621. [PubMed]
127. Virgo KS, Chan D, Handler BS, et al. Current practice of patient follow-up after potentially curative resection of cutaneous melanoma. Plast Reconstr Surg. 2000;106(3):590–597. [PubMed]
128. Mooney MM, Mettlin C, Michalek AM, Petrelli NJ, Kraybill WG. Life-long screening of patients with intermediate-thickness cutaneous melanoma for asymptomatic pulmonary recurrences: a cost-effectiveness analysis. Cancer. 1997;80(6):1052–1064. [PubMed]
129. Deurloo EE, Tanis PJ, Gilhuijs KG, et al. Reduction in the number of sentinel lymph node procedures by preoperative ultrasonography of the axilla in breast cancer. Eur J Cancer. 2003;39(8):1068–1073. [PubMed]
130. Motomura K, Inaji H, Komoike Y, et al. Gamma probe and ultrasonographically-guided fine-needle aspiration biopsy of sentinel lymph nodes in breast cancer patients. Eur J Surg Oncol. 2001;27(2):141–145. [PubMed]
131. Voit C, Mayer T, Proebstle TM, et al. Ultrasound-guided fine-needle aspiration cytology in the early detection of melanoma metastases. Cancer. 2000;90(3):186–193. [PubMed]
132. Bastiaannet E, Wobbes T, Hoekstra OS, et al. Prospective comparison of [18F]fluorodeoxyglucose positron emission tomography and computed tomography in patients with melanoma with palpable lymph node metastases: diagnostic accuracy and impact on treatment. J Clin Oncol. 2009;27(28):4774–4780. [PubMed]
133. Shumate CR, Urist MM, Maddox WA. Melanoma recurrence surveillance. Patient or physician based? Ann Surg. 1995;221(5):566–569. [PubMed]
134. Halpern AC, Mandal SK. Role of dermatologists in treating melanoma. J Natl Compr Canc Netw. 2006;4(7):695–702. [PubMed]
135. Hillman BJ. Self-referral: the time for whining is over. J Am Coll Radiol. 2004;1(11):795–796. [PubMed]
136. Pearson SD, Knudsen AB, Scherer RW, Weissberg J, Gazelle GS. Assessing the comparative effectiveness of a diagnostic technology: CT colonography. Health Aff (Millwood). 2008;27(6):1503–1514. [PubMed]
137. Earle CC, Park ER, Lai B, et al. Identifying potential indicators of the quality of end-of-life cancer care from administrative data. J Clin Oncol. 2003;21(6):1133–1138. [PubMed]
138. Lohr K. Medicare: A Strategy for Quality Assurance. Washington, DC: National Academy Press; 1990.
139. McColl A, Roderick P, Gabbay J, Smith H, Moore M. Performance indicators for primary care groups: an evidence based approach. BMJ. 1998;317(7169):1354–1360. [PMC free article] [PubMed]
140. National Cancer Policy Board Institute of Medicine and National Research Council. Ensuring Quality Cancer Care. Washington, DC: National Academy Press; 1999.
141. Mortenson LE. How to judge the cancer services benefit component of your health insurance plan. Cancer. 1998;82(10) suppl:2061–2067. [PubMed]
142. Dunnick NR, Applegate KE, Arenson RL. The inappropriate use of imaging studies: a report of the 2004 Intersociety Conference. J Am Coll Radiol. 2005;2(5):401–406. [PubMed]
143. Lysdahl KB, Hofmann BM. What causes increasing and unnecessary use of radiological investigations? A survey of radiologists’ perceptions. BMC Health Serv Res. 2009;9:155. [PMC free article] [PubMed]
144. Studdert DM, Mello MM, Sage WM, et al. Defensive medicine among high-risk specialist physicians in a volatile malpractice environment. JAMA. 2005;293(21):2609–2617. [PubMed]
145. Wilson IB, Dukes K, Greenfield S, Kaplan S, Hillman B. Patients’ role in the use of radiology testing for common office practice complaints. Arch Intern Med. 2001;161(2):256–263. [PubMed]
146. Berrington de Gonzalez A, Mahesh M, Kim KP, et al. Projected cancer risks from computed tomographic scans performed in the United States in 2007. Arch Intern Med. 2009;169(22):2071–2077. [PubMed]
147. Smith-Bindman R, Lipson J, Marcus R, et al. Radiation dose associated with common computed tomography examinations and the associated lifetime attributable risk of cancer. Arch Intern Med. 2009;169(22):2078–2086. [PubMed]
148. Gottlieb RH. Imaging for whom: patient or physician? AJR Am J Roentgenol. 2005;185(6):1399–1403. [PubMed]
149. Chu H, Cole SR. Bivariate meta-analysis of sensitivity and specificity with sparse data: a generalized linear mixed model approach. J Clin Epidemiol. 2006;59(12):1331–1332. author reply 1332–1333. [PubMed]
150. Harbord RM, Whiting P, Sterne JA, et al. An empirical comparison of methods for meta-analysis of diagnostic accuracy showed hierarchical models are necessary. J Clin Epidemiol. 2008;61(11):1095–1103. [PubMed]
151. Thornbury JR, Eugene W. Caldwell Lecture. Clinical efficacy of diagnostic imaging: love it or leave it. AJR Am J Roentgenol. 1994;162(1):1–8. [PubMed]

Articles from JNCI Journal of the National Cancer Institute are provided here courtesy of Oxford University Press