|Home | About | Journals | Submit | Contact Us | Français|
Accurate clinical diagnosis depends on the reliable recognition of signs and symptoms. This expertise comes from experience in seeing patients which has been traditionally gained over a long training period. Shortened specialist training (Modernising Medical Careers) has led to a greater reliance on structured teaching and skills transfer programmes. The accuracy of clinical diagnosis and the rate at which diagnostic skills improve during training is important for the assessment of trainees, and the delivery of care.
This study assessed the accuracy of clinical diagnosis of skin lesions by two junior plastic surgery trainees. They were asked to diagnose 120 consecutive skin lesions seen in a pigmented skin lesion clinic in 2005, with the histological diagnosis being confirmed following subsequent excision. The process was repeated a year later in 2006 to enable the rate of correct diagnosis to be compared.
Initially, 53.3% of diagnoses were correct. A year later, this had risen to 65.0%. Twenty-two different skin pathologies were present in excised specimens, and skin cancers comprised 30%. The trainees demonstrated 93.8% sensitivity in their initial diagnosis of malignancy (95% CI, 79.2–99.2) and 97.4% a year later (95% CI, 86.5–99.9). However, specificity was 69.3% (95% CI, 58.6–78.7) in 2005 and 71.6% (95% CI, 60.5–71.4) in 2006.
Accuracy in the diagnosis of the wide range of skin conditions presenting to an out-patient clinic was shown to increase over a 1-year period. We feel that this improvement resulted from regular clinical exposure supported by a structured learning programme. The shortening of the specialist training period may affect the acquisition of diagnostic skills by trainees and impact on the confidence with which they commence consultant practice.
The UK Government's restructuring of specialist training, outlined in Modernising Medical Careers (MMC),1 will have a significant impact on surgical training in all surgical specialties including plastic surgery. Frequent evaluation of both the surgical and diagnostic skills of trainees will be needed to determine the impact of shortened specialist training.
Skin lesions make up a large part of the plastic surgeon's out-patient workload and one of the effects of the Government's ‘two-week rule’ and guidelines for referral of suspected skin cancers has been an increase in total referral ratios from primary to secondary care.2 The rise in the number of suspect lesions being referred represents a significant service requirement and one that will result in junior registrars seeing an increasing number of skin lesions. Whilst several studies have compared the clinical accuracy of consultants and specialist registrars,3–5 few have addressed the learning curve of individual trainees with regard to clinical diagnosis.
There is little evidence to reflect how quickly (or even if) trainees' clinical diagnostic skills improve with time, given regular exposure to skin pathology. The clinically obvious melanoma or basal cell carcinoma can be recognised from the textbook; however, a number of skin lesions (both benign and malignant) do not always fit textbook descriptions. The diagnosis of these lesions and decisions regarding appropriate management can be difficult for the trainee and comes with experience.
In this study, we assessed the diagnostic accuracy of junior surgical trainees attending a rapid-access pigmented lesion clinic.
A weekly pigmented lesion clinic was started in January 1993 at Mount Vernon Hospital and has been described previously.6 Briefly, it provided a consultant-led service for rapid assessment and management of patients with suspected melanoma and non-melanoma skin cancers. The clinic operated on a ‘walk-in’ basis with patients only requiring a general practitioner's letter of referral and no formal appointment. After consultation, patients with suspicious lesions were offered urgent excision under local anaesthesia; patients with obviously benign lesions were re-assured and discharged. On occasion, benign lesions were listed for routine excision if they posed specific problems with regard anatomical location.
An audit of 120 consecutive lesions listed for excision by two surgical research fellows (WT and RAP) with similar experience in plastic surgery (11 months and 10 months, respectively, at SHO level) was performed in January 2005. A database was created using Microsoft Excel™ and patients' details including name, age, hospital number were recorded alongside clinical diagnosis at the time of pigmented lesion clinic consultation. Following excision, the histological diagnosis of all lesions was entered into the database. Research fellows had monthly feedback sessions where clinical photographs and histological results of excised lesions were discussed. A year later, a second cohort of 120 consecutive patients from January 2006 seen by the same two trainees were recruited by the same method. Accuracy of clinical diagnosis was then calculated and the two patient cohorts compared. Cases where the consultant's opinion was sought and the skin lesion excised were still included in the study with the trainee's initial recorded diagnosis entered into the database (even if that diagnosis was subsequently corrected by the consultant). Cases were excluded if no histology was available.
Statistical analysis was undertaken using SPSS v.14. Key findings were stated as percentages with Exact 95% confidence intervals. chi-squared and Fisher's Exact tests were used to assess significance, where relevant.
The histological diagnoses of the skin lesions excised in the two cohorts are shown in Table 1. In 2005, 19 different skin pathologies were excised with skin cancers comprising 26.6% (n = 32), which included 11 malignant melanomas. The following year, 20 different pathologies were excised. Skin cancers comprised 32.5% (n = 39) of the 120 cases, which included six malignant melanomas (plus one melanoma in situ). For classification purposes, a number of pathologies with small frequencies were grouped together into an ‘others’ group (see Table 1). Only significant differences in prevalence of sebhorreic keratosis and haemangioma (adjusted standardised residuals > 2) were seen between the two years. However, this did not unduly affect comparison between the two years.
Sensitivity refers to the proportion of true positives (malignancies) that are correctly identified by the clinician
Specificity refers to the proportion of true negatives (i.e. benign or non-malignant) that are correctly identified by the clinician
Positive predictive value (PPV) refers the proportion of patients with positive (malignant) test results who are correctly diagnosed
Negative predictive value (NPV) is the proportion of patients with negative (non-malignant) test results who are correctly diagnosed
Both the PPV and NPV will be influenced by changes in the prevalence of a condition even if the sensitivity and specificity of the test (or diagnosis) remain constant.
Cross tabulation of clinical versus histological diagnoses for the two cohorts (2005 and 2006) is shown in Table 2. Clinical diagnoses defined as ‘accurate’ are found along the principal diagonal. The accuracy of clinical diagnosis in 2005 was 53.3% (64 out of 120) compared with 65.0% (78 out of 120) in 2006. This difference between the two cohorts was not significant (Fisher's Exact test, P = 0.088). The level of agreement as assessed by the Kappa statistic increased from 0.471 (95% CI, 0.375–0.567) in 2005 to 0.592 (95% CI, 0.504–0.692) the following year.
Table 3 shows the sensitivity and specificity of the trainees' diagnosis of malignancy in the respective years using conventional diagnostic test measures. In 2005, the sensitivity was 93.8% (95% CI, 79.2–99.2) and the specificity 69.3% (95% CI, 58.6–78.7). The sensitivity improved to 97.4% (95% CI, 86.5–99.9) in 2006, and the specificity increased marginally to 71.6% (95% CI, 60.5–74.4). In 2005, the positive predictive value (PPV) was 52.6% and the negative predictive value (NPV) was 96.8%. In 2006, the equivalent values had improved to 62.3% and 98.3%, respectively. In 2005, there were two instances of malignancy being incorrectly diagnosed as benign lesions. Both lesions were basal cell carcinomas diagnosed as sebhorreic keratosis and dysplastic naevus respectively. In 2006, there was just one such false negative, a basal cell carcinoma incorrectly diagnosed as a dysplastic naevus.
Comparative diagnostic accuracy for five selected diagnoses (melanoma, squamous cell carcinoma, basal cell carcinoma, benign melanocytic naevus and dysplastic naevus) are presented in Table 4. There is evidence of improved sensitivity with respect to basal cell carcinoma and benign melanocytic naevus. However, the difference in sensitivity between 2005 and 2006 is statistically significant for benign melanocytic naevus only (P = 0.028, Fisher's Exact test).
Modernising Medical Careers (MMC) has shortened specialist surgical training and, in conjunction with the European Working Time Directive (EWTD), will reduce trainees' experience gained both in the operating theatre and in the clinic. MMC will lead to a greater reliance on operative skills transfer programmes and structured teaching to deliver surgeons with operative and diagnostic abilities suitable for consultant practice. It has not yet been proved that skills transfer programmes are an adequate substitute for actual operating hours to develop technical skills,7,8 nor that structured teaching over a shortened training period can match years of clinical experience. Carli et al.9 found a 4-h formal training session given to family doctors showed improvements in correct melanoma diagnosis rates; other studies have demonstrated similar improvements following skin cancer educational programmes.10,11 However, most of these studies have taken place over short periods only and focused on general practitioners and medical students rather than specialists in training. Gachon et al.12 suggested that clinicians most skilled in the diagnosis of melanoma rely on a subconscious comparison with the patient's remaining common naevi (the so-called ‘ugly duckling sign’) rather than a learned algorithm of morphologic criteria. Such comparative processes come with experience and regular exposure to skin pathologies rather than formal lectures or tutorials. Other studies support this; consultants with greater than 10 years' experience demonstrate significantly higher diagnostic accuracy rates compared to trainees with up to 5 years of specialist experience.4,5 This would indicate it is difficult to ‘fast-track’ an individual's diagnostic skills by replacing specialist apprenticeship with teaching sessions.
This study assessed how the diagnostic skills of two surgical trainees improved over a 1-year period with weekly clinical exposure to skin lesions. Regular teaching sessions providing feedback on the histological diagnoses of excised lesions in conjunction with clinical photographs provided a system analogous to the structured learning programmes detailed in the new surgical curriculum. Rates of diagnostic accuracy of the trainees improved, though not significantly, from 53.3% (2005) to 65.0% (2006).
Trainees demonstrated a high sensitivity for identifying malignancy, 93.3% in 2005 which improved slightly to 97.4% in 2006, though a lower specificity, 69.3% (2005) increasing to 71.6% (2006). Neither of these small increases was statistically significant. This was further reflected by the positive and negative predictive values (see Table 3), though the low values for PPV were partially due to the low prevalence of malignancy in the cohorts. Trainees understandably had a high index of suspicion for malignancy resulting in a number of false positives. Whilst distinguishing an atypical naevus from a melanoma clinically can be difficult, we feel the trainees' unfamiliarity with the breadth of benign pathology presenting through the clinic may have been a factor. One limitation of the study was that, by including only lesions that were listed for surgical excision (or biopsy), the cohort may have been skewed towards the more difficult clinical cases. Trainees were more likely to biopsy a lesion if they were uncertain of the diagnosis and it is acknowledged that a number of potentially correct (and incorrect) diagnoses on lesions that were not excised were not included. However, histology remains the only definitive method of assessing clinical accuracy. To our knowledge, over the study period there were no cases of missed malignancy.
Recent guidelines from the National Institute for Health and Clinical Excellence (NICE) state that suspected skin cancers should be initially referred by GPs to dermatologists;13 however, Cox2 has demonstrated that only 42% of skin cancers are referred by this route under the ‘two-week rule’.2 Trainee surgeons must be familiar with all forms of skin cancer, which may be erroneously referred as other pathologies or picked up incidentally on their patients (many of whom are at high risk of skin cancer). Moreover, it has been reported that up to 10% of melanoma patients, many of whom are under follow-up in plastic surgical clinics, will suffer a second melanoma within 2 years.14
The trainees in this study attended a ‘walk-in’ pigmented lesion clinic, and saw between 15–20 patients each week. This included a proportion of tertiary referrals (10%) indicating that the case mix of the clinic included a number of more complex cases. A specialist trainee attending a normal ‘appointment-based’ plastic surgery out-patients clinic would not see this volume and thus, potentially, take longer to achieve the same diagnostic accuracy. Studies have shown that accuracy of diagnosis is related to number of lesions a clinician is exposed to on a weekly basis.4 The biggest improvements for diagnostic accuracy here were seen in lesions the trainees were seeing regularly (benign melanocytic naevi and basal cell carcinomas), though the comparatively small numbers of histologically confirmed cases of melanoma makes interpretation more cautious. Melanomas comprised 7.5% of lesions excised in this series, which is similar to other published series on pigmented lesion clinics.9,15,16
Knowledge of how the clinical diagnostic accuracy of the trainee improves with experience is essential to assess the impact of shortened surgical training. Several studies have previously compared consultants to registrars with regard to their diagnostic capabilities;3–5 others have evaluated accuracy rates between different hospital specialists,17 though few have looked to evaluate an individual trainee's clinical accuracy over time. This study has demonstrated that improvements in clinical accuracy are gradual rather than dramatic. It is acknowledged that the educational journey of the two trainees over a 1-year period is not necessarily representative of all plastic surgical trainees on a national level, although their respective experience in their speciality at the time of the study would be equivalent to those entering ST3 level.
Accuracy in the clinical diagnosis of a wide range of skin lesions improved over a 1-year period. We feel this increase results from regular clinical exposure supported by a structured teaching programme. Shortened surgical training may impact the acquisition of these diagnostic skills with implications for trainees when they enter consultant practice.
The authors wish to thank The Restoration of Appearance and Function Trust (RAFT) and The Royal College of Surgeons of England for generously supporting this work.