Search tips
Search criteria 


Logo of brjgenpracRCGP homepageJ R Coll Gen Pract at PubMed CentralBJGP at RCGPBJGP at RCGP
Br J Gen Pract. 2011 May; 61(586): e205–e214.
Published online 2011 April 26. doi:  10.3399/bjgp11X572391
PMCID: PMC3080225

Identifying suspected breast cancer: development and validation of a clinical prediction rule

Colin McCowan, BSc, MSc, PhD, lecturer in health informatics and Peter T Donnan, BA, BSc, MSc, PhD, professor of epidemiology and biostatistics
University of Dundee, Dundee, UK
John Dewar, MA, DMRT, FRCP, FRCR
consultant clinical oncologist, Ninewells Hospital and Medical School, Dundee, UK.
Alastair Thompson, BSc, MD, FRCS, professor surgical oncology
University of Dundee, Dundee, UK
Tom Fahey, MSc, MD, MFPH, FRCGP, professor of general practice



An evidence-based approach is needed to identify women with breast symptoms who are most likely to have breast cancer so that timely and appropriate referral can take place.


To report the development and validation of a clinical prediction rule for the diagnosis of breast cancer.

Design and setting

Cohort study with two prospective groups of women: those presenting to a symptomatic breast clinic (derivation cohort) and a separate cohort presenting to 11 general practices (validation cohort) in Tayside, Scotland.


Regression analysis was used to derive a clinical prediction rule from presenting symptoms, personal and family history, and clinical findings. Validation consisted of estimating the number of breast cancers predicted to occur compared with the actual number of observed breast cancers across deciles of risk.


In the derivation cohort of 802 patients, 59 (7%) were diagnosed with breast cancer. Independent clinical predictors for breast cancer were: increasing age by year (adjusted odds ratio [AOR] 1.10, 95% confidence interval [CI] = 1.07 to 1.13); presence of a discrete lump (AOR 15.20, 95% CI = 4.88 to 47.34); breast thickening (AOR 7.64, 95% CI = 2.23 to 26.11); lymphadenopathy (AOR 3.63, 95% CI = 1.33 to 9.92); and lump ≥2 cm (AOR 5.41, 95% CI = 2.36 to 12.38). All eight patients with skin tethering had breast cancer. The regression model had good predictive power, identifying all five breast cancers in the validation cohort of 97 patients in the top two deciles of risk.


The clinical prediction rule discriminates between patients at high risk of breast cancer from those at low risk, and can be implemented as an evidence-based recommendation to enhance appropriate referral from general practice to a symptomatic breast clinic. Ongoing validation in further populations is required.

Keywords: breast cancer, diagnosis, primary care

How this fits in

Breast cancer is a common problem with most patients presenting to their GP. GPs need to identify patients at high risk and refer them urgently for specialist assessment, while offering reassurance to those at low risk. This clinical prediction rule provides evidence that should enable GPs to stratify patients according to the probability of breast cancer, thereby improving the referral process.


Breast cancer affects nearly one in every 11 women in the UK and is responsible for 21 000 deaths a year. Of the 36 000 new cases of breast cancer each year in England and Wales, most patients will present with primary operable disease.1 Around three-quarters of breast cancer cases are diagnosed from patients who are symptomatic.2

GPs act as gatekeepers responsible for clinical assessment and have to prioritise patients for referral to specialist breast clinics. It is estimated that a GP will see between six and 34 new patients with symptomatic breast problems every year, but only one case of breast cancer.37 Early diagnosis of breast cancer benefits patients in terms of improved chance of survival; timely and appropriate assessment and referral is a critical step in terms of improved early diagnosis.

Clinical guidelines have been developed to enhance the referral process for breast cancer. Initial research reported that these guidelines did improve the referral process, increasing appropriate referrals in those subsequently diagnosed with breast cancer and reducing potentially inappropriate referrals of those women at low risk of breast cancer.810

A further change in the procedure for diagnosing breast cancer was the introduction of the ‘2-week rule’ by the UK Department of Health, which set targets for clinics to see patients with suspected breast cancer within a 2-week period, prioritising patients as being ‘urgent’ and other referrals as ‘routine’.11 An improvement in the diagnostic process from this initiative has not been realised; observational research shows that the number of cases of breast cancers in the 2-week rule population has fallen, while the number of those in the routinely referred group has increased.12 Furthermore, over a third of referrals are deemed to be inappropriate and large differences in GP referral patterns persist.13,1419 This poor performance of breast cancer referral guidance is attributed to the limited diagnostic value of the clinical criteria on which the guidelines are based.12

Clinical prediction rules are clinical tools that quantify the contribution of the history, physical examination, and diagnostic tests, and stratify patients according to the probability of having a target disorder.20 The outcome of interest can be diverse and range across the diagnostic, prognostic, and therapeutic spectrum. Furthermore, clinical prediction rules have been developed, validated, and used across the primary, secondary, and tertiary care settings.20 Developing and validating a clinical prediction rule is a particular form of observational epidemiological research that requires reference to specific methodological standards. Conventionally, clinical prediction rules go through three distinct stages before full implementation in a clinical setting:

  1. Development of the clinical prediction rule, establishing the independent and combined effect of explanatory variables that can include symptoms, signs, or diagnostic tests.
  2. Narrow and broad validation: the explanatory variables or clinical predictors in the derivation clinical prediction rule set are assessed in separate populations.
  3. Impact analysis of the clinical prediction rule, assessed by means of a randomised controlled trial: the impact of applying the clinical prediction rule in a clinical setting is measured either by patient outcome, health-professional behaviour, resource use, or any combination of these outcomes.21

The aim of this study was to develop and validate a clinical prediction rule in women presenting with breast symptoms, so that a more evidence-based approach to referral — which would include urgent referral under the 2-week rule — could be implemented as part of clinical-practice guidance.


Study design and participants

The study comprised two cohorts of patients: a derivation cohort and a validation group. For the derivation study, consecutive patients attending the symptomatic breast clinic at Ninewells Hospital, Dundee, who were referred by a GP between February and June 2007, were recruited. Patients attending for reasons related to cosmetic surgery or previous conservative breast surgery were excluded.

Data collection was based on current clinical guidelines and previous studies that report the clinical features in women presenting with breast symptoms in a breast referral clinic.2225 A research nurse collected clinical information on history and examination findings by staff in the symptomatic breast clinic, with the agreed data collection form as the template. The research nurse was unaware of the clinical outcome or diagnosis of breast cancer at the time of data collection.

For the validation cohort, 11 participating general practices in the region agreed to recruit all women who attended for an initial consultation regarding symptomatic breast problems between January 2006 and June 2007. Patients were excluded if the consultation was related to issues around cosmetic surgery or breastfeeding problems. A structured history and examination was performed and documented by participating GPs for each patient. Eligible patients were identified by GPs to a research nurse, who contacted the patient to ask for consent to participate in the study; if granted, medical note review and a telephone interview conducted by the research nurse were then used to complete full data collection for all patients.

Patient outcome

All patients in the two cohorts were traced for a diagnosis of breast cancer using the regional prospective cancer audit in the year following their initial consultation. This audit identifies all diagnosed breast cancers to allow reporting on the standards of care against national targets and is the basis of cancer registry returns. For patients identified by general practices within the validation cohort, onward referral to the symptomatic clinic was recorded as a secondary outcome. Ascertainment of outcome was blind in relation to initial clinical assessment for both the derivation and validation cohorts.

Data handling and plan of analysis

To protect patient confidentiality and privacy, all data was anonymised by a data analyst at the Health Informatics Centre, University of Dundee, using standardised operating procedures. The anonymised dataset was then transferred to the researcher for analysis. The study was approved by the Tayside Research Ethics Committee and Caldicott Guardian.

Derivation of the clinical prediction rule was based on the cohort of patients recruited from the symptomatic breast clinic. Descriptive statistics were generated in terms of potential explanatory variables and clinical outcome of breast cancer. A consultant breast surgeon assessed symptoms for those patients who presented with bilateral symptoms that differed between breasts to identify which breast symptoms to include in the analysis.

The first stage of the analysis was to investigate the univariable associations for the explanatory variables — namely clinical history and examination findings — with the outcome of breast cancer. These results are expressed as odd ratios (ORs); values of >1 indicate increased odds of the presence of breast cancer, while values of <1 indicate decreased odds of breast cancer. For inclusion into the multivariable logistic regression model, explanatory variables had be considered of prior clinical importance or be associated with a threshold P-value of ≤0.15. The final multivariate regression model was used to create a simple-to-follow, text-based clinical prediction rule.

For validation of the clinical prediction rule, the predicted probabilities were compared with the observed probabilities for breast cancer by means of calibration of the model. The model was calibrated by applying the regression co-efficients from the derivation cohort to the individuals in the validation cohort, generating expected and observed probabilities of breast cancer. Deciles of risk categories of expected and observed breast cancer cases were generated and goodness of fit assessed by the Hosmer-Lemeshow test.26 All data analysis was carried out using Stata (version 8, Stata Corporation, Texas, US).


Descriptive statistics

There were 851 patient consultations recorded at the symptomatic breast clinic during the study period. Twelve women attended on two separate occasions; in these cases clinical features on initial presentation were used. Seven patients were excluded as they were seen for reasons other than symptomatic breast problems. Thirty patients were included in the validation cohort identified from general practices and so were excluded from the derivation cohort. This left 802 patients who were referred by 65 different general practices for symptoms suggestive of breast cancer.

There were a total of 59 (7%) patients diagnosed with cancer, one (0.1%) with ductal carcinoma in situ, 101 (13%) with a benign lump, 112 (14%) with a cyst or abscess, 59 (7%) with muscular problems, and 183 (23%) with other diagnoses.

Derivation cohort

Univariable associations for clinical features of women presenting with breast symptoms show that increasing age, perimenopausal status, number of pregnancies, difference on examination between breasts, presence of a discrete or smooth lump, a lump ≥2 cm, lymphadenopathy, and increasing number of clinical signs were all associated with breast cancer (Table 1). Of note, all eight women who presented with tethering of the skin were found to have breast cancer.

Table 1
Univariable associations between explanatory variables and breast cancer

In terms of independent clinical predictors for breast cancer, the final logistic regression model shows that increasing age, the presence of a discrete lump, thickening of the breast, presence of lymphadenopathy, and presence of a lump ≥2 cm are all independently associated with breast cancer (Table 2). All eight patients (1%) who were recorded as having a lump that was tethered to the skin or chest wall after examination were subsequently diagnosed with cancer; however, this could not be included in the final model because of an absent comparison group of women, who did not develop breast cancer.

Table 2
Independent associations between explanatory variables and breast cancer.

Validation cohort

There were 202 patients identified by 11 general practices as presenting with symptoms suggestive of breast cancer. However, telephone contact details for 59 patients could not be traced, six patients had the wrong phone contact, and 19 could not be contacted. Of the 118 patients contacted, 16 declined participation and five failed to return written consent; this gave a total of 97 patients providing data for the validation study. Of these, 73 (75%) were referred to the symptomatic breast clinic; five (5%) were subsequently diagnosed as having breast cancer.

The clinical characteristics of the validation cohort and the proportion diagnosed with cancer for each of the explanatory factors from the derivation cohort are shown in Table 3. The validation cohort was split into deciles of risk based on the derivation model and the expected and observed cancers recorded (Figure 1). All observed breast cancers occurred in the top two deciles (top quintile) of expected risk. A Hosmer-Lemeshow goodness-of-fit test for the calibration of the model (HLGOFCS) shows no significant difference between expected and observed breast cancers (HLGOFCS = 7.02, P = 0.73), but the plot suggests the number of cancers was overestimated for those at highest risk (top decile).

Table 3
Descriptive statistics of symptoms and signs in relation to breast cancer in validation cohort.
Figure 1
Expected versus observed breast cancers by decile of predicted risk in the validation cohort.



This clinical prediction rule shows that increasing age, presence of a discrete lump, presence of a lump ≥2 cm in size, thickening of the breast, lymphadenopathy, and the presence of a lump tethered to the skin or chest wall all independently increase the probability of a woman having breast cancer. Validation of the rule shows that probability of breast cancer is higher with an increasing number of these independent clinical predictive factors.

Use of the clinical prediction rule may enable a more rational and evidence-based approach to clinical assessment in primary care and would allow stratification in terms of routine and urgent referral to specialist breast care clinics. For women with a low probability of breast cancer, the clinical prediction rule would also enable alternative strategies, aside from immediate referral, such as watchful waiting.

Strengths and limitations

There are several strengths to this study. It is a pragmatic study, with few women excluded in the derivation dataset, ensuring high external validity. By generating independent clinical risk factors, the key relevant clinical ingredients can be gathered in a relatively easy way. By providing weighted scores, the incremental value of elements from the clinical prediction rule enables a quantified, continuous, risk-based approach.

The limitations of the study relate principally to the validation cohort, which is underpowered and, in terms of strength of evidence, represents a narrow validation of the clinical prediction rule.20,21 Only 97 of the 202 patients identified as eligible for the validation cohort consented to the study and, based on known prevalence figures, these numbers are low; it would seem likely that the practices may have not recruited all eligible women to the validation cohort. These shortcomings create a potential for spectrum bias.

There are other limitations of this study: the size of the validation cohort relative to the number of predictors is limited, and further validation is necessary in different primary care population groups that have larger numbers of cases of breast cancer. This is so that the robustness of the clinical values contained in the clinical prediction rule can be further assessed.27

Comparison with existing literature

A total of 7% of patients in the derivation cohort who were referred were found to have breast cancer, a figure similar to that given in two other UK studies of patients in breast care clinics who were referred.12,13 This enhances the external validity of this study's findings.20,21 In a US breast care clinic, breast cancer occurred in 22.8% of women who were referred; this higher figure probably reflects the differing referral and access patterns between healthcare systems in the UK and the US. Indeed, in the US cohort, referral was restricted only to women with a palpable breast lump.24,28,29

The five (5%) women who had breast cancer in the validation cohort set in 11 general practices is comparable to previous studies examining the underlying cause in women presenting with breast symptoms in US primary care.30,31

The clinical variables included in the clinical prediction rule have clinical and content validity. The presence of a breast lump is the most common presenting symptom in patients with breast cancer32 and is also the most predictive symptom and sign.28,31 Finding a lump of ≥2 cm increases the risk of cancer;24 additionally the incidence of breast cancer is consistently shown to be associated with increasing age.24,29

Implications for research and practice

Current clinical guidelines do not appear to offer clear advantages in terms of enhancing the referral process with regard to correctly prioritising urgent referrals, correctly reducing the number of breast cancer cases among non-urgent referrals, or diminishing the number of unnecessary referrals (women at low risk who could be ‘watchfully waited’ or reassured in primary care without recourse to be referred to a specialist breast clinic).12

A simplified scoring system based on the regression model using a standard technique can be generated; this is shown in Table 4.33 This would allow GPs to construct a simple score for patients who are symptomatic. If we selected a referral threshold based around a score of ≥4, this would be equivalent to a post-test probability of breast cancer of 5–8%.

Table 4
Scoring systemfor onward referral for breast cancer.a

Selecting a referral threshold would require more analysis — preferably within a much larger validation study — and would also need to acknowledge an acceptable trade-off, in terms of cost-effectiveness, between missed cancers and unnecessary investigations.

Using the regression model to calculate individual risk, these data would suggest that a 40-year-old woman with a discrete lump of ≥2.0 cm would have a 13% risk of breast cancer; a 60-year-old woman with a discrete lump, a 20% risk of breast cancer; and a 60-year-old woman with a discrete lump of ≥2.0 cm, a 60% risk of breast cancer. Estimated breast cancer risk could easily be accomplished by use of a programmed personal digital assessment that links to management strategies — namely watchful waiting or immediate referral — based on the regression co-efficients.34

A high proportion of patients with cancer in the derivation cohort presented with a discrete lump (90%, 53/59), while 53 of 284 (19%) women in the derivation cohort with a discrete lump had breast cancer. However, it is important to emphasise that the absence of a lump does not preclude breast cancer, as seen in 10% of this cohort, and that clinical breast examination may not detect all lumps in cases of breast cancer.35,36

This study is consistent with the emerging literature, which shows that the referral threshold of GPs to specialist breast clinics is falling. Previous studies report referral rates of >30% of women who are symptomatic and seen in primary care,4 whereas three-quarters of patients in the validation cohort were referred for assessment.

Factors driving a lower referral threshold are likely to be multifactorial and include patient preferences (women being averse to reassurance without specialist assessment), health professional related factors (lack of awareness concerning indications for referral or risk aversion, missing a case of breast cancer) and health system related factors (ease of availability of breast care clinic).

In the derivation cohort, over a third (36%) of patients did not meet referral criteria based on past history, symptoms, and signs, and would constitute a group of women at low risk of having breast cancer. Although GPs may have had other reasons to refer them, most likely due to patient preference, such a large proportion of referrals will have several untoward consequences, such as delaying assessment of patients at higher risk, while placing women at low risk at greater risk of iatrogenic harm through unnecessary diagnostic work-up.

Further validation of the rule, as well as assessment of its impact on referral practice, influence of improved prioritisation, and timely identification of women whose breast symptoms warrant urgent assessment, is needed. Future studies should compare these potential benefits against current clinical guidelines.


We thank Lynne Leys and Janice Broomhall for undertaking the data collection in the study; Professor Ian Ricketts and Claire Jones for the development of the data collection system; Alison Bell and Duncan Heather of the Health Informatics Centre, University of Dundee, for procedural assistance in preparing and managing the dataset; and all participating doctors, staff, and patients at the general practices and breast clinic.



Colin McCowan was funded as an MRC/CSO Health Services Research and Health of the Public Training Fellow, Project Grant G106/1164.

Ethical approval

The study was approved by the Tayside Research Ethics Committee (REC Reference No.04/S1401/171) and Caldicott Guardian.

Competing interests

The authors have declared no competing interests.

Discuss this article

Contribute and read comments about this article on the Discussion Forum:


1. Veronesi U, Boyle P, Goldhirsch A, et al. Breast cancer. Lancet. 2005;365(9472):1727–1741. [PubMed]
2. Ebbs S, Sierakowski A. Non-urgent breast referrals subsequently diagnosed with cancer. Br J Gen Pract. 2004;54(503):465–466. [PMC free article] [PubMed]
3. Nichols S, Waters WE, Wheeler MJ. Management of female breast disease by Southampton general practitioners. BMJ. 1980;281(6253):1450–1453. [PMC free article] [PubMed]
4. Newton P, Hannay DR, Laver R. The presentation and management of female breast symptoms in general practice in Sheffield. Fam Pract. 1999;16(4):360–365. [PubMed]
5. Roberts MM, Elton RA, Robinson SE, French K. Consultations for breast disease in general practice and hospital referral patterns. Br J Surg. 1987;74(11):1020–1022. [PubMed]
6. BRIDGE Study Group. The presentation and management of breast symptoms in general practice in South Wales. The BRIDGE Study Group. Br J Gen Pract. 1999;49(447):811–812. [PMC free article] [PubMed]
7. Samers JM, Galetakis S, Scott CJ, et al. Breast cancer management: the perspective of general practitioners in inner and eastern Melbourne. Breast. 2004;13(6):468–475. [PubMed]
8. Cochrane RA, Davies EL, Singhal H, et al. The National Breast Referral Guidelines have cut down inappropriate referrals in the under 50s. Eur J Surg Oncol. 1999;25(3):251–254. [PubMed]
9. Cochrane RA, Singhal H, Monypenny IJ, et al. Evaluation of general practitioner referrals to a specialist breast clinic according to the UK national guidelines. Eur J Surg Oncol. 1997;23(3):198–201. [PubMed]
10. RoshanLall C, Leinster S, Mitchell S, Holcombe C. Current patterns of referral in breast disease. Breast. 2000;9(6):334–337. [PubMed]
11. NHS Executive Department of Health. Referral guidelines for suspected cancer. 2000 (accessed 30 Mar 2011)
12. Potter S, Govindarajulu S, Shere M, et al. Referral patterns, cancer diagnoses, and waiting times after introduction of two week wait rule for breast cancer: prospective cohort study. BMJ. 2007;335(7614):288. [PMC free article] [PubMed]
13. Patel RS, Smith DC, Reid I. One stop breast clinics — victims of their own success? A prospective audit of referrals to a specialist breast clinic. Eur J Surg Oncol. 2000;26(5):452–454. [PubMed]
14. Khawaja AR, Allan SM. Has the breast cancer ‘two week wait’ guarantee for assessment made any difference? Eur J Surg Oncol. 2000;26(6):536–539. [PubMed]
15. Cant PJ, Yu DS. Impact of the ‘2 week wait’ directive for suspected cancer on service provision in a symptomatic breast clinic. Br J Surg. 2000;87(8):1082–1086. [PubMed]
16. Jones R, Rubin G, Hungin P. Is the two week rule for cancer referrals working? BMJ. 2001;322(7302):1555–1556. [PMC free article] [PubMed]
17. Sauven P. Two week rule for cancer referrals. Specialists, not GPs, may be best qualified to assess urgency. BMJ. 2001;323(7317):864–865. [PubMed]
18. Thrush S, Sayer G, Scott-Coombes D, Roberts JV. Grading referrals to specialist breast unit may be ineffective. BMJ. 2002;324(7348):1279. [PMC free article] [PubMed]
19. Green AM. Grading referrals to specialist breast units. Communication has been degraded to exchange of dataset. BMJ. 2002;325(7360):392. [PubMed]
20. Reilly BM, Evans AT. Translating clinical research into clinical practice: impact of using prediction rules to make decisions. Ann Intern Med. 2006;144(3):201–209. [PubMed]
21. McGinn TG, Guyatt GH, Wyer PC, et al. Users’ guides to the medical literature: XXII: how to use articles about clinical decision rules. Evidence-Based Medicine Working Group. JAMA. 2000;284(1):79–84. [PubMed]
22. National Institute for Health and Clinical Excellence. Referral guidelines for suspected cancer. Clinical Guideline 27. 2005 (accessed 30 Mar 2011)
23. Scottish Intercollegiate Guideline Network Guidelines. Edinburgh: Scottish Intercollegiate Guideline Network; 2005. Management of breast cancer in women: a national clinical guideline.
24. Reeves MJ, Osuch JR, Pathak DR. Development of a clinical decision rule for triage of women with palpable breast masses. J Clin Epidemiol. 2003;56(7):636–645. [PubMed]
25. Osuch JR, Reeves MJ, Pathak DR, Kinchelow T. BREASTAID: clinical results from early development of a clinical decision rule for palpable solid breast masses. Ann Surg. 2003;238(5):728–737. [PubMed]
26. Hosmer DW, Hosmer T, Le Cessie S, Lemeshow S. A comparison of goodness-of-fit tests for the logistic regression model. Stat Med. 1997;16(9):965–980. [PubMed]
27. Katz MH. Multivariable analysis: a primer for readers of medical research. Ann Intern Med. 2003;138(8):644–650. [PubMed]
28. Laver RC, Reed MW, Harrison BJ, Newton PD. The management of women with breast symptoms referred to secondary care clinics in Sheffield: implications for improving local services. Ann R Coll Surg Engl. 1999;81(4):242–247. [PMC free article] [PubMed]
29. Campbell C, Durning P, Cheema I, Naisby G. A simple tool for rapid access to a symptomatic breast clinic. Eur J Surg Oncol. 2004;30(3):248–251. [PubMed]
30. Barton MB, Elmore JG, Fletcher SW. Breast symptoms among women enrolled in a health maintenance organization: frequency, evaluation, and outcome. Ann Intern Med. 1999;130(8):651–657. [PubMed]
31. Aiello EJ, Buist DS, White E, et al. Rate of breast cancer diagnoses among postmenopausal women with self-reported breast symptoms. J Am Board Fam Pract. 2004;17(6):408–415. [PubMed]
32. Thulesius HO, Lindgren AC, Olsson HL, Hakansson A. Diagnosis and prognosis of breast and ovarian cancer — a population-based study of 234 women. Acta Oncol. 2004;43(2):175–181. [PubMed]
33. Sullivan LM, Massaro JM, D'Agostino RB Sr. Presentation of multivariate data for clinical use: the Framingham Study risk score functions. Stat Med. 2004;23(10):1631–1660. [PubMed]
34. Vaidya JS, Baum M. Psion of the times. Lancet. 1997;350(9093):1784. [PubMed]
35. Oestreicher N, White E, Lehman CD, et al. Predictors of sensitivity of clinical breast examination (CBE) Breast Cancer Res Treat. 2002;76(1):73–81. [PubMed]
36. Reintgen D, Berman C, Cox C, et al. The anatomy of missed breast cancers. Surg Oncol. 1993;2(1):65–75. [PubMed]

Articles from The British Journal of General Practice are provided here courtesy of Royal College of General Practitioners