Randomised controlled trials in primary care: case study

BMJ. 2000 July 1; 321(7252): 24–27.

PMCID: PMC27422

Randomised controlled trials in primary care: case study

Sue Wilson, senior research fellow, Brendan C Delaney, senior lecturer, Andrea Roalfe, medical statistician, Lesley Roberts, research associate, Val Redman, project officer, Andy M Wearn, lecturer, and F D Richard Hobbs, professor of primary care and general practice

Department of Primary Care and General Practice, Division of Primary Care, Public and Occupational Health, University of Birmingham, Medical School, Birmingham B15 2TT

Contributed by

Contributors: BCD and SW were responsible for the idea and the initial drafts of this paper. BCD was lead investigator of the Birmingham open access endoscopy study, which was designed by BCD, FDRH, and SW. LR and VR were responsible for practice recruitment, data collection, and validation. AR undertook the analyses. All authors were members of the trial management group and took part in redrafting the paper. SW and BCD are joint guarantors.

Correspondence to: S Wilson s.wilson/at/bham.ac.uk

Accepted April 25, 2000.

See editorial "The research needs of primary care " on page 2.

This article has been cited by other articles in PMC.

Although over 90% of patient contacts within the NHS occur in primary care, many of the interventions used in this setting remain unproved.¹ The relevance of research undertaken in secondary or tertiary care to general practice is questionable, and more research based in primary care is needed.² Increasing research in primary care will inevitably increase demand for randomised controlled trials in this setting. Some of the trials will be of health service interventions (pragmatic trials),³ where the focus lies in assessing the cost effectiveness of an intervention rather than efficacy or safety. The difficulties experienced in doing randomised controlled trials in primary care have been reported⁴^–⁶ and are not restricted to this setting.⁷^,8 We discuss some of the issues that must be considered when conducting and interpreting the results of trials in primary care using examples generated during a trial of the management of dyspepsia (box).

Birmingham open access endoscopy study

The study aimed to evaluate the effectiveness of two management strategies for patients presenting in primary care with symptoms of dyspepsia. Two randomised controlled trials were conducted concurrently, with eligibility being determined by the patient's age at presentation. Randomisation was done at the individual patient level by using sealed opaque, sequentially numbered envelopes during a primary care consultation for dyspepsia.

Initial endoscopy trial

Eligible patients—50 years of age or older.

Intervention—Referred for open access endoscopy.

Test and endoscopy trial

Eligible patients—Under 50 years.

Intervention—Tested for Helicobacter pylori antibodies with Helisal near patient test. Patients with positive results referred for open access endoscopy; those with negative results received symptomatic treatment only.

Control arms (both trials)—Managed according to “usual practice” excluding open access endoscopy. This included antacids, H₂ receptor antagonists, proton pump inhibitors, outpatient gastroenterology referral, facilitated or direct access endoscopy (for example, vetted by consultant), and testing for H pylori.

Outcomes—Primary outcomes were change in symptom score and cost effectiveness. Secondary outcomes included quality of life and acceptability.

Data collection—At recruitment, general practitioners completed a case report form providing patient identifiers and a limited amount of baseline data. Patients completed the dyspepsia symptom questionnaire and the quality of life questionnaire at recruitment and at six and 18 months after randomisation. A patient satisfaction questionnaire was also completed at 18 months. Data on use of health services were collected from general practice records and endoscopy units at 12 months after recruitment.

Summary points

All trials require a compromise between including sufficient practitioners to recruit a representative cohort of patients and the time and cost of recruiting and maintaining the motivation of these practitioners
Prior beliefs relating to the efficacy and direct or side effects of an intervention affect both doctor and patient participation
Trials in any setting are rarely fully representative with respect to both patient and disease related characteristics
Modelling, sensitivity analysis, and statistical estimates of uncertainty are necessary to determine the generalisability of trials and to particularise results to a given clinical setting
Trials in primary care should give more representative results and are preferable to applying results obtained in secondary care

Dyspepsia is a common clinical problem. About 2% of the population consult their general practitioner each year with dyspeptic symptoms,⁹ and it costs the NHS more than £1bn a year, with a large proportion of these costs relating to drug prescription.¹⁰ The evidence base has largely consisted of cohort studies of patients referred to secondary care for investigation¹¹^,12and economic models.¹³ In the absence of evidence from primary care, several conflicting consensus guidelines have been generated.¹⁴^,15 Dyspepsia represents a good example of a chronic disease that is largely managed in primary care and that requires high quality evidence from randomised controlled trials in primary care.

Why do research in primary care?

The natural course of any disease can be described as progression from the first occurrence of disease to the first episode of symptoms, which may lead to a primary care consultation and subsequent treatment. For some conditions patients will be referred to secondary care. The population available to the researcher at each of these stages differs in terms of severity of symptoms, stage of disease, patient attitudes, and response to treatment. Research undertaken in secondary care is subject to biases of case selection and referral and may underestimate the prevalence of disease and overestimate the impact on quality of life compared with observations in primary care. Interventions shown to be effective in secondary care may therefore have limited value in the community.

Important differences also occur in the outcomes of similar interventions in different healthcare settings. For example, most patients seen in primary care have earlier or milder disease than those referred to hospital. Therefore, the positive predictive value of diagnostic tests in primary care is lower than in secondary care, and invasive investigations may be less justified and less acceptable to patients. Management decisions taken by primary and secondary care doctors may also differ systematically, reflecting different experience and priorities.¹⁶

Unit of randomisation

The conduct of therapeutic trials in which selected groups of patients are randomised to two or more treatments is well established, and the unit of randomisation is invariably the patient. However, randomisation by patient may be inappropriate when evaluating some health services. For example, we may wish to evaluate the effect of issuing a new set of guidelines and therefore wish to randomise general practitioners into those who did or did not receive the guidelines. As practitioners may discuss guidelines within the practice and patients do not necessarily have continuity of care with individual practitioners, randomisation by practice may be necessary. Similarly, within the community neighbours talk to each other, and if a practitioner becomes known to have a particular interest in a condition then patients in a group practice may select to see him or her. This “contamination” of the study group may also necessitate randomisation by practice. Cluster randomisation brings statistical complexities and requires a larger sample size.¹⁷

Recruitment bias

Participating practices

All trials have to make a compromise between including sufficient practitioners to recruit a representative cohort of patients and the time and cost involved in recruiting and maintaining the motivation of these practitioners. These problems are more acute within primary care where, even for common conditions, the number of patients that practitioners see with the disease of interest represents only a small proportion of their total consultations.

Inevitably, not all practices within the defined catchment area will agree to participate in a trial. The self selected group of practitioners who agree to recruit patients can affect the representativeness of the study population. Not all groups of patients may be adequately represented. Practitioners with a particular interest in the condition under investigation may be more likely to participate and may manage their patients more effectively than the average practitioner, decreasing the effect size observed.¹⁸^,19

Only a quarter of general practices contacted for our dyspepsia trial actively participated (that is, recruiting [gt-or-equal, slanted]

5 patients) (table (table1).1). Participating practices had more partners and were located in less deprived areas (table (table2).2).

Table 1

Recruitment of practices into Birmingham endoscopy study

Table 2

Characteristics of participating and non-participating practices

Participating patients

Trials may fail to recruit a representative sample of patients, either because all eligible patients are not prepared to enter the study or because all eligible patients are not asked to enter the study. The Birmingham endoscopy trial had practice recruitment rates ranging from 0.1 to 15.6/10

000 population per month (see fig fig1).1).

Figure 1

Relation between recruitment rate and length of time in study. Recruitment rate is calculated as a 3 month moving average for each period after entry into the study. To adjust for differing practice populations, monthly recruitment rates have been directly (more ...)

The variability in recruitment rates may be due to differences in the prevalence of disease or presentation rates but is inevitably also due to differences in the proportion of eligible patients who were recruited. Interpretation of these differences requires access to the records of all patients within participating practices who have the relevant disease. However, it is rarely possible to obtain consent to access records from all patients not entering trials. The study denominator and representativeness of the sample can be determined by comparing the patient and disease related characteristics of participants with those of the total eligible population using anonymised data. However, not all practices routinely record all consultations on computer and manual searches of paper records are costly.

Factors affecting recruitment rates

Incident or prevalent cases

Figure Figure11 shows that the recruitment rate fell sharply over the first year to reach a relatively steady state in the second year. The fall in recruitment observed during the first year could be attributable to waning enthusiasm for the trial. However, an alternative explanation is the recruitment of existing (prevalent) cases. Once the pool of prevalent patients has been recruited eligible cases are restricted to those with incident disease. The inevitable mixture of incident and prevalent cases emphasises a further difficulty in analysing and interpreting data on chronic diseases irrespective of whether trials are conducted in primary or secondary care.

The primary care consultation rate for dyspepsia in the United Kingdom is 20/1000 population.⁹ Assuming the recruitment rate after 12 months of the study can be taken as a proxy for incidence, the observed incidence rate in the Birmingham dyspepsia trial was 1.67/1000 practice population. Even after practices with poor recruitment rates were excluded (<5 patients) the observed recruitment rate was only 1.97/1000. Thus, less than 10% of the eligible population seems to be have been recruited to the trial.

Although it is technically more appealing to restrict a trial to incident cases, definition of new disease is often difficult, and the findings will rarely be of relevance to general practitioners, whose caseloads comprise people with both new and existing disease.

Ethical issues

Patients may participate in trials to “please” their practitioner or because they are “afraid” to refuse. The need for reassurance that future management will not be compromised by non-participation may be greater in primary care, where the patient may see the same practitioner over many years. The appropriateness of general practitioners recruiting their own patients is further complicated if they receive financial rewards for recruiting cases, even if payment only covers costs.

Ethical trials require both participating clinicians and patients to be in equipoise.²⁰ Although robust evidence for the cost effectiveness of a particular intervention may not yet exist, if practitioners have a prior belief that one form of management is beneficial they may choose not to randomise patients to treatment.

When attempting to recruit practices we found that many general practitioners were enthusiastic about particular management strategies and were unwilling to randomise patients to different management options. Beliefs relating to the efficacy, direct effects, or side effects of an intervention affect both doctor and patient participation. Patients who are not prepared to accept randomisation are not eligible to be entered into a trial. However, the exclusion of patients who refuse endoscopy because they think it is an uncomfortable or painful procedure could affect estimates of the impact of the trial results. An intervention may be cost effective but unacceptable to most patients. Trials of management strategies should aim to determine the proportion of patients who refuse randomisation because of treatment preferences. Complex interventions may require the use of additional research tools to assess any barriers to accepting the intervention.²¹

Selective recruitment of patients

Lack of concealment of allocation can increase the effect size observed in randomised controlled trials.²² The most secure method requires the clinician to contact an external randomisation service after obtaining informed consent. When recruitment occurs within the routine consultation, any complexities in the randomisation process may deter general practitioners from recruiting patients. Assessing eligibility, explaining the trial, addressing patients' questions and obtaining consent, randomising, and collecting baseline data are time consuming for participating practitioners.²³

The additional workload led to some practices suspending recruitment at busy times (Monday mornings, holidays, flu season, etc). Furthermore, although a practice agreed to participate in the trial this did not necessarily mean that all partners were equally committed to recruiting patients. The effect of such variation in recruitment is not easy to quantify. Recruitment rates may be more predictable for trials conducted in the community, when independent research staff perform all the research activity.

Incomplete ascertainment of eligible patients may also be due to the selective recruitment of patients with particular sets of symptoms. Figure Figure22 shows that the practices with lower rates of recruitment tended to recruit patients with more severe symptoms. The reasons for this apparent selection bias are unknown, although possible explanations include practices that had suspended recruitment being reminded about the study when a patient with severe disease presents or practitioners applying differing definitions of disease.

Figure 2

Relation between practice recruitment rate and patients' mean symptom score at recruitment. Scores are sum of scores of four questions on dyspeptic symptoms each scored on a Likert scale from 0 to 4. High scores indicate more symptoms

Discussion

Trials in primary care are no different from those in secondary or tertiary care in terms of their lack of success in recruiting all patients with the disease of interest.²⁴ Similarly, trials in any setting are rarely fully representative with respect to both patient and disease related characteristics. Such limitations do not invalidate the use of the randomised controlled trial. They merely require additional work to be undertaken to establish the effect of the intervention on the total population.

Trials in primary care should recruit participants that are more representative of patients seen in the community than are participants in trials in secondary care. However, if the processes that operate to determine whether a patient is included in a trial are not random, trial participants may be skewed with respect to disease severity or other factors such as age or social class. Although this will not bias the trial result (internal validity), it may misrepresent the effect of the intervention in non-trial settings (external validity). Modelling, sensitivity analysis, and statistical estimates of uncertainty are necessary to determine the generalisability of the trial and to particularise results to a given clinical setting.²⁵

Primary care provides many opportunities but is not an easy place to conduct research. Trials must be designed and undertaken by multidisciplinary teams with expertise in both the context of clinical practice and research methods.

Acknowledgments

We acknowledge the support of the Dyspepsia Trials Collaborators Group, particularly Robbie Foy, Anne Duggan, and Jayne Parry. This manuscript was stimulated by our participation in the Birmingham open access endoscopy study. The success of this project has been dependent on the enthusiasm and cooperation of the general practices that participated.

Notes

Editorial by Thomas

Footnotes

Funding: The Birmingham open access endoscopy study received financial support from the NHS Research and Development Primary/Secondary Care Interface Programme, West Midlands NHS Research and Development, and the Astra Foundation. BCD holds an NHS Research and Development primary care career scientist award and LR holds a New Blood Fellowship awarded by the NHS Executive (West Midlands).

Competing interests: None declared.

References

1. Medical Research Council. MRC topic review; primary health care research review. London: MRC; 1997.

2. Department of Health. London: Stationery Office; 1996. Primary care: delivering the future. . (Cm 3512.)

3. Roland M. What are pragmatic trials? BMJ. 1998;316:285. [PMC free article] [PubMed]

4. Fairhurst K, Dowrick C. Problems with recruitment in a randomized controlled trial of counselling in general practice: causes and implications. J Health Serv Res Policy. 1996;1:77–80. [PubMed]

5. Tognoni G, Alli C, Avanzini F, Bettelli G, Colombo F, Corso R, et al. Randomised clinical trials in general practice: lessons from a failure. BMJ. 1991;303:969–971. [PMC free article] [PubMed]

6. Ward E, King M, Lloyd M, Bower P, Freidl K. Conducting randomised trials in general practice: methodological and practical issues. Br J Gen Pract. 1999;49:919–922. [PMC free article] [PubMed]

7. Barnett A. Are the results of the extracranial-intracranial bypass trial generalizable? N Engl J Med. 1987;316:820–825. [PubMed]

8. MacIntyre I. Tribulations for clinical trials. BMJ. 1991;302:1099. [PMC free article] [PubMed]

9. McCormick A, Fleming D, Charlton J. Morbidity statistics from general practice. Fourth national morbidity study 1991-1992. London: Office of Population Censuses and Surveys; 1995.

10. Asante M, Lord J, Mendall M, Northfield T. Endoscopy for Helicobacter pylori seronegative young dyspeptic patients: an economic evaluation based on a randomized trial. Eur J Gastroenterol Hepatol. 1999;11:851–856. [PubMed]

11. Sobala GM, Crabtree JE, Pentith JA, Rathbone BJ, Shallcross TM, Wyatt JI, et al. Screening dyspepsia by serology to Helicobacter pylori. Lancet. 1991;338:94–96. [PubMed]

12. Mendall MA, Jazrawi RP, Marrero JM, Molineaux N, Levi J, Maxwell JD, et al. Serology for Helicobacter pylori compared with symptom questionnaires in screening before direct access. Gut. 1995;36:330–333. [PMC free article] [PubMed]

13. Briggs AH, Sculpher RPH, Logan RPH, Aldous J, Ramsay ME, Baron JH. Cost effectiveness of screening for and eradication of Helicobacter pylori in management of dyspeptic symptoms in patients under 45 years of age. BMJ. 1996;312:1321–1325. [PMC free article] [PubMed]

14. Axon ATR, Bell GD, Jones RH, Quine MA, McCloy RF. Guidelines on appropriate indications for upper gastrointestinal endoscopy. BMJ. 1995;310:853–856. [PMC free article] [PubMed]

15. European Helicobacter Pylori Study Group. Current European concepts in the management of Helicobacter pylori infection—the Maastricht consensus report. Gut. 1997;41:8–13. [PMC free article] [PubMed]

16. Fendrick AM, Hirth RA, Chernew ME. Differences between generalist and specialist physicians regarding Helicobacter pylori and peptic ulcer disease. Am J Gastroenterol. 1996;91:1544–1548. [PubMed]

17. Kerry SM, Bland JM. Statistics notes: sample size in cluster randomisation. BMJ. 1998;316:549. [PMC free article] [PubMed]

18. Kinmonth AL, Woodcock A, Griffin S, Spiegal N, Campbell MJ. Randomised controlled trial of patient centred care of diabetes in general practice: impact on current wellbeing and future disease risk. BMJ. 1998;317:1202–1208. [PMC free article] [PubMed]

19. UK Prospective Diabetes Study (UKPDS) Group. Intensive blood-glucose control with sulphonylureas or insulin compared with conventional treatment and risk of complications in patients with type 2 diabetes (UKPDS 33) Lancet. 1998;352:837–853. [PubMed]

20. Lilford RJ, Jackson J. Equipoise and the ethics of randomisation. J R Soc Med. 1995;88:552–559. [PMC free article] [PubMed]

21. Bradley F, Wiles R, Kinmonth AL, Mant D, Gantley M. Development and evaluation of complex interventions in health services research: case study of the Southampton heart integrated care project (SHIP). The SHIP Collaborative Group. BMJ. 1999;318:711–715. [PMC free article] [PubMed]

22. Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA. 1995;27:408–412. [PubMed]

23. Wilson S, Delaney B, Roalfe A, Hobbs R. The ethics of paying GPs. BMJ. 1999;318:1484. [PMC free article] [PubMed]

24. Peto V, Coulter A, Bond A. Factors affecting general practitioners' recruitment of patients into a prospective study. Fam Practice. 1993;10:207–211. [PubMed]

25. Lilford R, Royston G. Decision analysis in the selection, design and application of clinical and health services research. J Health Serv Res Pol. 1998;3:159–166. [PubMed]

Articles from BMJ : British Medical Journal are provided here courtesy of
BMJ Publishing Group