|Home | About | Journals | Submit | Contact Us | Français|
It has been argued that in case-control studies, controls should be drawn from the base population that gives rise to the cases. In designing a study of occupational injury and risks arising from long-term illness and prescribed medication, we lacked data on subjects’ occupation, without which employed cases (typically in manual occupations) would be compared with controls from the general population, including the unemployed and a higher proportion of white-collar professions. Collecting the missing data on occupation would be costly. We estimated the potential for bias if the selection rule were ignored.
We obtained published estimates of the frequencies of several exposures of interest (diabetes, mental health problems, asthma, coronary heart disease) in the general population, and of the relative risks of these diseases in unemployed vs. employed individuals and in manual vs. non-manual occupations. From these we computed the degree of over- or underestimation of exposure frequencies and exposure odds ratios if controls were selected from the general population.
The potential bias in the odds ratio was estimated as likely to fall between an underestimation of 14% and an overestimation of 36.7% (95th centiles). In fewer than 6% of simulations did the error exceed 30%, and in none did it reach 50%.
For the purposes of this study, in which we were interested only in substantial increases in risk, the potential for selection bias was judged acceptable. The rule that controls should come from the same base population as cases can justifiably be broken, at least in some circumstances.
It has been argued that in the proper design of a case-control study, controls should be drawn from the base population that gives rise to the cases . As many standard textbooks explain, the basic parameter of interest, the odds ratio (OR), involves a comparison of relative exposure frequency (or more strictly, exposure odds) in cases and in the population at risk of becoming cases [2,3]. The purpose of the control group is thus to give representative information on the exposure(s) of interest in the population at risk.
This is better assured (although not guaranteed) if cases and controls are ascertained from the same, discrete, well-defined study population. For example, in investigating the relation between shift work and ischaemic heart disease (IHD), McNamee et al  focussed on a cohort of workers from a particular company who started work at age ≤ 50 years between 1 Jan 1950 and 31 Dec 1992; the cases were cohort members who died from IHD at age ≤75 years during this period, while controls were chosen from living cohort members individually matched to cases by age and date of hire. Cases and controls were compared for their exposure to shift work as documented in company records. Because sampling was ‘nested’ within a well-defined occupational cohort, the precondition that controls should be liable to be identified as cases in the event of dying from IHD (i.e. be at risk) was easily met.
The objective that exposure data be ‘representative’ requires further that the selection process for controls should be independent of the exposures of interest. In the above study of shift workers, we have no expectation that the selection algorithm would have systematically led to an erroneous estimate of shift work frequency among controls relative to all non-cases within defined matching strata.
Practical considerations, however, may sometimes mandate departures from the ideal. For example, in hospital-based case-control studies, controls are sampled from hospital patients with health problems other than the disease of interest . An advantage of this method is that the recruitment of controls may be cheaper, and response rates higher. However, there is a danger that the exposures of controls might not accurately represent those in the population at risk of becoming cases. For example, if the focus of a study were risks from smoking, then, as many hospital-treated diseases are smoking-related, it can be appreciated that careless selection of controls might lead to an overestimate of exposure frequency in the at-risk population and an underestimate of the corresponding OR.
These issues in control selection have been well covered elsewhere [1,2,3,4]. In this paper we describe another instance in which convenience, costs and practical considerations conflict with the ideal. We illustrate for our example how reasonable quantitative estimates of the potential extent of bias can inform competing choices in study design.
Increasingly, as response rates to other forms of investigation have fallen , researchers have looked to exploit routinely collected datasets, some of which are amenable to case-control analysis. In the UK, one of these, the General Practice Research Database (GPRD), offers a log of all consultation episodes associated with significant events, illnesses, or medical activity (diagnosis, referral, prescription etc) among patients from some 370 participating general practices (an estimated 3,000,000 episodes of care covering 6% of all residents of England and Wales) . This resource with its large sample size and its wealth of routinely collected health and prescription data has been successfully exploited in numerous pharmaco-epidemiological studies of case-control design . However, some variables of interest are typically missing, including occupational history.
We identified a study question of high policy relevance that we wished to address using the GPRD database. The populations of westernised countries are aging. In future, therefore, the frequency of common age-related health conditions is likely to rise among the workforce, as is the proportion of workers taking prescribed medicines. Potentially, certain widely used medicines that impair arousal, concentration, cognition, and psychomotor performance, and some common illnesses that result in sudden incapacity, impaired judgment, or sensory deficit could increase the risk of accidental injury at work. But which drugs and diseases, by how much, in what circumstances, and with what consequences? The British Government has announced strategic plans to maximise job retention rates among experienced older workers, but in delivering these plans employers require an evidence base to manage injury risks, the aim being to ensure safe job placement while at the same time avoiding needless restriction of job opportunities. However, when we conducted a systematic review on the topic  we found few relevant data, both overall and by type of injury (eg fractured femur) and external cause (eg fall). And we identified a need to improve upon cross-sectional studies with self-reported exposures and self-reported outcomes, by mounting investigations with objective measure of outcome and documented timing of exposures (to counter worries about common instrument reporting bias and reverse causation) .
The GPRD database overcame some of these limitations and fulfilled several requirements for a case-control analysis of occupational injury risk, co-morbidity and medication. It allowed an operational case definition (namely, male patients with a consultation episode for an injury coded as occupational, or involving plant or off-road vehicles or machinery or tools likely to be used only at work); and for each case, plentiful controls could be identified who were well matched by age, sex, and general practice. A preliminary scoping exercise suggested that we would find some 1,700 cases, to whom we could match 8,500 controls. For each injury we could establish relevant exposure parameters, including the diagnostic Read code and date of first consultation; all prescriptions, with dates, within the 24 months preceding the event; and all diagnoses, with dates, preceding the event. We thus envisaged an analysis to establish the frequency and main reasons for consultation in the 24 months before injury consultation; the frequency of prescribing over this time, the main prescribed drugs, and relative exposure odds of various illnesses and treatments in cases versus controls. As risks could vary according to time since first prescription of a drug or first onset of a new illness, so analysis could encompass various exposure time windows. Several aspects of confounding could be addressed through the matching algorithm (age, sex, geographical area) or via proxy measures available within the health-rich dataset (e.g. alcoholic liver disease as a proxy for alcohol misuse).
Unfortunately, occupation was poorly recorded in the database, which raised concerns of the kind outlined in our introduction. Specifically, cases of occupational injury must necessarily come from the employed subfraction of the study population, whereas controls - in the absence of employment information - would be drawn from the whole population, among whom a proportion would be unemployed and not at risk of occupational injury. Also, cases would be more likely than employed controls to come from manual occupations, as the potential for occupational injury is greater in blue-collar work. Bias could arise if controls over-represented the prevalence of diseases and treatments that prevent work and are more common in the unemployed; or if they under-represented the (generally worse) health characteristics of manual workers. Finally, although practical experience suggests that such selection applies to only a few high-risk jobs, in theory people with health problems could be excluded from jobs with higher injury potential, and if these jobs were less common in controls then any risks of injury from ill health would tend to be underestimated. It should be noted that these potential biases, which relate to representativeness of exposure information among controls, do not all operate in the same direction.
The missing information could only be obtained at a cost. To contact study subjects and to ascertain their employment status by a questionnaire or interview was feasible but would carry significantly higher administrative costs and effort, a need for more elaborate ethical permissions and suitably anonymised third party mailings by collaborators with data control, and the potential for one bias (related to non-response) to be substituted for another. Some of the economic advantages of a routine publicly available dataset would be lost.
The case series method of analysis , which compares the relative incidence of events of interest only among cases (in time windows of exposure and non-exposure), might seem to offer an attractive alternative. Each case would provide his or her own reference information. Since the technique is based solely on the experience of cases, this would circumvent any concern about differences in work and employment experience that arose from differences in case and referent sampling frames. However, the method is only suited to short-term exposures that impact on risk for a limited time period, such as acute intercurrent illnesses, exacerbations of pre-existing disease and newly prescribed treatments (for which purposes we intend using it). Over the much longer time frames of chronic illness and long-term treatment, potential exists for employment conditions to alter markedly within individuals. For such long-term exposures, the case-control design is still the preferred choice.
Faced with this dilemma, we decided to assess quantitatively the potential bias arising if controls were selected without employment information. How much would it matter that cases came from a subfraction of the population from which controls were sampled, breaking the rule on control selection often repeated in standard textbooks? We addressed this practical question focusing on four common exposures that would be of interest in our hypothetical case-control study – namely, diabetes, anxiety-depression, asthma, and coronary heart disease.
Considering the question – “how great is the potential bias when controls are sampled from whole practice lists, rather than patients in work?” - the logic that underlies quantitative estimation is as follows:
To give an example:
In this example ‘y’ overestimates the true value of ‘p’ by 10%.
We obtained estimates of ‘y’ from a previously published analysis of the GPRD, which covered 1,007,913 men (employed and unemployed) registered with 288 practices in England and Wales during 1996 . Estimated RRs for exposures of interest (diabetes, mental health problems, asthma and coronary heart disease) in unemployed vs. employed men were chosen following a brief literature review (details available on request) for their congruence with the published data [10-20] and to ensure that our assumptions were realistically founded.
In Table 1 we have solved for ‘p’ to estimate the true prevalence in working controls, and present information on the extent to which ‘y’ would overestimate ‘p’ and the impact this would have on our mooted study, assuming that a given exposure truly increases the odds of a work-related injury by a factor of 2 or 3. (The method by which these figures are derived is illustrated separately in an Appendix for one of the row items in Table 1.)
We then repeated the process using RRs for exposures in manual vs. non-manual workers, to illustrate separately the likely bias arising from this source (Table 2) [24-37]. The logic in calculation is similar but not identical:
‘p’ = y/1.0636. According to published estimates, y is 3.89% . Hence, p= 3.89/1.0636=3.6574; but the value of interest is 1.24p, the prevalence in manual controls, which equals 4.535%. ‘y’ would underestimate this by (4.35-3.89)/3.89 = 16.5%.
It may be seen from Tables 1 and and22 that the potential for bias in the odds ratio would generally be less than 20%. It was estimated as likely to fall between an underestimation of 14% and an overestimation of 36.7% (95th centiles). In fewer than 6% of simulations did the error exceed 30%, and in none did it reach 50%. This is comparable to, or smaller than the bias which might arise from incomplete response had such a case-control study been undertaken by attempting to contact patients directly [38,39].
For simplicity, the analysis in Table 2 assumes that all the cases come from manual occupations; in reality some occupational injuries would arise in non-manual workers. Thus, Table 2 somewhat overstates the likely bias.
It should be noted that the direction of bias is different in the two tables, leading to an underestimate in Table 1 and an overestimate in Table 2. In practice, controls would represent a mixture of manual workers, non-manual workers and the unemployed. Hence, the actual bias would be expected to lie between the values in the two tables.
In designing this case-control protocol, practical considerations encouraged us to violate a well-rehearsed axiom of control selection. A trade-off then existed between cost and possible loss of internal validity. Although different input assumptions would yield different values, with the quantitative assumptions presented we judged the potential bias as acceptable - particularly when set against the alternative bias that might arise from attempted patient contact and incomplete response.
The method by which we estimated the potential extent of bias in this planning exercise was somewhat similar to that which has been applied when estimating possible impacts of uncontrolled confounding after data collection has been completed [40,41]. This reflects a conceptual overlap between selection bias and confounding. Thus, the controls in our proposed study would include unemployed people, whose health status and use of medication is likely to differ systematically from that of those in employment. Viewed one way, the resultant unrepresentativeness of controls could be classed as a selection bias. Alternatively, however, employment status could be considered as a confounding variable, associated with the risk factors of interest (health status and use of medication), and independently determining the risk of occupational injury.
The findings of our analysis are not wholly unexpected, since the potential for bias must reflect the weighted average of risks of exposure in component subgroups (employed vs. unemployed on the one hand and manual vs. non-manual on the other). In relation to unemployment, it would be limited because unemployed subjects are in the minority; and in the case of type of work by the moderate RRs between manual and non-manual occupations (although RRs are greater at the extremes – social class V vs. I – these represent only a fraction of the whole population).
When a study is being planned, the potential extent of bias that is deemed tolerable will depend on the use that will be made of its findings. This is analogous to consideration of the scope for random error – the statistical power that is required of a study will vary according to the context. In the example that we have given, our interest was only in detecting substantially increased risks of occupational injury. Minor hazards would have no impact on employment decisions. For this reason, a possible error of 20% was judged acceptable. However, in other circumstances, the investigator might be interested in discriminating much smaller deviations from the null, in which case a possible error of 20% in risk estimates might be considered unacceptable.
We conclude that in this particular study the absence of data on employment status, although a drawback, would not be a critical limitation. Studies involving other exposures and outcomes would need to be considered on their individual merits. However, simple calculations and externally published data can be used to obtain an estimate of the potential for bias. The selection rule can justifiably be broken, at least in some circumstances.
We would like to thank Dr Clare Harris for her help in compiling some of the published risk estimates on which calculations are based, and for her careful proof-reading of the manuscript.
Funding The authors are in receipt of core research funding from the Medical Research Council, UK.
Consider the example of diabetes and the effect of unemployment status, with the following input assumptions….
|RR||Prevalence (%) among controls|
in our sample (y)
|Prevalence (%) in|
workers (solve for ‘p’)
|3.00||1.59 = (0.921*p) + (0.079*3p)||1.373%||1.72|
The extract from Table 1 (above) shows that the estimated prevalence of diabetes in working controls (p) is 1.373%, and that we the OR of 2.0 can be expected to be biased downwards to 1.72. This last figure is derived as follows:
If all the controls were workers, 1.373% of 8,500 i.e. 116.705 (without rounding) would be diabetics and the remainder (8383.295) would not.
In fact, as our controls include some unemployed men, and as a whole have a prevalence of 1.59%, we estimate in error that 135.15 controls would have diabetes and 8364.85 would not.
Imagine first the ‘true’ 2×2 table, confined to workers, among whom the true OR for injury is 2.
|Yes||A||(1700 – A)||1700|
This table has one unknown, but OR = 2. Thus, (8383.295*A)/(116.705*(1700-A)) =2 Solving for ‘A’ gives a value of 46.05:
Using ‘all’ controls rather than ‘working’ controls will alter the bottom row of this table as follows:
Thus, instead of an OR of 2, the estimated OR would become: (46.05*8364.85)/(135.15*1653.95) = 1.723
Competing interests: None.
The Corresponding Author has the right to grant on behalf of all authors and does grant on behalf of all authors, an exclusive licence (or non-exclusive for government employees) on a worldwide basis to the BMJ Publishing Group Ltd and its Licensees to permit this article (if accepted) to be published in Occupational and Environmental Medicine and any other BMJPGL products to exploit all subsidiary rights, as set out in our licence (http://oem.bmj.com/ifora/licence.pdf).
All web addresses accessed on 01/07/09