PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptNIH Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Cancer. Author manuscript; available in PMC Mar 7, 2007.
Published in final edited form as:
PMCID: PMC1811062
NIHMSID: NIHMS12004
Interventions to Improve Follow-Up of Abnormal Findings in Cancer Screening
Roshan Bastani, Ph.D.,1 K. Robin Yabroff, Ph.D., M.B.A.,2 Ronald E. Myers, Ph.D., D.S.W.,3 and Beth Glenn, Ph.D.1
1 Department of Health Services, School of Public Health, and Jonsson Comprehensive Cancer Center, University of California–Los Angeles, Los Angeles, California.
2 Health Services and Economics Branch, Applied Research Program, Division of Cancer Control and Population Sciences, National Cancer Institute, Rockville, Maryland.
3 Department of Medicine, Jefferson Medical College, Thomas Jefferson University, Philadelphia, Pennsylvania.
Address for reprints: Roshan Bastani, Ph.D., Division of Cancer Prevention and Control Research, Jonsson Comprehensive Cancer Center, School of Public Health, University of California–Los Angeles, 650 Charles Young Drive South, Room A2-125, CHS, Los Angeles, CA 90095-6900; Fax: (310) 206-3566; E-mail: bastani/at/ucla.edu
The potential reduction in morbidity and mortality through cancer screening cannot be realized without receipt of appropriate follow-up care for abnormalities identified via screening. In this paper, the authors critically examine the existing literature on correlates of receipt of appropriate follow-up care for screen-detected abnormalities, as well as the literature on interventions designed to increase rates of receipt of follow-up care. Lessons learned describe what is known and not known about factors that are related to or predict receipt of follow-up care. Similarly, effective interventions to increase follow-up are described and gaps identified. A conceptual model is developed that categorizes the health care system in the United States as comprising four levels: policy, practice, provider, and patient. Some patient-level factors that influence follow-up receipt are identified, but the lack of data severely limit the understanding of provider, practice, and policy-level correlates. The majority of intervention studies to increase follow-up receipt have focused on patient-level factors and have targeted follow-up of abnormal Papanicolaou smears. Insufficient information is available regarding the effectiveness of provider, practice, or policy-level interventions. Standard definitions of what constitutes appropriate follow-up are lacking, which severely limit comparability of findings across studies. The validity of various methods of obtaining outcome data has not been clearly established. More research is needed on interventions targeting provider, system, and policy-level factors, particularly interventions focusing on follow-up of colorectal and breast abnormalities. Standardization of definitions and measures is needed to facilitate comparisons across studies.
Keywords: abnormal findings, cancer screening, interventions, evaluation, methodologic issues
Along the continuum from cancer screening through diagnosis, treatment, and rehabilitation, little attention has focused on intervening to assure timely, effective diagnosis once an abnormality has been identified. To achieve the goal of reducing cancer morbidity and mortality, it is imperative that patients receive timely and appropriate follow-up for detected abnormalities, as a prerequisite to appropriate treatment. Failure to obtain appropriate diagnostic services can have a significant effect on psychosocial sequelae, morbidity, and mortality, as well as cost implications for the individual and the health care system. A recent review of studies on follow-up of abnormal screening examinations reported that, in the majority of studies, fewer than 75% of patients receive adequate follow-up care.1 The negative implications of failure to follow-up are substantial, especially at the population level.
Unlike routine cancer screening, which can be initiated by the patient and often can be obtained without the need for a physician’s or provider’s referral, the identification and resolution of an abnormal finding is inextricably tied to a clinical setting and a medical provider. Because the delivery of care for the follow-up of an abnormality necessarily involves contact with the health care system, this central theme is utilized throughout the article. In an attempt to synthesize the research findings to date and to formulate future directions, we have developed a conceptual framework that reflects this theme. We have named this the Model for Intervention Research, Dissemination, and Implementation (MIRDI; Fig. 1).
FIGURE 1
FIGURE 1
Model for intervention research, dissemination, and implementation.
As shown in Figure 1, we conceptualize the health care system in this country as comprising four levels: policy, practice, provider, and patient. The policy level refers to the larger organizational context within which health care is provided. This level includes the way in which health care in general is financed and delivered, as well as more specific public and private payer cancer screening-related policies at the federal, state, and/or local levels that can directly or indirectly affect follow-up care. An example is the National Breast and Cervical Cancer Early Detection Program (NBCCEDP). In all states, the District of Columbia, and 12 Native American tribes, the NBCCEDP provides funding for breast and cervical cancer screening and for clinical diagnostic services related to abnormal screening results. Policy-level influences form the broad context within which care for abnormal findings is provided and therefore have the potential to significantly influence background follow-up rates as well as to enhance the probability of success of interventions applied in research settings.
The practice level refers to the structural and operational features of the specific health care settings in which follow-up care is recommended and provided. This level includes sites such as hospitals, primary care offices, community clinics, and specialist practices. Here, decisions are made about the types of follow-up procedures required by the patient, the settings in which the procedures are to be performed, and the timing of follow-up. Characteristics of the practice setting (e.g., size, location, access to resources, staffing) are likely to influence the delivery of care and to affect the potential success of interventions designed to enhance the timeliness and appropriateness of follow-up.
The provider and patient levels of the health care system are the most proximal to the delivery/receipt of follow-up care, and are influenced by the more distal policy and practice levels. At the provider level, factors that may influence physicians’ recommendations and orders for follow-up include physicians’ background, training, and experience; perceptions related to cancer screening and follow-up; social influence; and characteristics of the patients themselves. Interactions in the physician–patient encounter help to shape patients’ understandings concerning the nature and importance of follow-up and the steps involved in diagnostic evaluation. Interventions designed to affect physicians’ knowledge, attitudes, and behaviors are key to actively engaging the patients who are in their care. Personal background, cognitive, and psychosocial characteristics of practitioners and patients are factors that are likely to shape patients’ response to recommended follow-up.
Factors operating across and within all four levels of the health care system can affect the quality and timeliness of follow-up care and can affect the success of research intervention trials designed to identify effective methods for improving follow-up patterns. One of the primary challenges associated with testing and implementing interventions to increase diagnostic follow-up of screening abnormalities is the interdependence of factors at the patient, provider, practice, and health policy levels. For example, routine reminders to patients with incomplete follow-up of an abnormal finding require the existence of information systems that can be used to identify and track abnormal findings and their evaluation. In addition, if individuals with abnormal findings are prompted by the reminders to visit their primary care physician for a specialty referral, that provider must also perceive the abnormal findings as important and recommend appropriate follow-up. Specialists must be available and then accept the referral from the primary care provider, if a referral is made. Patients must be able to pay for diagnostic workup, either out of pocket or by associated copayment within an insurance program. Financial constraints or lack of specialist capacity can also influence receipt of diagnostic evaluation. Policy-level factors, such as lack of or limited third-party coverage for diagnostic evaluation, can determine the effectiveness of interventions conducted at other levels in the health care system.
Although many challenges to testing and implementing interventions to improve follow-up are similar for the various levels of the health care system in our MIRDI framework, the scope of the challenge may vary across levels. Such challenges can also vary by intervention type and may include costs or resources, the need for development of intervention-specific infrastructures, and sustainability beyond the intervention period. For example, the costs associated with patient-level telephone counseling are likely to have both fixed and variable components, so that the cost of the intervention may increase for every additional patient. Conversely, development and integration of a system for tracking and monitoring screening abnormalities and follow-up may require large up-front investments in computer systems, but costs are mainly fixed, so that the overall cost of the intervention per patient may actually decline with larger numbers of patients or as more types of screening tests are included. The type of infrastructure required for intervention delivery and potential sustainability without additional investments also varies for these two intervention examples.
Penetrance, or the number of individuals reached by an intervention, is likely to vary by intervention level and is an important consideration. Over a lifetime of regular screening, a sizeable portion of individuals will have an abnormal screening test (either a true or false-positive result). However, most individuals will not have multiple screening abnormalities.2,3 As a result, interventions targeted to the level of the individual patient will likely affect only those patients to whom the intervention is directed and who actually receive the intervention. Conversely, interventions implemented at the provider or practice levels have the potential to affect the care of many more patients over a much longer period of time because, over a lifetime of practice, a provider may see hundreds of patients with screening abnormalities. Similarly, changes at the health policy level can have a very wide effect, over a long time period, because they are often not dependent on delivering a specific intervention to a specific patient but rather form the context within which follow-up care is provided.
As suggested by the heuristic model presented in the current article, research findings may be disseminated through a number of channels. Researchers routinely disseminate results of studies via presentations at professional meetings and through publications. Research findings are also disseminated through articles in the popular media and by campaigns promoted by advocacy groups. Furthermore, research findings can be disseminated in the form of guidelines developed by professional organizations and via rules and regulations adopted by public agencies. The processes of conducting intervention research, disseminating research findings, and implementing results into the health care system are not well documented in relation to follow-up care. It is often assumed that dissemination will occur if the results of intervention studies are published in the peer-reviewed literature. However, this is usually not the case. Increasingly, it has been recognized that targeted efforts need to focus on understanding the most effective means of dissemination and on ensuring that the process actually occurs.4 Relatively few studies conducted to date have evaluated the effectiveness of various methods of dissemination. More research is needed in this area. Although recognizing the importance of dissemination and ultimate implementation of research findings, the main focus of this article will be on intervention research to improve follow-up of abnormal findings in cancer screening, as depicted on the left side of our MIRDI framework (Fig. 1).
The current report is restricted to interventions designed to influence the recommendation, ordering, performance, or receipt of follow-up for abnormal findings. Our definition of abnormal screening results includes any clinical indications from a screening test or examination that suggest the need for further evaluation to determine the presence or absence of disease. Follow-up is defined as diagnostic procedures recommended, ordered, performed, or received to accomplish this goal. We refer mainly to randomized, controlled trials or concurrently controlled studies (also called quasiexperimental studies). Studies using pre–post or other observational designs are also mentioned, as they can provide information on the feasibility of implementing a specific type of intervention, although conclusions regarding efficacy or effectiveness of interventions are usually not warranted. Table 1 lists the lessons learned from interventions to improve follow-up of abnormal findings in cancer screening.
TABLE 1
TABLE 1
Lessons Learned Regarding Interventions Aimed at Improving Follow-Up of Abnormal Cancer Screening Findings
Lesson 1: Although Some Information Is Available Regarding Patient-Level Factors That Influence Receipt of Follow-Up, Understanding of Provider-, Practice-, and Policy-Level Correlates of Follow-Up Is Severely Limited by Lack of Data
A number of patient-level factors have been associated with incomplete diagnostic evaluation after an abnormal screening test result. Patients may not recall notification of their abnormal result or may not understand the purpose of follow-up procedures.58 Fatalism and fear of painful invasive procedures, treatment, and/or a diagnosis of cancer are attitudes associated with incomplete or less timely follow-up.57,9,10 Lack of social support is also associated with incomplete follow-up.1114 Finally, being uninsured or underinsured is associated with delayed or incomplete follow-up.5,15,16
However, very few studies have systematically examined how factors at the provider, practice, and policy levels are related to receipt of follow-up care. Myers et al.17,18 found that provider-related factors—such as board certification, number of years of experience, belief that complete diagnostic evaluation is efficacious, and perception that complete diagnostic evaluation is part of standard practice—were related to the intention to complete diagnostic evaluation for an abnormal colorectal finding. Physician forgetfulness, belief that the abnormal findings were trivial, and the anticipation that the patient was expected in the clinic anyway have also been cited as being related to the lack of appropriate follow-up by physicians.19 At the practice level, health care delivery factors, such as systems for reporting abnormal findings and tracking completion of additional procedures (or lack of such systems), can affect follow-up care. A recent national study of colorectal cancer screening in health plans reported that only 40% of plans have any system in place for colorectal cancer screening delivery or monitoring. Of these plans, < 20% had measures for abnormal screening results, whether or not follow-up procedures were obtained, or the outcome of follow-up procedures.20 At the policy level, financial constraints at the federal, state, health plan, and insurer levels can affect referral and completion of diagnostic tests, particularly for uninsured patients. A recent study of medical directors at community health centers reported that uninsured patients frequently fail to receive specialty care for follow-up services due to the inability to pay.21
Given the paucity of information regarding consistent correlates of follow-up receipt, it is important to conduct more research in this area, particularly at the provider, practice, and policy levels. Achieving a thorough understanding of factors associated with follow-up receipt is a necessary prerequisite for the development and implementation of efficacious interventions.
Lesson 2: Various Patient-Level Interventions Are Effective in Increasing Follow-Up Rates; Effective Interventions Include Mail and Telephone Reminders, Telephone Counseling, and Print Educational Interventions, but the Majority of These Studies Have Focused on Follow-Up of Abnormal Papanicolaou Smears, whereas Very Little Is Known Regarding How To Increase Follow-Up Rates for Patients with Abnormal Findings on Breast and Colorectal Cancer Screening
Most intervention studies to increase diagnostic follow-up have been conducted at the patient level. These interventions address many of the barriers to diagnostic follow-up. Reminders delivered by mail or telephone have been shown to lead to improved follow-up.2225 Interventions that provided education and addressed fears related to an abnormal finding through telephone counseling, pamphlets, or other materials also were largely successful in improving timely follow-up.5,2527 Interventions delivered via the telephone were particularly effective, leading to improvements in follow-up of 24–26% compared with usual care.5,25 Intervention strategies that addressed barriers associated with a lack of health insurance and attempted to improve patient compliance through payment vouchers or transportation incentives are also associated with increased follow-up compliance.23,24,28 Strategies that may increase social support, such as use of lay health workers selected from the target population, have been tested in observational studies and appear to be feasible, but study design limitations preclude assessment of efficacy.29
Currently, the majority of interventions to improve diagnostic follow-up of abnormal screening tests conducted at any of the levels in our MIRDI model have been conducted among women with abnormal Papanicolaou (Pap) tests.5,2328,3038 Several interventions have been conducted to increase follow-up in women with abnormal mammograms or clinical breast examinations.35,39 We identified 2 interventions to increase follow-up after abnormal colorectal cancer screening, one for men and women with abnormal fecal occult blood tests (FOBTs)40,41 and the other for findings of polyps > 5 mm or multiple diminutive polyps with flexible sigmoidoscopy.42 We also identified two interventions tested to improve follow-up for multiple screening tests, including abnormal Pap smears, FOBTs, and mammograms.22,43 Most of these interventions have been successful in increasing diagnostic follow-up. Although the value of prostate cancer screening with the prostate-specific antigen (PSA) test is controversial, use of the test remains widespread, generating a substantial pool of men who have elevated PSA levels and who require decision-making regarding the appropriate follow-up action. As yet, no published studies have examined predictors or rates of diagnostic follow-up after an abnormal PSA result, most likely because of the ongoing debate surrounding the value of aggressive follow-up and treatment.
Lesson 3: Practice-Level Barriers to Follow-Up of Abnormal Findings Are Potentially Modifiable, but Few Intervention Trials Have Tested Such an Approach
Despite potentially modifiable practice-level barriers to follow-up of abnormal screening tests, few controlled, system-targeted interventions to increase adherence to follow-up have been conducted. One of the few intervention studies conducted at this level targeted a predominantly minority, medically indigent population of women with abnormal Pap smears. A specialized clinic—including a nurse case manager, tracking system, reminder calls, rescheduling of missed appointments, and clinic staffing with on-site colposcopy—achieved a significantly increased follow-up rate compared with a randomly assigned control group.30 Using a quasiexperimental design, Kaplan et al.28 tested a computerized tracking system and centralization of follow-up services, accompanied by transportation and financial incentives, among indigent women and found only modest intervention effects. Other successful practice-level interventions have addressed issues related to appropriate referral, including same-day follow-up of abnormal Pap smears31 or of abnormal flexible sigmoidoscopy42 with on-site colposcopy or colonoscopy, respectively. Patient navigators,43 case management systems,32 and tracking systems44 have also been evaluated, but in uncontrolled settings that preclude assessment of effectiveness.
Lesson 4: Insufficient Information Is Available Regarding the Effectiveness of Provider-Focused Interventions Aimed at Increasing Receipt of Follow-Up for Patients with Abnormal Findings; Although Policies at the Federal, State, and Local Levels Have the Potential To Significantly Increase Rates of Follow-Up for Such Patients, Systematic Evaluation of These Types of Policies Has Not Been Conducted
Only one controlled trial targeting physician-level factors was found in the literature. In that trial, however, the physician-focused components of the intervention were delivered in conjunction with several system-focused interventions. The practices in the participating health plan that were randomized to the intervention group received physician reminders for patients with abnormal findings, followed by feedback on completion of follow-up, along with education consisting of academic detailing visits about colorectal cancer screening and diagnostic follow-up, a tailored letter, and a reminder call.40 The intervention was associated with an increase in physician referral for complete diagnostic evaluation and with increased receipt of complete diagnostic follow-up among patients within the intervention practices.41 An uncontrolled study evaluated computerized provider reminders, demonstrating the feasibility of this approach. However, the lack of a control group limited the assessment of effectiveness.39
To our knowledge, no controlled intervention trials to improve follow-up of abnormal screening test results have been conducted at the policy level. The passage of the Breast and Cervical Cancer Treatment and Prevention Act of 2000 provided support for case management and treatment for low-income, uninsured women (within the Centers for Disease Control and Prevention-sponsored NBCCEDP) with abnormal findings or a cancer diagnosis. Implementation is performed on a state-by-state basis and requires coordination with the Medicaid program. However, no systematic assessment of the effectiveness of this program has been reported, likely due to the difficulties in rigorously evaluating such widely disseminated programs. In addition, measures of follow-up of abnormal screening tests have been pilot tested for potential inclusion in the Health Plan Employer Data and Information Set (HEDIS) quality of care measures, but these measures have not been added either to existing measures of breast and cervical cancer screening or to the newly added measure of colorectal cancer screening. Addition of measures, on follow-up of abnormal findings, to HEDIS would likely exert pressures to increase provision of appropriate and timely follow-up of abnormal findings. Because such policy-level measures are widely disseminated, have the potential to stimulate large changes in health care access and utilization, and involve considerable financial investments, it is important to use creative, albeit nonexperimental, designs to evaluate their effects.45
Lesson 5: Receipt of an Abnormal Cancer Screening Result has Been Found to be Associated with a Number of Negative Psychological States, although Very Little Is Known about the Effects of Negative Psychosocial States on Subsequent Completion of Diagnostic Follow-Up; Studies Should Include Both the Psychological Sequelae of an Abnormal Finding and the Actual Receipt of Care as Outcomes
Although many studies have been conducted to increase follow-up care for an abnormality, relatively few have assessed how receipt of an abnormal result influences patient perceptions, quality of life, psychologic states, and attitudes toward subsequent screening if a false-positive result is obtained. A number of studies have found that women who receive abnormal screening results, including false-positive mammograms or false-positive Pap smears, experience a variety of negative mood states including anxiety, distress, and intrusive thoughts.4650 Fewer studies have examined the psychologic effects of false-positive results for cancers that affect men, but the results of these studies also indicate the presence of both immediate and sustained psychologic symptoms.5153 The resolution of abnormal results should theoretically relieve these transient emotional states, and this outcome has been observed in some studies.5356 However, many of the studies among women found subsets of women who continue to experience lingering psychologic symptoms weeks and even months after the resolution of the abnormality.47,49,5759 This finding suggests the need for research to understand and address the long-term consequences of psychologic distress, particularly as they may relate to future participation in cancer screening and receipt of other health care. Because the majority of studies that find significant psychologic distress associated with abnormal screening results have been conducted with self-referred samples that tend to be of fairly high socioeconomic status, the relevance of the findings for the general population of persons who have screen-detected abnormalities remains unclear. Several studies that attempted to intervene to decrease the negative psychologic effect after receipt of an abnormal result have had mixed results.50,60,61 Although, the goal of reducing even transient negative emotional states among individuals who receive a negative result may be warranted in and of itself, reducing distress among these patients may have a positive effect on receipt of follow-up procedures.48,57,62,63 Still, most of the studies that have focused on psychosocial outcomes have failed to include any information on receipt of follow-up care. Therefore, no clear link can be established between psychosocial sequelae of an abnormality and actual receipt of care. Inclusion of both outcomes in studies would help to advance the science in this area.
The appropriate and consistent measurement of these psychosocial outcomes is an important methodologic issue that can profoundly influence the results obtained. Studies have varied widely in the specific quality of life and psychologic outcomes measured, making comparisons between studies difficult.64 For example, some studies have assessed psychologic outcomes that are specific to the situation under study including cancer concern, illness worry, or concerns about follow-up procedures.46,49,57,61 Other studies have assessed more general psychosocial outcomes such as anxiety, depression, and generic measures of quality of life.47,50,53,65 Study samples in most of these studies are small,62,66 are mainly recruited from large medical centers,55,59 and many have been retrospective in nature.65,67 Many of these studies have been conducted outside the United States, so differences in characteristics between health care systems have implications for generalizability.46,47,49,50,53,61,68,69
Lesson 6: Substantial Variability Exists across Studies Regarding the Categories of Patients Who Are Considered Eligible for Follow-Up, the Definition of Appropriate Follow-Up, and the Time Window within Which Follow-Up Is Measured; Interpretation of the Literature Would Benefit Greatly from Standardization of These Definitions
Conditions under which follow-up is indicated
It is important in population studies to clarify conditions under which patients with abnormal screening test results should be considered eligible for follow-up. For example, it is not certain whether patients who received an abnormal screening FOBT result, but who underwent colonoscopy within a short period of time before the abnormal screening result, should undergo colonoscopic evaluation immediately. Similarly, older adults who have significant comorbidities may not be ideal candidates for endoscopic follow-up of an abnormal FOBT result. From a methodologic point of view, greater clarity is needed concerning conditions under which follow-up is recommended.
Definition of appropriate follow-up
We also found substantial variation in how outcome measures are defined in both observational and intervention studies of diagnostic follow-up after screening abnormalities. Outcome measures included physician intention to recommend complete diagnostic evaluation,41 consultation with a physician,70 receipt of at least one of the recommended procedures,11 receipt of a specific procedure (e.g., breast biopsy or colposcopy),5,25,71 or diagnostic resolution of abnormality (i.e., cancer or not cancer).10 This variation in outcome definition limits comparisons of follow-up in different populations across studies as well as the identification of the relative effectiveness of different types of interventions. Factors that may influence this variation include the level of the health care system that is targeted for the intervention, the heterogeneity of patient populations included in the study, and the lack of explicit or clear management guidelines for diagnostic follow-up.
Time period within which follow-up is measured
The lack of clear and generally available guidelines contributes to wide variations among studies regarding the time period within which measurement of outcomes is conducted.6,7275 Presumably, the appropriate recommended time interval for follow-up should relate to the natural history of the disease, but the interval may be influenced by the availability of recommended procedures and other organizational characteristics such as the size of the facility, staffing ratios, and wait time. Published reports are not always clear regarding whether such considerations explicitly determined the follow-up interval selected for the study or whether the follow-up period was selected on the basis of more practical considerations, such as availability of funding or the duration of a funded project. The selection of the time period within which measurement of outcomes is conducted is tied very closely to the related concept of timely follow-up and its converse, follow-up delay. A number of studies have examined predictors of follow-up (after development of a symptom) behavior on the part of patients as well as providers and the health care system. It is not clear, however, whether the results of such studies have informed intervention research designed to increase follow-up of abnormalities. In fact, some studies have used timeliness of follow-up as the main outcome.22,32,71,76,77 However, a widely accepted definition of timely follow-up has not been established, and what is considered timely varies significantly depending on the specific cancer in question and the specific type and severity of abnormality detected.7881 Ideally, all follow-up procedures should be completed so as to initiate treatment (if indicated) before further disease progression that could negatively affect morbidity and/or mortality. Also, a very short time to follow-up may be of little clinical importance in the case of very small cancers detected by screening.
A review of the published literature indicates that many studies, when operationalizing the time interval for follow-up, have taken into account the time period within which a specific procedure has been recommended. For example, studies may assess adherence to a repeat Pap smear over 6 months but may assess adherence to colposcopy over a much shorter period of time.6,13,73,80,82,83 However, even after considering the differences due to the type of recommended follow-up procedures, the time interval used to measure follow-up has varied substantially across studies. Some studies assessed adherence within 1 week after a recommended procedure was to be completed,27 whereas other studies assessed adherence months72,84 or even years after the recommended time interval for the follow-up procedure.73,74 In addition, some studies assessed adherence at one specific time,6,75 whereas other studies divided the follow-up window to determine adherence rates at multiple time points within the study period.22,32,71
The measurement time window is an important issue, because it can substantially influence the follow-up receipt rate that is observed. To some degree, the lengthier the observation period, the higher the follow-up receipt rate is likely to be. The length of time over which outcomes are measured can also affect whether or not an intervention effect is detected. The firmest support for the suggestion that the measurement time window can influence observed adherence rates is obtained from studies that have calculated adherence rates at more than one time point across the study period. Burack et al.71 found that by 60 days, only 32% of their sample was adherent to repeat mammogram recommendations after an initial abnormal mammogram, whereas 66% of the sample was adherent when assessed across the entire 11-month period of the study. Similarly, Frisch32 found that by 6 months, only 37% of the study sample had received a repeat Pap smear, whereas by 14 months, 69% of the sample had received one. Lurie and Welch85 found that only one-half of the total follow-up procedures completed by 8 months were completed by the first 2 months after an initial abnormal FOBT.
Physician recommendation for follow-up as an intermediate outcome
In defining outcomes of interventions to increase receipt of follow-up, it is important to distinguish between actual receipt of follow-up care and the intermediate step of whether a clinical recommendation for a follow-up service or procedure was made. The measurement of such intermediate outcomes is not always simple. This information is not consistently recorded in medical charts, is rarely available from administrative data, and may be unreliably recalled on patient self-report. Nevertheless, physician recommendation represents a critical step in the process of receipt of appropriate care and could provide important clues regarding the success or failure of an intervention. Some interventions are designed to change provider behavior in relation to the recommendation of appropriate diagnostic evaluations. Therefore, physician recommendation or ordering of the appropriate tests is a legitimate outcome measure. Such an intermediate measure may be insufficient for interventions directed at the patient level, where actual receipt of follow-up care by the patient is more relevant. However, to compare physician recommendation with patient receipt of appropriate diagnostic services, studies including both these outcomes are needed.
A major factor contributing to the measurement inconsistencies outlined in this section is that widely available guidelines related to cancer screening (e.g., guidelines from the U.S. Preventive Services Task Force and the American Cancer Society) have provided few details related to the nature of appropriate follow-up for patients who have an abnormal screening test result. Even when relatively clear specialty guidelines exist (e.g., the National Comprehensive Cancer Network), they are not widely known or disseminated, and the level of adherence by medical professionals to such guidelines is not clear.
Lesson 7: Studies Are Needed To Establish the Validity of Various Methods of Obtaining Data on Follow-Up; Until Such Information Is Reliably Available, Investigators Should Seriously Consider Incorporating Multiple Modes of Data Collection into Their Studies
There is considerable variability in how data on receipt of follow-up care are collected, including medical records,23,28,30,63 administrative data,85 and self-report.7 Differences in how data are collected also can affect the follow-up rates observed. For example, a study of follow-up after positive FOBTs conducted within a managed care organization compared follow-up rates by using different approaches to measurement including physician self-report, medical charts, and administrative data.40 Substantial variation was found, depending on the specific data collection method used.
Use of administrative data for measurement of follow-up can be complicated by health care plan coverage policies, because these data may only contain information on services that are reimbursed by a specific health plan or program. Most administrative data are related to billing. If follow-up care is provided by a physician outside the health plan or the follow-up procedure is not covered by the plan, administrative data will not contain this information. Also, although administrative data usually can identify whether a test occurred, the reason for the procedure and the results of the test often are not included in administrative databases. Finally, patient self-report of follow-up may be problematic when the sequence of events or procedures is complicated. In addition, several studies have suggested that patients overreport screening procedures compared with procedures included in the medical record.8688 Patients identified as having an abnormal screening result may also be more likely to overreport completion of follow-up procedures. However, this bias may be less pronounced, because the event of receiving follow-up for an abnormal result is likely to be more salient and thus recalled more accurately.45
Unfortunately, because of limitations associated with all types of measurement of follow-up care, there is no clear, gold standard reporting mechanism. Differences in reporting methods can result in differing levels of misclassification across studies, thus limiting comparisons of observational studies. One solution is to use several methods, so that estimates from one method can be supplemented by estimates from another. For example, in colorectal cancer screening, Myers et al.40 showed that an internal chart audit (primary care physician report) combined with administrative data from a managed care organization produced abnormal FOBT follow-up rates comparable to combined external chart audit plus administrative data. This issue is of less concern for interventions that use randomized designs because individuals in intervention and control groups may be equally likely to have data recorded in medical records, in administrative data, or to overreport their experience.
Lesson 8: Studies on the Follow-Up of Abnormal Findings Have Used a Variety of Patient Inclusion Criteria, Which Generally Are Based on the Degree of Abnormality; Because the Observed Follow-Up Rate and the Probability of Intervention Effectiveness Are Likely To Be Related to the Severity of the Abnormal Finding, It Is Important To Keep These Relations in Mind when Interpreting Findings
The types of abnormalities selected for inclusion in studies vary considerably. Inclusion criteria are often based on the degree, level, or seriousness of the abnormality.6,13,7173,83,84,8991 For example, Sienko et al.91 only included women with a mammographically detected abnormality necessitating a biopsy, whereas Haas et al.90 included women with any type of abnormal mammogram results as well as women with clinical breast symptoms. In the area of cervical cancer, Palm et al.83 included a wide range of abnormalities from mild dysplasia to invasive cancer, whereas Batal et al.84 included only women with atypical squamous cells of undetermined significance or any dysplasia. This lack of consistency in the types of abnormalities included in the studies results in very different patient populations and makes comparisons across studies difficult.
In addition, the follow-up rates observed as well as the probability of intervention success are directly related to the types of abnormalities targeted. The literature provides clear evidence for a positive relation between the severity of the abnormality and follow-up rates. This observation holds true for abnormalities related to breast, cervical, and colorectal cancer and has been demonstrated in both observational82,91 and intervention trials.23,35,63 The degree of abnormality is a consistent predictor of follow-up rates, even when the intervention strategy implemented has not been specifically tailored to the severity of the abnormality.35,72 This underlying relation between the severity of the abnormality and the probability of follow-up care has the potential to be a serious influence on baseline follow-up rates, as well as on the capacity of an intervention to have an effect. For example, if only severe abnormalities are included in a trial, a modest intervention may not be able to achieve an additional effect over and above the strong trend toward receipt of follow-up care. Conversely, more severe abnormalities may be more sensitive to patient, physician, or system-directed interventions.
Another issue to consider is that, because not all abnormalities are equally serious, interventions might be more effective if they were targeted or tailored according to the degree of severity. However, very few studies have taken the degree of abnormality into account in the development and delivery of their interventions. Examples include studies by Marcus et al.23,24 and Kaplan et al.,28 in which more intensive interventions were provided to women with more severe cervical abnormalities. Similarly, Paskett et al.27 designed two different educational/motivational brochures— one for benign atypia one for dysplasia.
In conducting intervention research to increase follow-up of abnormal findings, our MIRDI framework cautions us to be cognizant of all four levels of the health care system (patient, provider, practice, policy), even if a particular intervention is directed only at one of the levels. The model assumes that interactions among the various levels can substantially influence intervention effectiveness. Although the decision to target only one level of the system is often a pragmatic one, it is imperative that researchers at least consider the potential influence of other levels on study outcomes. Consideration of the interdependence of factors influencing follow-up care at all levels is critical, and future studies should examine the comparative and marginal benefit of interventions targeting multiple levels of the health care system.
The majority of interventions aimed at improving follow-up of screening abnormalities have been conducted at the patient level and among women with abnormal Pap smears. Expansion of our research focus to improve follow-up of breast and colorectal abnormalities and to target provider, practice, and policy levels is important. No reports were found in the literature on interventions targeting follow-up of an abnormal prostate cancer screening examination. This lack of information is understandable, given the lack of consensus regarding the value of aggressive follow-up and treatment of screen-detected abnormalities. Because the outcomes that should be measured in intervention trials of follow-up for an abnormal prostate screening examination are not clear-cut, it becomes important to consider the realm of informed decision-making.92
A number of methodologic issues compromise our ability to assess accurately the effect of interventions that have been tested. These issues also limit our ability to make comparisons across studies. Attention needs to be directed to establishing standards that encourage the uniform assessment of outcomes in future trials, thus allowing comparability across studies.
It is important to move beyond the immediate intervention context and study the effect of increased receipt of diagnostic follow-up on population trends in incidence (e.g., cervical and colorectal cancer) and mortality (e.g., breast, cervical, colorectal, prostate cancer) by using simulation models. Most simulation models of the effects of population screening on incidence and mortality assume very high rates of follow-up. The reality falls far short of these assumptions. Therefore, simulation models should take into consideration various rates of follow-up completion as well as the timing within which follow-up occurs, while simultaneously taking into account the various recommended screening intervals (e.g., annual, every 2 years, every 5 years, etc.) even for the same cancer.
Another issue to consider is that in randomized, controlled trials of screening effectiveness, due to self selection of participants, the rates of receipt of follow-up care are likely to overestimate the benefit of screening among individuals in the general population, for whom follow-up rates are likely lower. Thus, the realized benefits of population-based screening are also likely to be lower. This likelihood argues strongly in favor of the need to intensify efforts to test interventions to increase receipt of appropriate follow-up care.
We will also need to broaden our definitions of “screening” to accommodate the rapid advances in technology for the detection of disease earlier in its natural course (e.g., tests for human papilloma virus infection and for neoplastic changes identified in breast ductal fluid) and genetic tests for cancer susceptibility. As a whole new class of screening modalities emerges, we will need to change our thinking regarding the “conditions” for which we are screening as well as physician and patient decision-making related to follow-up of abnormal findings.
The ultimate goal of conducting intervention research is to identify the most effective strategies, which can then be implemented widely, to achieve a populationwide impact on cancer morbidity and mortality. An important aspect of achieving the transfer of research findings to wide-scale application is communication between the research and practice communities.4 In our MIRDI framework (Fig. 1), this step is represented in the middle portion of the diagram. Researchers themselves can participate in the dissemination process by publishing articles in scientific journals and presenting their findings at professional meetings, although the reach of this strategy is likely to be fairly limited. Advocacy groups, the media, and governmental agencies are additional channels for dissemination. These channels have the capability of reaching a much broader audience and potentially having a larger effect, particularly at the policy level. Finally, research on dissemination itself should be part of the public health agenda.
Footnotes
The opinions expressed herein do not necessarily reflect the views of the National Cancer Institute or the U.S. Government.
*This article is a U.S. Government work and, as such, is in the public domain in the United States of America.
1. Yabroff KR, Washington KS, Leader A, Neilson E, Mandelblatt J. Is the promise of cancer-screening programs being compromised? Quality of follow-up care after abnormal screening results. Med Care Res Rev. 2003;60:294–331. [PubMed]
2. Elmore JG, Barton MB, Moceri VM, Polk S, Arena PJ, Fletcher SW. Ten-year risk of false positive screening mammograms and clinical breast examinations. N Engl J Med. 1998;338:1089–1096. [PubMed]
3. Christiansen CL, Wang F, Barton MB, et al. Predicting the cumulative risk of false-positive mammograms. J Natl Cancer Inst. 2000;92:1657–1666. [PubMed]
4. Glasgow RE, Marcus AC, Bull SS, Wilson KM. Disseminating effective cancer screening interventions. Cancer. 2004;101(5 Suppl):1239–1280. [PubMed]
5. Lerman C, Hanjani P, Caputo C, et al. Telephone counseling improves adherence to colposcopy among lower-income minority women. J Clin Oncol. 1992;10:330–333. [PubMed]
6. Paskett ED, White E, Carter WB, Chu J. Improving follow-up after an abnormal Pap smear: a randomized controlled trial. Prev Med. 1990;19:630–641. [PubMed]
7. McKee MD, Lurio J, Marantz P, Burton W, Mulvihill M. Barriers to follow-up of abnormal Papanicolaou smears in an urban community health center. Arch Fam Med. 1999;8:129–134. [PubMed]
8. Kerner JF, Yedidia M, Padgett D, et al. Realizing the promise of breast cancer screening: clinical follow-up after abnormal screening among black women. Prev Med. 2003;37:92–101. [PubMed]
9. Funke BL, Nicholson ME. Factors affecting patient compliance among women with abnormal Pap smears. Patient Educ Couns. 1993;20:5–15. [PubMed]
10. Mandelblatt J, Traxler M, Lakin P, Kanetsky P, Kao R. Targeting breast and cervical cancer screening to elderly poor black women: who will participate? The Harlem Study Team. Prev Med. 1993;22:20–33. [PubMed]
11. Crane LA. Social support and adherence behavior among women with abnormal Pap smears. J Cancer Educ. 1996;11:164–173. [PubMed]
12. Kaplan CP, Bastani R, Marcus A, Breslow L, Nasseri K, Chen L. Low-income women with cervical abnormalities: individual and system factors affecting follow-up. J Womens Health. 1995;4:179–188.
13. McCarthy BD, Yood MU, Boohaker EA, Ward RE, Rebner M, Johnson CC. Inadequate follow-up of abnormal mammograms. Am J Prev Med. 1996;12:282–288. [PubMed]
14. Michielutte R, Diseker RA, Young LD, May WJ. Noncompliance in screening follow-up among family planning clinic patients with cervical dysplasia. Prev Med. 1985;14:248–258. [PubMed]
15. Cejtin HE, Komaroff E, Massad LS, et al. Adherence to colposcopy among women with HIV infection. J Acquir Immune Defic Syndr. 1999;22:247–252. [PubMed]
16. Melnikow J, Chan BK, Stewart GK. Do follow-up recommendations for abnormal Papanicolaou smears influence patient adherence? Arch Fam Med. 1999;8:510–514. [PubMed]
17. Myers RE, Balshem AM, Wolf TA, Ross EA, Millner L. Screening for colorectal neoplasia: physicians’ adherence to complete diagnostic evaluation. Am J Public Health. 1993;83:1620–1622. [PubMed]
18. Myers RE, Hyslop T, Gerrity M, et al. Physician intention to recommend complete diagnostic evaluation in colorectal cancer screening. Cancer Epidemiol Biomarkers Prev. 1999;8:587–593. [PubMed]
19. Boohaker EA, Ward RE, Uman JE, McCarthy BD. Patient notification and follow-up of abnormal test results. A physician survey. Arch Intern Med. 1996;156:327–331. [PubMed]
20. Klabunde CN, Riley GS, Mandelson MT, Frame PS, Brown ML. Health plan policies and programs for colorectal cancer screening: a national profile. Am J Manag Care. 2004;10:273–279. [PubMed]
21. Gusmano MK, Fairbrother G, Park H. Exploring the limits of the safety net: community health centers and care for the uninsured. Health Aff (Millwood) 2002;21:188–194. [PubMed]
22. Manfredi C, Lacey L, Warnecke R. Results of an intervention to improve compliance with referrals for evaluation of suspected malignancies at neighborhood public health centers. Am J Public Health. 1990;80:85–87. [PubMed]
23. Marcus AC, Kaplan CP, Crane LA, et al. Reducing loss-to-follow-up among women with abnormal Pap smears. Results from a randomized trial testing an intensive follow-up protocol and economic incentives. Med Care. 1998;36:397–410. [PubMed]
24. Marcus AC, Crane LA, Kaplan CP, et al. Improving adherence to screening follow-up among women with abnormal Pap smears: results from a large clinic-based trial of three intervention strategies. Med Care. 1992;30:216–230. [PubMed]
25. Miller SM, Siejak KK, Schroeder CM, Lerman C, Hernandez E, Helm CW. Enhancing adherence following abnormal Pap smears among low-income minority women: a preventive telephone counseling strategy. J Natl Cancer Inst. 1997;89:703–708. [PubMed]
26. Lauver D, Rubin M. Message framing, dispositional optimism, and follow-up for abnormal Papanicolaou tests. Res Nurs Health. 1990;13:199–207. [PubMed]
27. Paskett ED, Phillips KC, Miller ME. Improving compliance among women with abnormal Papanicolaou smears. Obstet Gynecol. 1995;86:353–359. [PubMed]
28. Kaplan CP, Bastani R, Belin TR, Marcus A, Nasseri K, Hu MY. Improving follow-up after an abnormal Pap smear: results from a quasi-experimental intervention study. J Womens Health Gend Based Med. 2000;9:779–790. [PubMed]
29. Ell K, Padgett D, Vourlekis B, et al. Abnormal mammogram follow-up: a pilot study in women with low income. Cancer Pract. 2002;10:130–138. [PubMed]
30. Engelstad LP, Stewart SL, Nguyen BH, et al. Abnormal Pap smear follow-up in a high-risk population. Cancer Epidemiol Biomarkers Prev. 2001;10:1015–1020. [PubMed]
31. Holschneider CH, Felix JC, Satmary W, Johnson MT, Sandweiss LM, Montz FJ. A single-visit cervical carcinoma prevention program offered at an inner city church: a pilot project. Cancer. 1999;86:2659–2667. [PubMed]
32. Frisch LE. Effectiveness of a case management protocol in improving follow-up and referral of Papanicolaou smears indicating cervical intraepithelial neoplasia. J Am Coll Health. 1986;35:112–115. [PubMed]
33. Block B, Branham RA. Efforts to improve the follow-up of patients with abnormal Papanicolaou test results. J Am Board Fam Pract. 1998;11:1–11. [PubMed]
34. Gifford MS, Stone IK. Quality, access, and clinical issues in a nurse practitioner colposcopy outreach program. Nurse Pract. 1993;18:25–26. [PubMed]
35. Lacey L, Whitfield J, DeWhite W, et al. Referral adherence in an inner city breast and cervical cancer screening program. Cancer. 1993;72:950–955. [PubMed]
36. Michielutte R, Dignan M, Bahnson J, Wells HB. The Forsyth County Cervical Cancer Prevention Project—II. Compliance with screening follow-up of abnormal cervical smears. Health Educ Res. 1994;9:421–432. [PubMed]
37. Prislin MD, Dinh T, Giglio M. On-site colposcopy services in a family practice residency clinic: impact on physician test-ordering behavior, patient compliance, and practice revenue generation. J Am Board Fam Pract. 1997;10:259–264. [PubMed]
38. Curtis P, Varenholt JJ, Skinner B, Addison L, Resnick J, Kebede M. Development of a Pap smear quality-assurance system in family practice. Fam Med. 1993;25:135–139. [PubMed]
39. Monticciolo DL, Sickles EA. Computerized follow-up of abnormalities detected at mammography screening. AJR Am J Roentgenol. 1990;155:751–753. [PubMed]
40. Myers RE, Fishbein G, Hyslop T, et al. Measuring complete diagnostic evaluation in colorectal cancer screening. Cancer Detect Prev. 2001;25:174–182. [PubMed]
41. Myers RE, Turner B, Weinberg DS, et al. Impact of a physician-oriented intervention on follow-up in colorectal cancer screening. Prev Med. 2004;38:375–381. [PubMed]
42. Stern MA, Fendrick AM, McDonnell WM, Gunaratnam N, Moseley R, Chey WD. A randomized, controlled trial to assess a novel colorectal cancer screening strategy: the conversion strategy—a comparison of sequential sigmoidoscopy and colonoscopy with immediate conversion from sigmoidoscopy to colonoscopy in patients with an abnormal screening sigmoidoscopy. Am J Gastroenterol. 2000;95:2074–2079. [PubMed]
43. Freeman HP, Muth BJ, Kerner JF. Expanding access to cancer screening and clinical follow-up among the medically underserved. Cancer Pract. 1995;3:19–30. [PubMed]
44. Eilers GM, Swanson TK. Using reminder systems to improve Papanicolaou test follow-up. An example of continuous quality improvement. Arch Fam Med. 1993;2:1136–1140. [PubMed]
45. Vernon SW, Briss PA, Tiro J, Warnecke RB. Some methodological lessons learned from cancer screening research. Cancer. 2004;101(5 Suppl):1131–1145. [PubMed]
46. Aro AR, Pilvikki AS, van Elderen TM, van der PE, van der Kamp LJ. False-positive findings in mammography screening induces short-term distress— breast cancer-specific concern prevails longer. Eur J Cancer. 2000;36:1089–1097. [PubMed]
47. Bell S, Porter M, Kitchener H, Fraser C, Fisher P, Mann E. Psychological response to cervical screening. Prev Med. 1995;24:610–616. [PubMed]
48. Lerman CE, Rimer BK. Psychosocial impact of cancer screening. Oncology (Huntingt) 1993;7:67–72. [PubMed]
49. Lowe JB, Balanda KP, Del Mar C, Hawes E. Psychologic distress in women with abnormal findings in mass mammography screening. Cancer. 1999;85:1114–1118. [PubMed]
50. Peters T, Somerset M, Baxter K, Wilkinson C. Anxiety among women with mild dyskaryosis: a randomized trial of an educational intervention. Br J Gen Pract. 1999;49:348–352. [PMC free article] [PubMed]
51. Gustafsson O, Theorell T, Norming U, Perski A, Ohstrom M, Nyman CR. Psychological reactions in men screened for prostate cancer. Br J Urol. 1995;75:631–636. [PubMed]
52. Mant D, Fitzpatrick R, Hogg A, et al. Experiences of patients with false positive results from colorectal cancer screening. Br J Gen Pract. 1990;40:423–425. [PMC free article] [PubMed]
53. Parker MA, Robinson MH, Scholefield JH, Hardcastle JD. Psychiatric morbidity and screening for colorectal cancer. J Med Screen. 2002;9:7–10. [PubMed]
54. Morris T, Greer S. Psychological characteristics of women electing to attend a breast screening clinic. Clin Oncol. 1982;8:113–119. [PubMed]
55. Romsaas EP, Malec JF, Javenkoski BR, Trump DL, Wolberg WH. Psychological distress among women with breast problems. Cancer. 1986;57:890–895. [PubMed]
56. Scaf-Klomp W, Sanderman R, van de Wiel HB, Otter R, van den Heuvel WJ. Distressed or relieved? Psychological side effects of breast cancer screening in The Netherlands. J Epidemiol Community Health. 1997;51:705–710. [PMC free article] [PubMed]
57. Brett J, Austoker J. Women who are recalled for further investigation for breast screening: psychological consequences 3 years after recall and factors affecting re-attendance. J Public Health Med. 2001;23:292–300. [PubMed]
58. Brett J, Austoker J, Ong G. Do women who undergo further investigation for breast screening suffer adverse psychological consequences? A multi-centre follow-up study comparing different breast screening result groups five months after their last breast screening appointment. J Public Health Med. 1998;20:396–403. [PubMed]
59. Lauver DR, Kruse K, Baggot A. Women’s uncertainties, coping, and moods regarding abnormal Papanicolaou results. J Womens Health Gend Based Med. 1999;8:1103–1112. [PubMed]
60. Wilkinson C, Jones JM, McBride J. Anxiety caused by abnormal result of cervical smear test: a controlled trial. BMJ. 1990;300:440. [PMC free article] [PubMed]
61. Wolfe C, Doherty I, Raju KS, Holtom R, Richardson P. First steps in the development of an information and counselling service for women with an abnormal smear result. Eur J Obstet Gynecol Reprod Biol. 1992;45:201–206. [PubMed]
62. Lerman C, Miller SM, Scarborough R, Hanjani P, Nolte S, Smith D. Adverse psychologic consequences of positive cytologic cervical screening. Am J Obstet Gynecol. 1991;165:658–662. [PubMed]
63. Paskett ED, Rimer BK. Psychosocial effects of abnormal Pap tests and mammograms: a review. J Womens Health. 1995;4:73–82.
64. Rimer BK, Bluman LG. The psychosocial consequences of mammography. J Natl Cancer Inst Monogr. 1997;22:131–138. [PubMed]
65. Gram IT, Lund E, Slenker SE. Quality of life following a false positive mammogram. Br J Cancer. 1990;62:1018–1022. [PMC free article] [PubMed]
66. McDonald TW, Neutens JJ, Fischer LM, Jessee D. Impact of cervical intraepithelial neoplasia diagnosis and treatment on self-esteem and body image. Gynecol Oncol. 1989;34:345–349. [PubMed]
67. Lerman C, Trock B, Rimer BK, Boyce A, Jepson C, Engstrom PF. Psychological and behavioral implications of abnormal mammograms. Ann Intern Med. 1991;114:657–661. [PubMed]
68. Bennetts A, Irwig L, Oldenburg B, et al. PEAPS-Q: a questionnaire to measure the psychosocial effects of having an abnormal Pap smear. Psychosocial Effects of Abnormal Pap Smears Questionnaire. J Clin Epidemiol. 1995;48:1235–1243. [PubMed]
69. Miles AE, Cockburn J, Smith R, Wardle FJ. A perspective from countries using organized screening programs. Cancer. 2004;101(5 Suppl):1201–1213. [PubMed]
70. Cummings KM, Michalek A, Mettlin C, Mittelman A. Screening for colorectal cancer using the Hemoccult II stool guaiac slide test. Cancer. 1984;53:2201–2205. [PubMed]
71. Burack RC, Simon MS, Stano M, George J, Coombs J. Follow-up among women with an abnormal mammogram in an HMO: is it complete, timely, and efficient? Am J Manag Care. 2000;6:1102–1113. [PubMed]
72. Cardin VA, Grimes RM, Jiang ZD, Pomeroy N, Harrell L, Cano P. Low-income minority women at risk for cervical cancer: a process to improve adherence to follow-up recommendations. Public Health Rep. 2001;116:608–616. [PMC free article] [PubMed]
73. Helvie MA, Pennes DR, Rebner M, Adler DD. Mammographic follow-up of low-suspicion lesions: compliance rate and diagnostic yield. Radiology. 1991;178:155–158. [PubMed]
74. Kavanagh AM, Simpson JM. Predicting nonattendance for colposcopy clinic follow-up after referral for an abnormal Pap smear. Aust N Z J Public Health. 1996;20:266–271. [PubMed]
75. Lerman C, Ross E, Boyce A, et al. The impact of mailing psychoeducational materials to women with abnormal mammograms. Am J Public Health. 1992;82:729–730. [PubMed]
76. Chang SW, Kerlikowske K, Napoles-Springer A, Posner SF, Sickles EA, Perez-Stable EJ. Racial differences in timeliness of follow-up after abnormal screening mammography. Cancer. 1996;78:1395–1402. [PubMed]
77. Sainsbury R, Johnston C, Haward B. Effect on survival of delays in referral of patients with breast-cancer symptoms: a retrospective analysis. Lancet. 1999;353:1132–1135. [PubMed]
78. Coates AS. Breast cancer: delays, dilemmas, and delusions. Lancet. 1999;353:1112–1113. [PubMed]
79. Facione NC. Delay versus help seeking for breast cancer symptoms: a critical review of the literature on patient and provider delay. Soc Sci Med. 1993;36:1521–1534. [PubMed]
80. Mitchell H, Medley G. Adherence to recommendations for early repeat cervical smear tests. BMJ. 1989;298:1605–1607. [PMC free article] [PubMed]
81. Richards MA, Westcombe AM, Love SB, Littlejohns P, Ramirez AJ. Influence of delay on survival in patients with breast cancer: a systematic review. Lancet. 1999;353:1119–1126. [PubMed]
82. Goodman KA, Birdwell RL, Ikeda DM. Compliance with recommended follow-up after percutaneous breast core biopsy. AJR Am J Roentgenol. 1998;170:89–92. [PubMed]
83. Palm BT, Kant AC, Visser EA, Vooijs GP, van den Bosch WJ, van Weel C. The effect of the family physician on improving follow-up after an abnormal Pap smear. Int J Qual Health Care. 1997;9:277–282. [PubMed]
84. Batal H, Biggerstaff S, Dunn T, Mehler PS. Cervical cancer screening in the urgent care setting. J Gen Intern Med. 2000;15:389–394. [PMC free article] [PubMed]
85. Lurie JD, Welch HG. Diagnostic testing following fecal occult blood screening in the elderly. J Natl Cancer Inst. 1999;91:1641–1646. [PubMed]
86. Gordon NP, Hiatt RA, Lampert DI. Concordance of self-reported data and medical record audit for six cancer screening procedures. J Natl Cancer Inst. 1993;85:566–570. [PubMed]
87. Hiatt RA, Perez-Stable EJ, Quesenberry C, Jr, Sabogal F, Otero-Sabogal R, McPhee SJ. Agreement between self-reported early cancer detection practices and medical audits among Hispanic and non-Hispanic white health plan members in northern California. Prev Med. 1995;24:278–285. [PubMed]
88. McGovern PG, Lurie N, Margolis KL, Slater JS. Accuracy of self-report of mammography and Pap smear in a low-income urban population. Am J Prev Med. 1998;14:201–208. [PubMed]
89. Carey P, Gjerdingen DK. Follow-up of abnormal Papanicolaou smears among women of different races. J Fam Pract. 1993;37:583–587. [PubMed]
90. Haas JS, Cook EF, Puopolo AL, Burstin HR, Brennan TA. Differences in the quality of care for women with an abnormal mammogram or breast complaint. J Gen Intern Med. 2000;15:321–328. [PMC free article] [PubMed]
91. Sienko DG, Hahn RA, Mills EM, et al. Mammography use and outcomes in a community. The Greater Lansing Area Mammography Study. Cancer. 1993;71:1801–1809. [PubMed]
92. Rimer BK, Briss PA, Zeller PK, Chan E, Woolf SH. Informed decision making: what is its role in cancer screening? Cancer. 2004;101(5 Suppl):1214–1228. [PubMed]