PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of jgimedspringer.comThis journalToc AlertsSubmit OnlineOpen Choice
 
J Gen Intern Med. 2012 July; 27(7): 831–838.
Published online 2012 February 2. doi:  10.1007/s11606-012-1991-y
PMCID: PMC3378745

Clinical Decision Support to Promote Safe Prescribing to Women of Reproductive Age: A Cluster-Randomized Trial

Eleanor Bimla Schwarz, MD, MS,corresponding author1,2,3,4 Sara M. Parisi, MS, MPH,1 Steven M. Handler, MD, PhD,4,5 Gideon Koren, MD,6 Elan D. Cohen, MS,1 Grant J. Shevchik, MD,4 and Gary S. Fischer, MD1,4

ABSTRACT

BACKGROUND

Potentially teratogenic medications are frequently prescribed without provision of contraceptive counseling.

OBJECTIVE

To evaluate whether computerized clinical decision support (CDS) can increase primary care providers’ (PCPs’) provision of family planning services when prescribing potentially teratogenic medications.

DESIGN

Cluster-randomized trial conducted in one academic and one community-based practice between October of 2008 and April of 2010.

PARTICIPANTS/INTERVENTIONS

Forty-one PCPs were randomized to receive one of two types of CDS which alerted them to risks of medication-induced birth defects when ordering potentially teratogenic medications for women who may become pregnant. The ‘simple’ CDS provided a cautionary alert; the ‘multifaceted’ CDS provided tailored information and links to a structured order set designed to facilitate safe prescribing. Both CDS systems alerted PCPs about medication risk only once per encounter.

MAIN MEASURES

We assessed change in documented provision of family planning services using data from 35,110 encounters and mixed-effects models. PCPs completed surveys before and after the CDS systems were implemented, allowing assessment of change in PCP-reported counseling about the risks of medication-induced birth defects and contraception.

KEY RESULTS

Both CDS systems were associated with slight increases in provision of family planning services when potential teratogens were prescribed, without a significant difference in improvement by CDS complexity (p = 0.87). Because CDS was not repeated, 13% of the times that PCPs received CDS they substituted another potential teratogen. PCPs reported significant improvements in several counseling and prescribing practices. The multifaceted group reported a greater increase in the number of times per month they discussed the risks of medication use during pregnancy (multifaceted: +4.9 ± 7.0 vs. simple: +0.8 ± 3.2, p = 0.03). The simple CDS system was associated with greater clinician satisfaction.

CONCLUSIONS

CDS systems hold promise for increasing provision of family planning services when fertile women are prescribed potentially teratogenic medications, but further refinement of these systems is needed.

Electronic supplementary material

The online version of this article (doi:10.1007/s11606-012-1991-y) contains supplementary material, which is available to authorized users.

KEY WORDS: preconception care, contraceptive counseling, health IT, decision support, birth defects

INTRODUCTION

U.S. women of reproductive-age annually receive 12 million prescriptions for potentially teratogenic medications.1 Physicians who provide preconception and contraceptive counseling when prescribing a potentially teratogenic medication may help women avoid medication-induced birth defects, as women who are using effective contraception at the time they fill potentially teratogenic prescriptions are less likely to become pregnant.2 Although many women depend on their clinicians to inform them when a medication poses a risk during pregnancy,3,4 less than 50% of women prescribed potentially teratogenic medications in ambulatory care settings receive contraceptive counseling.1,2 Due in part to this lack of counseling, approximately 6% of US pregnancies are exposed to potentially teratogenic medications;5,6 some women choose to terminate pregnancies exposed to teratogens even when the risk of a birth defect is low.79

Primary care physicians (PCPs) prescribe the majority of potentially teratogenic medications.1,2 Thus, the goal of this study was to evaluate clinical decision support (CDS) designed to alert PCPs when they prescribed potentially teratogenic medications to women who may become pregnant. Because in most cases an equally effective non-teratogenic medication is not available, the goal was not to reduce use of potentially teratogenic medications. Rather, we hypothesized that this CDS would increase the frequency with which PCPs counseled their patients about the risks of birth defects and use of contraception. We also hypothesized that multifaceted CDS with a structured order set and tailored alerts incorporating information on women’s likelihood of pregnancy would lead to greater improvements than CDS that simply warned PCPs about the use of medication during pregnancy.

METHODS

Setting and Participants

We conducted a cluster-randomized trial at two practices in Western Pennsylvania. One was an urban, academic general internal medicine clinic, the other, a suburban, community-based family medicine practice. Both belonged to a large health system and had been using the EpicCare® electronic medical record (EMR) (Epic Systems Corporation, Verona, Wisconsin) since 1999. All PCPs who were not co-investigators were invited to participate.

Intervention Design

PCPs were randomized to receive either “simple” or “multifaceted” CDS between October 2008 and June 2009. From July 2009 to April 2010, the multifaceted CDS was de-activated and we continued to follow the PCPs who had received multifaceted CDS, allowing us to compare the effect of the simple CDS to no CDS. The simple CDS stated “Concern has been raised about the use of this medication during pregnancy” when a potentially teratogenic medication was ordered for a 18–50-year-old female with no indication of sterility in her EMR. The multifaceted CDS expanded upon this by providing a structured order set and tailored alert text that incorporated intake data on women’s pregnancy intentions and contraceptive use (see online appendix). Both CDS systems delivered disruptive alerts requiring PCP acknowledgement.

The design of this CDS was informed by focus groups with clinicians10,11 and patients,3 and the CDS literature.12,13 Due to prior concerns about misclassification of potentially teratogenic medications,14 alerts were triggered by a subset of the US FDA class D or X medications and other medications which were felt by experts in the field of teratology to pose a significant risk in early pregnancy (see online appendix). CDS alerts fired when the PCP ordered or renewed a prescription. In an effort to avoid “alert fatigue”, an alert only appeared for the first potentially teratogenic medication ordered during an encounter. PCPs were not specifically trained about teratogenesis, contraception, or use of this CDS; study clinicians had used multiple other CDS systems for years.

Data Collection and Outcomes

These systems were evaluated using EMR data and physician surveys. De-identified EMR data was abstracted for all encounters with study physicians by females aged 18–50 years during three time periods: “T0”—10-month period prior to CDS activation, “T1”—9-month period during which physicians received either simple or multifaceted CDS, and “T2”—9 month period during which the multifaceted CDS was deactivated. We abstracted the following data: type of clinical encounter (new vs. return, with usual PCP vs. different PCP); all potentially teratogenic medications prescribed (as defined by our list); all pregnancy tests ordered; new or existing contraceptive prescriptions; documentation of contraceptive counseling within the past 3 months; referrals for placement of a contraceptive device; patients’ age, race and marital status. Our primary outcome, provision of family planning services when a potentially teratogenic medication was prescribed, was a composite defined as EMR evidence of any of the following: pregnancy testing, a new contraceptive prescription, a non-expired contraceptive prescription, contraceptive counseling, or referral for placement of a contraceptive device. Encounters with indication of surgical sterilization, hysterectomy, menopause, or infertility were excluded.

PCPs were asked to complete an online survey prior to CDS activation and a follow-up survey one year later. PCPs were asked how many times in the last month they or their staff provided preconception counseling or counseling about risks of medication-induced birth defects. PCPs were also asked how many times in the last month they or their staff provided contraceptive counseling, or contraceptive prescriptions, or referrals. Finally, physicians provided information on sociodemographics and practice characteristics and rated their satisfaction with the CDS on a 10-point scale.

Statistical Analyses

EMR data were analyzed at the encounter level. We compared the proportion of encounters with a teratogenic prescription and the proportion of such encounters with evidence of family planning services over the three time points, overall and by CDS group. We tested for significant differences in changes between CDS groups by using adjusted mixed-effects logistic regression models including an interaction term of study group and time point.15 We calculated marginal predicted proportions (holding covariates at their grand means) and estimated the adjusted percentage point difference in improvement between the groups with 95% confidence intervals. Separate models were run to assess the change between T0 and T1, and between T1 and T2. Models were adjusted for physician type, physician gender, patient age, usual PCP visit (Y/N), and new patient visit (Y/N). Models were not adjusted for patient race or marital status because of concern about the accuracy of this data; however, we confirmed that neither variable affected point estimates in preliminary models. Initial models included physician and patient as crossed random effects, but the physician random effect was dropped from the family planning models because it was not significantly different from zero. We also tested for evidence of a significant linear trend across all three-time periods within encounters that received simple CDS alerts. For comparison, we investigated the change in provision of family planning services during visits that did not involve prescription of a potential teratogen (i.e. visits to simple and multifaceted PCPs that did not involve a CDS alert). Finally, we calculated the residual intra-class correlation coefficient (ICC).16

Using the survey data, we compared the mean level of satisfaction with the CDS received and self-reported changes in practice patterns. Overall changes in practice patterns were assessed using Wilcoxon matched pair signed-rank tests. To compare changes by CDS group, change scores were calculated for each PCP and differences by study group in the mean change score were assessed using independent samples t-tests or Wilcoxon rank sum tests. Only PCPs with complete pre-and post-intervention surveys were included in this analysis.

We also conducted a series of secondary EMR analyses. We looked at how often the multifaceted group accessed the structured order set. We also investigated whether the medication that triggered the CDS was prescribed despite the CDS warning. We calculated the predicted change for each practice. Finally, we looked at patterns in provision of specific family planning services, and patterns by the clinical indication of the medication. All P-values reported about the EMR data are from mixed-effects logistic regression models adjusted for clustering. All t-tests were two-tailed. P-values less than 0.05 were considered statistically significant. This study was approved by the University of Pittsburgh Institutional Review Board. All analyses were performed with Stata 11.0 IC (StataCorp, College Station, TX).

RESULTS

Figure 1 describes the flow of participating physicians and study encounters. There were no significant differences in PCP characteristics by CDS group (Table 1). We abstracted EMR data from 35,110 encounters with study physicians by 9,972 female patients aged 18–50 years with no EMR indication of sterilization, infertility or menopause (whether or not a potential teratogen was prescribed). At baseline, there were no significant differences between encounters with PCPs who received simple or multifaceted CDS. (Table 1)

Figure 1
CONSORT flow diagram describing physician clusters and patient encounters from the time of recruitment to analysis. CDS indicates clinical decision support. ‘Encounter’ = visit made to a study PCP by a woman aged 18–50 years with ...
Table 1
Baseline Characteristics of Participating Physicians and Encounters*

Electronic Medical Record Outcomes

There was minimal change in prescription of teratogenic medications during the study period and no significant difference between CDS groups (Table 2). Prior to CDS implementation (period T0), 24.2% of visits in which a potentially teratogenic medication was prescribed had documented provision of family planning services. Following CDS implementation (period T1), the proportion of visits with documented provision of family planning services when a potentially teratogenic medication was prescribed increased to 26.5%, an increase of +1.0 adjusted percentage points (95% CI:-0.2 to 2.1, p = 0.08). This slight increase was observed in both CDS groups [(simple: +1.1 adjusted percentage points (95% CI:-0.8 to 3.0) vs. multifaceted: +0.9 adjusted percentage points (95% CI:-0.6 to 2.4)], but the difference in change between the groups was not significant (Table 2). After the multifaceted CDS was deactivated (period T2), improvement in the group formerly receiving the multifaceted CDS slowed, while further improvement was seen among PCPs continuing to receive the simple CDS (Fig. 2); however, there was not a significant difference between the groups in adjusted models (Table 2). PCPs receiving simple CDS displayed a significant upward linear trend over the 3 time points in provision of family planning services when a potentially teratogenic medication was prescribed (adjusted p = 0.03). In contrast, provision of family planning services during visits that did not include the prescription of a potentially teratogenic medication stayed relatively flat (Fig. 2), representing a significantly greater improvement in provision of family planning services over time among visits receiving simple CDS in response to a potentially teratogenic prescription compared to visits without teratogenic prescriptions (adjusted p = 0.008). The ICC for provision of family planning for the patient clusters was 0.84 (95% CI: 0.81-0.88), indicating that outcomes within patients were highly correlated; 26% percent of women had evidence of family planning services at all clinic visits, while 69% had no evidence of family planning services at any visit.

Table 2
Change in Study Outcomes by Intervention Group Following Implementation of Clinical Decision Support*
Figure 2
Proportion of encounters with documented provision of family planning services at three study time points for three types of visits. ‘Encounter’ = visit made to a study PCP by a woman aged 18–50 years with no evidence of sterilization, ...

Physician Survey Outcomes

Seventy-six percent (n = 31) of participating PCPs completed both pre- and post-intervention surveys. There were no significant baseline differences by CDS group. Following CDS implementation, PCPs reported significant improvement in several clinical practices (Table 3). When comparing improvement by group (Table 4), PCPs who received the multifaceted CDS reported a greater increase in the number of times per month they discussed the risk of medication use during pregnancy with their patients than PCPs who received the simple CDS (+4.9 + 7.0 multifaceted vs. +0.8 + 3.2 simple, p = 0.03). Although no other comparisons were significant, PCPs receiving multifaceted CDS reported larger improvement in most practice patterns. However, PCPs receiving simple CDS reported greater satisfaction with their CDS [median(inter-quartile range): 8(3.5) simple vs. 5(3) multifaceted, on a 10-point scale, p = 0.006).

Table 3
Changes in Physician Self-Report of Counseling, Referral, and Prescribing Behaviors*
Table 4
Change in Physician Self-Report of Counseling, Referral and Prescribing Behaviors by CDS Group

Secondary Analyses

PCPs receiving multifaceted CDS accessed the linked order set provided to them only 16% of the time. Of concern, 13% (195/1,548) of the time physicians received CDS, they cancelled the prescription that triggered the CDS and prescribed another potentially teratogenic medication which would have also triggered CDS, if the system had been designed to repeatedly alert clinicians when a teratogen was prescribed.

CDS had similar effects in both the academic and community-based practice: [(academic: +2.6 adjusted percentage points (95% CI:-1.5 to 6.6) vs. community-based: +0.4 adjusted percentage points (95% CI:-0.3 to 1.2)]. The upward trend in provision of family planning services reflects a slight increase in prescriptions for hormonal contraception (OR: 1.8, 95% CI: 0.97-3.39), rather than an increase in pregnancy testing, documented counseling, orders for contraceptive devices or referrals, in both the community-based and academic practice. The clinical indication for which a potentially teratogenic medication was prescribed did not affect the frequency with which family planning services were documented, with two exceptions: women receiving isotretinoin (n = 13, 0.3% of visits with teratogenic prescriptions) were most likely to have documented family planning services (92.3% of such encounters), while women receiving warfarin (n = 52, 1% of visits with teratogenic prescriptions) were least likely (11.5% of such encounters).

DISCUSSION

This cluster-randomized trial found that CDS holds promise for promoting safe prescribing to women of reproductive-age. PCPs receiving either type of CDS reported an increase in the number of times they provided counseling about the risks of medication-induced birth defects to women prescribed potentially teratogenic medications. EMR data corroborated a slight increase in provision of contraceptive prescriptions when such medications were prescribed. These CDS systems had similar effects in academic and community-based settings, but this study was not powered to detect differences between settings. The lack of significant differences between the CDS types is not surprising, as PCPs infrequently accessed the supplemental links provided by the multifaceted CDS. However, if the multifaceted CDS had been used for a longer time period, PCPs may have learned to use the additional tools provided, and we might have seen a further increase in provision of family planning services. The lack of difference between PCPs receiving simple CDS and those no longer receiving CDS is disappointing; the possibility of the deactivated multifaceted CDS having residual effects on PCPs’ prescribing behavior must be considered.

As the effect of these interventions was less than we had hoped, further refinement of the CDS is needed to ensure safe prescribing. In particular, in order to avoid having physicians inadvertently replace one potentially teratogenic medication with another, the CDS should alert PCPs to medication risk as many times as needed during a given encounter. In addition, such CDS may be needed by physicians caring for minors. Our finding that, even with the introduction of this CDS, many women were prescribed potentially teratogenic medications without documented receipt of family planning services confirms prior work in this field.1,2,17,18 Time limitations and lack of clinician reimbursement have been cited as barriers to provision of teratogenic risk counseling.11 Prior work has shown that PCPs provide contraceptive counseling relatively rarely,19 and many could benefit from additional training in contraceptive counseling.1921 Thus, it may be helpful to incorporate information about the safety of different contraceptive options into future CDS, particularly for women receiving warfarin who were least likely to receive family planning services, likely due to limited PCP awareness of contraceptives that do not increase risk of thrombosis. Alternatively, systems that facilitate referral of women who need contraception to a local gynecologist may be of value.22 The fact that women receiving isotretinoin (i.e. Accutane) were most likely to have evidence of family planning services is expected given that the iPledgeTM program requires documented contraception before prescribing.

Strengths of this study include the use of EMR data to corroborate PCP reports of prescribing practices; we were able to identify every single visit with a potentially teratogenic prescription, and it allowed us to adjust for several visit level covariates, as well as pre-intervention levels. The development of this CDS within EpicCare®, a widely-used ambulatory care EMR increases the external validity of the findings. The physician survey data provided additional information on discussions regarding medication risk, an outcome not sufficiently captured in the EMR data.

However, this study has several limitations. These findings may not be generalizable, as study physicians were well-versed in the use of EMRs and CDS. We could not corroborate provision of counseling regarding medication risk because ICD-9 codes for counseling are not used regularly. It is likely that counseling was provided more frequently than is documented in the EMR;23 we did not review physician notes, which may have informally documented provision of counseling. There was also no way to tell which women were currently pregnant or attempting pregnancy; prior studies in this patient population have shown that the large majority (74%) of fertile patients seen in primary care settings are trying to avoid pregnancy.24 The EMR does not document use of non-prescription contraceptives such as condoms, a partner’s vasectomy, or contraceptive services obtained elsewhere. Nor does it reliably document women’s sexual orientation or fertility. Thus, some women we studied were not at risk for medication-induced birth defects. However, lack of documented counseling or family planning services leaves PCPs at risk of lawsuits should birth defects occur. Programming this CDS to fire only once during an encounter, although designed to reduce alert fatigue, may have led us to underestimate the true potential of this CDS, as doctors ultimately substituted another potential teratogen 13% of the time they received a CDS alert. As some PCPs did not complete the follow-up survey, selection bias may make analyses of the survey data less generalizable than those of the EMR data. Physician survey data are also susceptible to both recall and social acceptability bias. Finally, because this study did not have a true control group, we cannot conclude that the changes observed are due solely to the introduction of CDS. Given the small effect seen in both groups, it is possible that these findings simply reflect increased awareness of reproductive health unrelated to these interventions or improved EMR use rather than improvements in delivery of family planning services. However, both study practices had used the EMR and similar CDS for many years

In conclusion, this cluster randomized trial found that CDS may be useful in promoting safe prescribing to fertile women, but further refinement is needed to help PCPs consistently inform their patients of teratogenic risks and contraceptive options in a cost-effective fashion that allows for meaningful use of the EMR.

Electronic Supplementary Material

ESM 1(172K, doc)

(DOC 171 kb)

ACKNOWLEDGEMENTS

This investigator-initiated study was funded by the Agency for Healthcare Research and Quality (AHRQ R18HS017093). Further support was received from NICHD K23 funds (Dr. Schwarz) and grants from the NIH and The Roadmap/NCRR/University of Pittsburgh Multidisciplinary Clinical Research Career Development Award (Dr. Handler). Many thanks to Wishwa Kapoor, MD, MPH for help with project implementation and Doug Landsittel, PhD for statistical guidance. An abstract on the physician survey data was presented at the AHRQ Health IT grantee meeting, Washington DC, June 2, 2010. An abstract on the EMR data was presented at the 1st European Congress on Preconception Care and Health, Brussels, Belgium, October 8, 2010. This work was also presented at the Epic Users’ Group Meeting, Madison, Wisconsin, September 21, 2011.

Conflict of Interest

The authors declare that they do not have a conflict of interest.

Footnotes

Trial Registration

ClinicalTrials.gov Identifier: NCT00766207

References

1. Schwarz EB, Maselli J, Norton M, Gonzales R. Prescription of teratogenic medications in United States ambulatory practices. Am J Med. 2005;118(11):1240–9. doi: 10.1016/j.amjmed.2005.02.029. [PubMed] [Cross Ref]
2. Schwarz EB, Postlethwaite DA, Hung YY, Armstrong MA. Documentation of contraception and pregnancy when prescribing potentially teratogenic medications for reproductive-age women. Ann Intern Med. 2007;147(6):370–6. [PMC free article] [PubMed]
3. Santucci AK, Gold MA, Akers AY, Borrero S, Schwarz EB. Women's perspectives on counseling about risks for medication-induced birth defects. Birth Defects Res A Clin Mol Teratol. 2010;88(1):64–9. [PMC free article] [PubMed]
4. Nordeng H, Koren G, Einarson A. Pregnant women's beliefs about medications–a study among 866 Norwegian women. Ann Pharmacother. 2010;44(9):1478–84. doi: 10.1345/aph.1P231. [PubMed] [Cross Ref]
5. Andrade SE, Gurwitz JH, Davis RL, et al. Prescription drug use in pregnancy. Am J Obstet Gynecol. 2004;191(2):398–407. doi: 10.1016/j.ajog.2004.04.025. [PubMed] [Cross Ref]
6. Lee E, Maneno MK, Smith L, et al. National patterns of medication use during pregnancy. Pharmacoepidemiol Drug Saf. 2006;15(8):537–45. doi: 10.1002/pds.1241. [PubMed] [Cross Ref]
7. Sanz E, Gomez-Lopez T, Martinez-Quintas MJ. Perception of teratogenic risk of common medicines. Eur J Obstet Gynecol Reprod Biol. 2001;95(1):127–31. doi: 10.1016/S0301-2115(00)00375-4. [PubMed] [Cross Ref]
8. Pole M, Einarson A, Pairaudeau N, Einarson T, Koren G. Drug labeling and risk perceptions of teratogenicity: A survey of pregnant Canadian women and their health professionals. J Clin Pharmacol. 2000;40(6):573–7. doi: 10.1177/00912700022009341. [PubMed] [Cross Ref]
9. Bankole A, Singh S, Haas T.Reasons why women have induced abortions: Evidence from 27 countries. International Fam Plann Perspect. 1998;24(3):117-27, 152.
10. Akers AY, Gold MA, Borrero S, Santucci A, Schwarz EB. Providers' perspectives on challenges to contraceptive counseling in primary care settings. J Womens Health (Larchmt) 2010;19(6):1163–70. doi: 10.1089/jwh.2009.1735. [PMC free article] [PubMed] [Cross Ref]
11. Schwarz EB, Santucci A, Borrero S, Akers AY, Nikolajski C, Gold MA. Perspectives of primary care clinicians on teratogenic risk counseling. Birth Defects Res A Clin Mol Teratol. 2009;85(10):858–63. doi: 10.1002/bdra.20599. [PMC free article] [PubMed] [Cross Ref]
12. Schedlbauer A, Prasad V, Mulvaney C, et al. What evidence supports the use of computerized alerts and prompts to improve clinicians' prescribing behavior? J Am Med Inform Assoc. 2009;16(4):531–8. doi: 10.1197/jamia.M2910. [PMC free article] [PubMed] [Cross Ref]
13. Wipfli R, Lovis C. Alerts in clinical information systems: Building frameworks and prototypes. Stud Health Technol Inform. 2010;155:163–9. [PubMed]
14. Raebel MA, Carroll NM, Kelleher JA, Chester EA, Berga S, Magid DJ. Randomized trial to improve prescribing safety during pregnancy. J Am Med Inform Assoc. 2007;14(4):440–50. doi: 10.1197/jamia.M2412. [PMC free article] [PubMed] [Cross Ref]
15. Murray DM. Design and analysis of group-randomized trials. New York: Oxford University Press; 1998.
16. Raudenbush SW, Bryk AS. Hierarchal linear models: Applications and data analysis methods. Thousand Oaks, CA: Sage Publications; 2002.
17. Schwarz EB, Longo LS, Zhao X, Stone RA, Cunningham F, Good CB. Provision of potentially teratogenic medications to female veterans of childbearing age. Med Care. 2010;48(9):834–42. doi: 10.1097/MLR.0b013e3181e57946. [PubMed] [Cross Ref]
18. Hillemeier MM, Weisman CS, Chase GA, Dyer AM, Shaffer ML. Women's preconceptional health and use of health services: Implications for preconception care. Health Serv Res. 2008;43(1 Pt 1):54–75. [PMC free article] [PubMed]
19. Schreiber CA, Harwood BJ, Switzer GE, Creinin MD, Reeves MF, Ness RB. Training and attitudes about contraceptive management across primary care specialties: A survey of graduating residents. Contraception. 2006;73(6):618–22. doi: 10.1016/j.contraception.2006.01.014. [PubMed] [Cross Ref]
20. Spencer AL, Kern LM. Primary care program directors' perceptions of women's health education: A gap in graduate medical education persists. J Womens Health (Larchmt) 2008;17(4):549–56. doi: 10.1089/jwh.2007.0473. [PubMed] [Cross Ref]
21. Parisi SM, Zikovich S, Chuang CH, Sobota M, Nothnagle M, Schwarz EB. Primary care physicians’ perceptions of rates of unintended pregnancy. Contraception. Epub 2011 Dec 14 [doi: 10.1016/j.contraception.2011.11.004]. [PMC free article] [PubMed]
22. Henderson JT, Weisman CS, Grason H. Are two doctors better than one? Women's physician use and appropriate care. Womens Health Issues. 2002;12(3):138–49. doi: 10.1016/S1049-3867(02)00134-2. [PubMed] [Cross Ref]
23. Gilchrist VJ, Stange KC, Flocke SA, McCord G, Bourget CC. A comparison of the National Ambulatory Medical Care Survey (NAMCS) measurement approach with direct observation of outpatient visits. Med Care. 2004;42:276–80. doi: 10.1097/01.mlr.0000114916.95639.af. [PubMed] [Cross Ref]
24. Lee JK, Parisi SM, Akers AY, Borrero S, Schwarz EB. The impact of contraceptive counseling in primary care on contraceptive use. J Gen Intern Med. 2011;26(7):731–6. doi: 10.1007/s11606-011-1647-3. [PMC free article] [PubMed] [Cross Ref]

Articles from Journal of General Internal Medicine are provided here courtesy of Society of General Internal Medicine