PILOT STUDY DEMONSTRATING EFFECTIVENESS OF TARGETED EDUCATION TO IMPROVE INFORMED CONSENT UNDERSTANDING IN AIDS CLINICAL TRIALS

doi:10.1080/09540121.2011.565031

AIDS Care. Author manuscript; available in PMC 2012 November 1.

Published in final edited form as:

AIDS Care. 2011 November; 23(11): 1382–1391.

Published online 2011 June 14. doi: 10.1080/09540121.2011.565031

PMCID: PMC3205427

NIHMSID: NIHMS290628

PILOT STUDY DEMONSTRATING EFFECTIVENESS OF TARGETED EDUCATION TO IMPROVE INFORMED CONSENT UNDERSTANDING IN AIDS CLINICAL TRIALS

Sohini Sengupta,¹ Bernard Lo,² Ronald P. Strauss,³ Joseph Eron,⁴ and Allen L. Gifford⁵

¹Center for Faculty Excellence, The University of North Carolina at Chapel Hill (UNC-CH), Chapel Hill, NC, USA

²Department of Medicine, University of California, San Francisco (UCSF), San Francisco, CA, USA

³Executive Associate Provost, and UNC Schools of Dentistry and Medicine, UNC-CH, Chapel Hill, NC, USA

⁴Department of Medicine, Division of Infectious Diseases, and Center for AIDS Research (CFAR), UNC-CH, Chapel Hill, NC, USA

⁵VA New England Healthcare System, Center for Healthcare Quality, Outcomes and Economic Research, Bedford, MA, USA

⁵Department of Health Policy and Management and Department of Medicine, Boston University, Boston, MA, USA

Corresponding author and to whom request for reprints should be made: Sohini Sengupta, PhD, MPH, UNC-CH Center for Faculty Excellence, Campus Box 3470, Chapel Hill, NC 27599-3470 USA, Tel: 919-966-1741; Fax: 919-962-5236

The publisher's final edited version of this article is available at AIDS Care

Abstract

Assessing and improving informed consent understanding is equally important as obtaining consent from participants in clinical trial research, but developing interventions to target gaps in participants’ informed consent understanding remains a challenge. We used a randomized controlled study design to pilot test an educational intervention to improve actual informed consent understanding of new enrollees in the Adult AIDS Clinical Trial Group (AACTG). Questionnaires were administered to 24 enrollees to assess their baseline understanding on eight elements of informed consent associated with AIDS clinical trials. Enrollees who scored 18/21(85%) or less were randomly assigned to in-person, targeted education (intervention) or delayed education (control). Two follow-up assessments were administered. Repeated measures ANOVA was performed to determine intervention effectiveness in improving actual informed consent understanding over time. Actual understanding improved at the immediate post-intervention time point with a significant score difference of 2.5 when comparing the intervention and delayed groups. In addition, there was a significant score difference of 3.2 when comparing baseline to 3-month follow-up for the two groups, suggesting a statistically significant intervention effect to improve actual understanding of the basic elements of informed consent. The findings demonstrated that one-time targeted education can improve actual informed consent understanding one week after the intervention, but retention of these concepts may require periodic monitoring to ensure comprehension throughout the course of a clinical trial.

Keywords: HIV, AIDS, Informed Consent, Ethics Research, Clinical Trial, Intervention Studies

INTRODUCTION

In clinical trials of therapeutic medications and procedures, major codes of ethical research practice state that it is mandatory for human subjects, or their representatives, to provide written informed consent (Beachamp & Childress, 1994; Carlson, Boyd, & Webb, 2004; Council for International Organization of Medical Sciences [CIOMS], 1993; Levine 1988; National Institutes of Health [NIH], 1979). According to U.S. federal regulations, a valid informed consent document requires eight basic elements about the research to be disclosed to human subjects: (1) Explanation of the research and subjects’ expected participation; (2) description of foreseeable risks to subjects; (3) description of benefits to subjects; (4) disclosure of alternative courses of treatment; (5) description of how subjects’ confidentiality will be protected; (6) for research involving more than minimal risk, an explanation of any compensation with respect to injury; (7) explanation of whom to contact about the research and research subjects' rights; and (8) a statement that participation is voluntary (U.S. Department of Health and Human Services [DHHS], 2009). Ethics committees or Institutional Review Boards (IRBs) are required to review and approve the process of obtaining informed consent and ensure that the eight basic elements are disclosed in any given consent form (DHHS, 2009; Jefford & Moore, 2008). What is less enforced, however, is the extent to which human subjects actually understand the informed consent elements even after signing the consent form (Flory & Emanuel, 2004).

A 2004 literature review examined studies that assessed clinical trial subjects’ understanding of the informed consent form through interventions that aimed to improve their perceived and/or actual understanding (Flory & Emanuel, 2004). The review suggested “limited evidence” that in-person, extended discussion interactions as part of the consent process were more effective in improving informed consent understanding, compared to enhanced consent form strategies (e.g., language simplifying or shortening the consent form) or video/multi-media interventions. Furthermore, while test/feedback interventions demonstrated effectiveness in improving knowledge scores, the authors criticized them for focusing more on rote memorization concerning details of a specific clinical trial, rather than trying to improve subjects’ understanding of the eight basic elements of informed consent.

Since 2004, there have been several intervention studies to increase informed consent understanding in clinical trials, but the majority has assessed specifics of a clinical trial rather than on the eight basic elements of informed consent (Campbell, Goldman, Boccia, & Skinner, 2004; Hutchison, Cowan, & Paul, 2007; Moseley, Wiggins, & O’Sullivan, 2006; Olver, Whitford, Denson, Peterson, & Olver, 2009; Sarkar et al., 2010; Wray, Stryker, Winer, Demetri, & Emmons, 2007). While we do not view HIV/AIDS clinical trials as unique in their informed consent process, we do know that there currently are over 1000 HIV/AIDS clinical trial protocols, with numerous sites per protocol, actively, or soon to be recruiting potential subjects both nationally and internationally (NIH, 2010). Aside from one intervention study to test a video’s effectiveness in improving informed consent understanding among HIV-uninfected Haitian clinical trial enrollees (Joseph et al., 2006), no other studies, to our knowledge, have tested interventions to improve actual informed consent understanding among HIV-infected AIDS clinical trial enrollees.

The purpose of this pilot study was to gather preliminary data to determine the effectiveness of targeted, in-person education to improve informed consent understanding among newly enrolled HIV-infected subjects in a U.S. Adult AIDS Clinical Trial (AACTG) site. The AACTG conducts clinical trials of therapeutic drugs and strategies to treat HIV and HIV-related infections (see https://actgnetwork.org/mission for AACTG’s complete mission). As Flory & Emanuel (2004) recommended, we designed a study to test our educational intervention, and focused on increasing actual understanding of the eight basic elements of informed consent, regardless of the specific clinical trial protocol in which subjects had enrolled.

METHODS

Study Design

The pilot study was a randomized controlled study design, with a delayed intervention control group, and two follow-up time points (immediate post-intervention and at 3-months). Figure 1 illustrates the study design and data collection time points. A delayed intervention was appropriate to uphold clinical equipoise since current informed consent procedures implemented by the AACTG vary by site. Furthermore, while sites may include an assessment of understanding, their procedures may or may not address all eight basic elements of informed consent. This study was approved by the University of North Carolina at Chapel Hill (UNC-CH) Biomedical Institutional Review Board and the UNC-CH General Clinical Research Center.

Figure 1

Diagram of Study Design and Data Collection Time Points

Sample and Recruitment

The target population was newly enrolled HIV-infected subjects in any one of the protocols conducted at the UNC-CH AACTG site in 2007–2008. “Newly enrolled” meant that subjects were recruited into our study within the first month (Week 0 to Week 4) of their recruitment into an AACTG protocol (study durations of protocols were between 2–4 years). It is important to note that UNC-CH AACTG site’s informed consent procedures did not include their own assessment of understanding before enrolling subjects. Furthermore, our pilot study was independent of this site’s informed consent procedures. Study coordinators responsible for particular AACTG protocols were, however, used as liaisons to access newly enrolled subjects for our study. For the subjects who were interested in participating in this study, written informed consent was obtained, followed by the baseline questionnaire assessing perceived and actual understanding of the eight basic elements of informed consent. Each subject received a $10 gift card for completing the baseline assessment.

Randomization

Computerized random assignment was done by a biostatistician, and concealed, numbered envelopes with the random assignment groups (intervention or delayed intervention) were given to the research team. To determine subjects’ assignments, the envelopes were consecutively opened for each subject who incorrectly answered 4 or more items on the 21-item instrument measuring actual understanding at baseline. Neither the research team nor the subjects could be blinded to assignment.

Actual and Perceived Understanding Assessment

The Quality of Informed Consent (QuIC) is a measure to assess both perceived (12 items) and actual understanding (20 items) of the eight basic elements of informed consent (Joffe, Cook, Cleary, Clark, & Weeks, 2001). This measure was originally developed to assess perceived and actual informed consent understanding in therapeutic cancer clinical trials, and reported a mean score of 78% (standard deviation of 9.4) on the actual understanding items. The QuIC was adapted for this study, which involved modifying the original QuIC’s items to focus on HIV/AIDS clinical trials, reviewing this new set of items with PIs and study coordinators at the local AACTG, and pretesting a penultimate version of the instrument with the local AACTG’s Community Advisory Board. After further modification or deletion of items, the final version for HIV/AIDS clinical trials was comprised of 10 items measuring perceived understanding, and 21 items measuring actual understanding. Response options for the adapted version remained the same as in the original QuIC: perceived understanding items ranged from 5 (“I understood this very well”), to 1 (“I didn’t understand this at all”); and actual understanding items ranged from “Agree,” “Unsure,” or “Disagree.” We re-coded actual understanding items by giving a 1 for correctly answered items (responses could be “Agree” or “Disagree), and 0 for incorrectly answered items (responses could be “Agree” or “Disagree,” and “Unsure”). We added the number of correct responses to obtain the summary score, with 21 being the highest score to represent actual understanding of the basic elements of informed consent. Table 1 presents the items asked for perceived and actual understanding, and to which basic elements of informed consent they underlie.

Table 1

Adapted QuIC Items Corresponding to the Eight Basic Elements of Informed Consent

Both the perceived and actual understanding items were administered at baseline and immediate post-intervention. In addition, the perceived and actual understanding items were administered at a 3-month follow-up time point to determine retention of actual informed consent understanding after both groups received the targeted educational intervention. Baseline assessments were 15–20 minutes in length, and included socio-demographic questions and questions asking about motivations to participate. Immediate post-intervention and 3-month follow-up assessments were 10–15 minutes in length, and conducted over the telephone.

Intervention

Both intervention and delayed intervention groups received 20 minutes of targeted education to improve their actual understanding of the eight basic elements of informed consent. We tried to model the targeted educational intervention after what the AACTG study coordinators typically do when obtaining consent from potential study participants before they enroll. This included subjects being provided a copy of the consent form, and being able to ask any questions of the AACTG protocol in which they were participating. The main difference, however, was that, since our intervention took place soon after a participant enrolled in a given AACTG protocol, the consent form we provided to them served as a guide from which we could review with subjects all incorrectly answered items from the baseline assessment, thus making it targeted education. For example, if a subject incorrectly disagreed with “If I am injured or become ill as a result of participation in this clinical trial, costs of any medical care related to the injury will be billed to me and/or my insurance company,” the intervention would involve showing the subject where in the AACTG consent form that statement was written, and what the statement meant in practical terms.

Data Analysis

Descriptive statistics were analyzed using PASW, version 18.0, and testing of the effectiveness of the intervention occurred in SAS, version 9.2. Demographic characteristics and motivations to participate were compared to make sure groups were similar on demographic and other relevant variables; any of these variables that were not equally distributed would be treated as confounders in the statistically analysis. We also calculated Pearson correlation coefficients to see how well subjects’ actual understanding correlated with their perceived understanding on the eight basic elements of informed consent at baseline.

Next, we were interested in testing the effectiveness of the targeted educational intervention to improve actual understanding of basic elements of informed consent. A biostatistician, blinded to assignment, used a repeated measures ANOVA with appropriate follow-up contrasts to compare the actual understanding summary scores for the intervention and delayed intervention groups at baseline, immediate post-intervention, and at the 3-month follow-up, using an alpha level of .05. Estimates of the score difference for each of the contrasts were reported, along with their associated p-values and 95% confidence intervals.

RESULTS

A total of 24 subjects who were asked to participate agreed to be in the pilot study. Only three of the 24 subjects were not eligible for random assignment since their scores ranged from 19/21 to a perfect score (21/21) at their baseline assessment. The remaining 21 subjects were randomly assigned to the intervention (n = 12) and delayed intervention (n = 9) groups. Thus, we did not meet our sample size requirement of 15 per group to estimate a 2-point (or 10%) actual understanding difference between groups at the immediate post-intervention. Gift card incentives were provided for each follow-up time point and for the educational intervention in which subjects participated.

Baseline Comparisons

Table 2 presents the descriptive statistics that demonstrate how the intervention and delayed intervention groups were similar. This provided us some indication that the random assignment worked, and that there were no socio-demographic or motivations to participate variables we needed to treat as confounders. P-values are not presented in Table 2 because of the small sample sizes of the descriptive variables’ sub-categories. Furthermore, the correlation between perceived and actual understanding scores was moderate to strongly positive (.59) at p < .002 (not shown in Table 2).

Table 2

Sociodemographics and Motivations to Participate in Clinical Trials Comparing Intervention and Delayed Intervention Groups

Testing the Effectiveness of the Educational Intervention

Of the 21 subjects who were randomized to the intervention or delayed intervention, 1 (5%) subject assigned to the delayed intervention group became too ill to complete the pilot study after completing the baseline assessment. One other subject (5%) who was assigned to the intervention group was lost to follow-up at the 3-month assessment. For this subject, we used maximum likelihood estimation to impute a single value to replace this subject’s 3-month actual understanding score (Yuan, accessed 2010). Thus the sample sizes for the intervention and delayed intervention groups to test the intervention were n = 12 and n = 8, respectively.

Actual understanding of the basic elements of informed consent was measured by 21 informed consent knowledge items at baseline, immediate post-intervention, and at a follow-up three months after the baseline assessment. To test our hypothesis that actual understanding scores will improve more in the intervention group versus the delayed intervention group, we compared the mean scores at each time point for both groups. Table 3 presents the repeated measures ANOVA estimates of score difference and their 95% confidence intervals (CIs) (null value = 0), indicating how large the mean score differences were at baseline, immediate post-intervention, and at the 3-month follow-up.

Table 3

Repeated Measures ANOVA with Contrasts

The estimate of −0.75, 95%CI(−2.5, .95) at baseline indicated no significant differences in actual understanding of informed consent principles when comparing the intervention and delayed intervention groups. At the immediate post-intervention time point, however, the estimate increased to 2.5, 95%CI (0.76, 4.2), indicating a significant intervention effect on actual understanding of the basic elements of informed consent at the immediate post-intervention time point. At the 3-month follow-up—a time point at which both groups had received the intervention (see Figure 1)—the estimate decreased to −.51, 95%CI (−2.2, 1.2), indicating that the improvement in actual understanding waned to less than baseline levels approximately five weeks after both groups received the intervention. When we examined the time effect on these groups, the baseline to follow-up difference estimate was 3.2, 95%CI (1.2, 5.2), indicating an intervention effect and better overall retention (i.e., actual understanding) of the basic elements of informed consent in the intervention group than in the delayed group by three points.

To examine which of the 21 items demonstrated the greatest improvement in actual understanding of the basic elements of informed consent as a result of the targeted educational intervention, we compared the number of correct responses for each item at baseline and immediate post-intervention for the intervention and delayed intervention groups (data not shown). At baseline, 10 of 21 items (#s 3, 4, 6, 9, 11, 12, 16–18, 20 from Table 1) were incorrectly answered by at least 25% of the participants in the intervention group. These 10 items reflected the following basic elements of informed consent: purpose and subjects expected participation (item #s 3, 4, 6, and 9); risks (item #s 11–12); alternatives to participation (item #16); confidentiality of data (item #17); injuries associated with greater than minimal risk research (item #18); and voluntariness of consent to research (item #20). At the immediate post-intervention time point, more participants in the intervention group correctly responded to all 10 of these items, but only #s 12 and 18 demonstrated statistically significant improvement with p-values < .05.

DISCUSSION

Since the 2004 literature review assessing clinical trial subjects’ understanding of informed consent (Flory & Emanuel, 2004), little has been published to rigorously test interventions to improve actual understanding of the eight basic elements of informed consent stated in the U.S. federal regulations (DHHS, 2009). While the AACTG and other HIV clinical trial networks do incorporate assessments of understanding in their informed consent procedures, this pilot study is a first step in helping to improve actual informed consent understanding of the eight basic elements of informed consent among AACTG enrollees. Several contributions can be noted.

First, the study developed and evaluated an adapted version of the QuIC instrument (Joffe et al., 2001) to assess the eight basic elements of informed consent, rather than developing an original instrument that assesses rote memorization of specifics on any given AACTG protocol. This not only facilitates cross-trial site comparisons within the AACTG, but also can help AACTG researchers within a trial site identify aspects of the informed consent that their enrollees are not understanding, and educate enrollees on those particular elements. For example, our findings demonstrated that items measuring purpose and subjects expected participation, risks, alternatives to participation, confidentiality of data, injuries associated with greater than minimal risk research, and voluntariness of consent to research may not have been discussed with, or were confusing to AACTG enrollees at the time of the baseline assessment. With targeted education, the intervention group improved their understanding of these elements of informed consent. Two of the items measured—The treatment being researched in my clinical trial has been proven to be the best treatment, and Compared with standard treatments, my clinical trial does not carry any additional risks or discomforts—are related to the therapeutic misconception often associated in clinical trials where subjects misunderstand the clinical trial as being therapeutic (Miller, 2008). Indeed, these two items also were incorrectly answered in the original QuIC study (Joffe et al., 2001), demonstrating that these items may be difficult to understand across clinical trials regardless of the disease.

Lastly, the pilot study used a randomized controlled study design to test a targeted, educational intervention to improve actual understanding of the basic elements of informed consent. This contribution responds to Flory’s and Emanuel’s (2004) recommendations to employ rigorous methods to test an intervention, and to focus on interventions that emphasize in-person discussions since other informed consent interventions (e.g., focus on simplifying or shortening the consent form, and/or video/multi-media interventions) did not demonstrate effectiveness to improve informed consent understanding (actual or perceived). This study demonstrated a statistically significant intervention effect, with a 3-point difference for follow-up versus baseline of the intervention and control groups. The targeted educational intervention also has the advantage of being brief, and along with administering the adapted QuIC instrument, could be feasible to do as part of any informed consent process undertaken by an AACTG trial site. This would be our recommendation to AACTG sites that would be interested in implementing initial informed consent education and assessment to their recruitment process, and/or periodic monitoring to ensure comprehension throughout the course of a clinical trial.

This pilot study does have some limitations that should be noted. The primary limitations have to do with sample size and recruitment. We did not meet our intended sample size goal, mainly because our subjects represented only 3 of 5 AACTG protocols from which we were allowed to recruit, and at the time of our study, these protocols’ recruitment efforts were either winding down, or these protocols were recruiting 1–2 individuals per month. Second, we also were concerned with the possibility of selection bias, given that we were dependent on the AACTG site’s study coordinators letting us know which subjects we could ask to participate in our study. We expect this is the reason why all of the subjects who we were allowed to approach agreed to participate in our pilot study. To minimize selection bias, the ideal recruitment scenario for our study would have been that we could ask all newly enrolled AACTG subjects to participate in our study to assess both refusal and participation rates, and to possibly collect demographics data for the refusals. Despite not being able to meet our intended sample size, and problems with our recruitment process, we were able to minimize selection bias with very low loss to follow-up, as well as demonstrate effectiveness of the targeted educational intervention. A third limitation is that our pilot study subjects represented one U.S. AACTG site, HIV-infected adults, and enrollees in therapeutic HIV/AIDS trials, thus decreasing generalizability to pediatric AIDS clinical trials, international AACTG sites and possibly other U.S. AACTG sites, and HIV prevention trials that may include non HIV-infected individuals. One final limitation relates to whether using forced-choice checklists, like our adapted QuIC instrument, really measures actual understanding of informed consent, based on a 2006 published study that compared methods of assessing actual understanding of informed consent in HIV vaccine trials (Lindegger et al., 2006). Based on our findings, we do agree with the authors that forced-choice checklists may overestimate actual understanding, and used alone, will not help study participants understand the basic concepts of informed consent. While our targeted educational intervention did not use open-ended narratives and vignettes, as the Lindegger et al. (2006) study recommended, we do believe that targeted education does allow for open-ended discussion of the basic elements of informed consent, and is a more feasible option for AACTGs to incorporate into their informed consent process since it would not require additional resources to collect and analyze the open-ended (qualitative) data.

In conclusion, the informed consent process in AIDS clinical trials has not always been successful in communicating to new enrollees the eight basic elements of informed consent stated in the U.S. federal regulations. Learning these elements could be useful to enrollees to better understand the clinical trial itself and their expected participation. Monitoring consent in such a manner would ensure not only a signed agreement between researchers and subjects, but also uphold the underlying assumption that subjects understand the content of the consent form throughout the course of an AIDS clinical trial.

ACKNOWLEDGEMENTS

This pilot study was funded by a Career Developmental Award in Research Ethics through the National Institute of Allergy and Infectious Diseases (NIAID), Grant# K01AI055247. The research also was supported by the UNC-CH Center for AIDS Research (CFAR), NIH Grant #P30 AI50410, CTSA (Grant #: UL1RR025747), and UNC-CH’S GCRC (Grant #: M01RR00046). We would like to thank AACTG study personnel for helping us with participant recruitment. We also would like to thank Chris Wiesen of the UNC-CH Odum Institute for Research and Social Sciences for his statistical support.

REFERENCES

Beauchamp TL, Childress JF. Principles of Biomedical Ethics. 4th ed. New York: Oxford University Press; 1994.
Campbell FA, Goldman BD, Boccia ML, Skinner M. The effect of format modifications and reading comprehension on recall of informed consent information by low-income parents: a comparison of print, video, and computer-based presentations. Patient Education and Counseling. 2004;53(2):205–216. [PubMed]
Carlson RV, Boyd KM, Webb DJ. The revision of the Declaration of Helsinki: past, present and future. British Journal of Clinical Pharmacology. 2004;57(6):695–713. [PubMed]
Council for International Organization of Medical Sciences. [accessed February 15, 2011];International Ethical Guidelines for Biomedical Research Involving Human Subjects. 2002 Retrieved from: http://www.cioms.ch/publications/guidelines/guidelines_nov_2002_blurb.htm.
Flory J, Emanuel E. Interventions to improve research participants' understanding in informed consent for research: a systematic review. JAMA. 2004;292(13):1593–1601. [PubMed]
Hutchison C, Cowan C, Paul J. Patient understanding of research: developing and testing of a new questionnaire. European Journal of Cancer Care (Engl) 2007;16(2):187–195. quiz 195-16. [PubMed]
Jefford M, Moore R. Improvement of informed consent and the quality of consent documents. Lancet Oncology. 2008;9(5):485–493. [PubMed]
Joffe S, Cook EF, Cleary PD, Clark JW, Weeks JC. Quality of informed consent in cancer clinical trials: a cross-sectional survey. Lancet. 2001;358(9295):1772–1777. [PubMed]
Joseph P, Schackman BR, Horwitz R, Nerette S, Verdier RI, Dorsainvil D, Fitzgerald DW. The use of an educational video during informed consent in an HIV clinical trial in Haiti. Journal of Acquired Immune Deficiency Syndrome. 2006;42(5):588–591. [PubMed]
Levine RJ. Ethics and Regulation of Clinical Research. New Haven: Yale University Press; 1988.
Lindegger G, Milford C, Slack C, Quayle M, Xaba X, Vardas E. Beyond the checklist: assessing understanding for HIV vaccine trial participation in South Africa. Journal of Acquired Immune Deficiency Syndrome. 2006;43(5):560–566. [PubMed]
Miller FG. Twenty-five years of therapeutic misconception. Hastings Center Report. 2008;38(2):6. [PubMed]
Moseley TH, Wiggins MN, O'Sullivan P. Effects of presentation method on the understanding of informed consent. British Journal of Ophthalmology. 2006;90(8):990–993. [PMC free article] [PubMed]
National Institutes of Health, Office of Human Subjects Research. [accessed on February 15, 2011];Belmont Report. Ethical Principle and Guidelines for the Protection of Human Subjects of Research. 1979 Retrieved from: http://ohsr.od.nih.gov/guidelines/belmont.html.
National Institutes of Health. [accessed on February 15, 2011]; Clinical trials.gov search engine. Available at: www.clinicaltrials.gov (key word = HIV and limit to open trials)
Olver IN, Whitford HS, Denson LA, Peterson MJ, Olver SI. Improving informed consent to chemotherapy: a randomized controlled trial of written information versus an interactive multimedia CD-ROM. Patient Education and Counseling. 2009;74(2):197–204. [PubMed]
Sarkar R, Sowmyanarayanan TV, Samuel P, Singh AS, Bose A, Muliyil J, Kang G. Comparison of group counseling with individual counseling in the comprehension of informed consent: a randomized controlled trial. BMC Med Ethics. 2010;11:8. [PMC free article] [PubMed]
U.S. Department of Health and Human Services. [accessed on February 15, 2011];Title 45 Code of Federal Regulations, Part 46 Protection of Human Subjects (45 CFR 46) 2009 Retrieved from: http://ohsr.od.nih.gov/guidelines/45cfr46.html.
Wray RJ, Stryker JE, Winer E, Demetri G, Emmons KM. Do cancer patients fully understand clinical trial participation? A pilot study to assess informed consent and patient expectations. Cancer Education. 2007;22(1):21–24. [PubMed]
Yuan YC. [accessed on February 15, 2011];Multiple imputation for missing data: Concepts and new development. (no date). Retrieved from: http://faculty.smu.edu/slstokes/stat6380/MI.pdf.