Effect of public deliberation on  attitudes toward surrogate consent for dementia research

doi:10.1212/WNL.0b013e31823648cb

Neurology. 2011 December 13; 77(24): 2097–2104.

PMCID: PMC3235352

Effect of public deliberation on attitudes toward surrogate consent for dementia research

S.Y.H. Kim, MD, PhD, H.M. Kim, ScD, D.S. Knopman, MD, R. De Vries, PhD, L. Damschroder, MS, and P.S. Appelbaum, MD

From the Center for Bioethics and Social Sciences in Medicine (S.Y.H.K., R.D.V.), Department of Psychiatry (S.Y.H.K.), and Center for Statistical Consultation and Research (H.M.K.), University of Michigan, Ann Arbor; Department of Neurology (D.S.K.), Mayo Clinic, Rochester, MN; Health Services Research and Development (L.D.), Ann Arbor VA Medical Center, Ann Arbor, MI; and Division of Law, Ethics, and Psychiatry (P.S.A.), Department of Psychiatry, Columbia University and New York State Psychiatric Institute, New York, NY.

Corresponding author.

Address correspondence and reprint requests to Dr. Scott Kim, Center for Bioethics and Social Sciences in Medicine, 300 North Ingalls St, 7C27, Ann Arbor, MI 48109, E-mail: scottkim/at/med.umich.edu

Study funding: Supported by NIH R01AG029550 and the Greenwall Foundation Faculty Scholars Award in Bioethics.

Received February 10, 2011; Accepted April 7, 2011.

This article has been cited by other articles in PMC.

Abstract

Objective:

To assess the informed, deliberative views of the older general public toward a policy of allowing surrogate consent for Alzheimer disease (AD) research.

Methods:

A total of 503 persons aged 50+ recruited by random digit dialing were randomly assigned to 1 of 3 groups: deliberation, education, or control. The deliberation group attended an all-day education/peer deliberation session; the education group received written information only. Participants were surveyed at baseline, after deliberation session (or equivalent time), and 1 month after the session, regarding their attitudes toward a policy of allowing surrogate consent for research studies of varying risks and potential benefits (a lumbar puncture study, a drug randomized controlled trial, a vaccine randomized controlled trial, and an early phase gene transfer trial).

Results:

At baseline, a policy of surrogate consent for AD research was supported by 55%–91%, depending on the scenario. The education group had a transient increase in support for one research scenario after receiving the information materials. In the deliberation group, support for surrogate consent was higher after deliberation for all scenarios (67% to 97%), with much of the increase sustained 1 month after the deliberation session. No changes occurred in the control group. The study's limitations include self-selection of participants due to the demanding nature of attendance at the deliberation sessions.

Conclusions:

This sample of the older general public generally supported a policy of surrogate consent for AD research at baseline. Their support increased with democratic deliberation involving informed, in-depth exploration of the relevant scientific and ethical issues.

Alzheimer disease (AD) research involves persons with significant cognitive impairment. Although some persons with mild AD may be able to provide consent, the disease leads to early decisional incapacity,^¹,² and surrogate consent for research is usually necessary. The dilemma is intensified when research is invasive and burdensome, with unpredictable risks.^³–⁵ When to allow surrogate consent for research involving decisionally impaired adults has been debated for several decades with little progress in policy.^⁶ The current US regulations allow consent by legally authorized representatives (LAR) (45CFR46, 102c, 111.a4, and 116), but the regulations defer to states to define the LAR; few states have done so^⁷,⁸ and state laws differ on how to balance potential benefits with risks.^⁸

When a bioethical controversy has broad public health implications, the considered ethical opinions of the public may be especially important to informed policy-making.^⁹ Although surveys have shown support for surrogate consent among older Americans^¹⁰,¹¹ and persons at high risk for AD,^¹² the issue has complex scientific, legal, historical, and bioethical dimensions that are unfamiliar to the lay public, raising concerns whether these traditional surveys tap reliable and valid opinions.^¹³

Deliberative democracy, a framework developed in political theory, provides an approach to incorporating public opinion in policy-making.^¹⁴ A variety of deliberative methods have been used to engage the public.^{¹⁵–¹⁷} These methods provide balanced, comprehensive information in sessions involving interactive consultations with experts and deliberations with peers. We report here on how attitudes toward a policy of allowing surrogate consent for dementia research changed in a sample of older persons from the general public following participation in a democratic deliberation (DD) exercise.

METHODS

A detailed account of the theoretical basis and methodologic procedures for this study is available.^¹³

Participants.

Members of the general public (≥50 years old) within a 50-mile radius of Ann Arbor, MI, were recruited through the Survey Research Center of the Institute for Social Research at the University of Michigan via random-digit dialing. This age group was chosen to elicit the views of those more likely to have had experience with dementia, who would find the policy issues more salient, and still capture a broad swath of the public.

Of 2,402 eligible individuals contacted, 700 (29%) agreed to participate in the study; of these, 503 (72%) completed the baseline survey (survey 1) and were randomized into 1 of 3 groups (control group, education group, DD group) at a 3:3:4 ratio (see figure e-1 on the Neurology® Web site at www.neurology.org). This ratio was based on a previous study^¹⁸ and a pilot study that estimated the no-show rate in the group assigned to DD, with the goal of achieving approximately similar numbers of respondents in each group. As described below, the DD group attended an all-day education and deliberation session. The education group received annotated slide presentations of the 2 experts used in the DD session by mail (see appendix e-1). The control group received only surveys.

Of 1,702 eligible individuals who declined to participate in the study, 879 (52%) answered 3 follow-up questions. Compared to these decliners, the 503 individuals randomized into the study were younger (63 vs 66 years old, p < 0.001), and more likely to have had a relationship (spouse, loved one, or friend) with a person with dementia (71% vs 43%, p < 0.001) and to have been a caregiver or decision-maker for a patient with dementia (23% vs 17%, p = 0.004).

Standard protocol approvals, registrations, and patient consent.

This study was deemed exempt from federal regulations by the University of Michigan's Institutional Review Board.

Study materials.

DD session presentations.

We developed 2 45-minute presentations (appendix e-1): “AD Clinical Research” described features of AD, current treatment, types of research on AD and treatment development process, and how subjects are enrolled in research; “Ethical Issues in Surrogate-Based Research” described well-known human subject abuses (including the Nazi experiments, Tuskegee study, and Willowbrook), the resulting regulatory process, the unsettled policy on surrogate consent, and the reasons for and against surrogate consent for dementia research.^¹³ In developing these presentations, the research team worked closely with an advisory panel consisting of a political science expert in deliberative democracy methods, a senior AD researcher, a bioethicist-sociologist, a geriatrician, a director of a human subject protections program at an academic medical center, a qualitative research expert, a gerontologic nurse, and a caregiver of a person with AD. These presentations were further refined, based on a final systematic review by the members of the advisory panel, additional external experts (in both AD research and bioethics), and laypersons, totaling 11 persons. They received a systematic quantitative survey (using a 5-item Likert scale with a range of strongly disagree to strongly agree) regarding accuracy, balance, and how easy the material was to understand, on the basis of which each subsection of the presentations was modified. The reviewers provided open-ended comments as well, to ensure completeness of information and appropriateness of the materials for a general public audience.^¹³

Measures.

This study's survey has been used in previous studies.^¹⁸ It contains an introduction to AD and to the ethical dilemma of involving decisionally impaired subjects in research, and presents 4 research scenarios of approximately 120 words each: a study involving a lumbar puncture to develop a diagnostic test, a randomized clinical trial of a new drug, a randomized clinical trial of an AD vaccine, and an early-phase neurosurgical gene-transfer trial. Respondents are then asked: “If patients cannot make their own decisions about being in studies like this one, should our society allow their families to make the decision in their place?” The 4 response options were as follows: definitely not allow, probably not allow, probably allow, and definitely allow.

We used a 17-item questionnaire (12 multiple-choice and 5 true/false; range of scores 0–17) to test subjects' level of knowledge about AD (prevalence, risk factors, course of disease, treatment options), purpose and types of clinical research in AD, ethical regulation of clinical research, and current policy regarding surrogate consent for clinical research.

Attendees completed an evaluation form at the end of the DD day containing 8 questions (table 4, items 1–8) rated on a 10-point scale (1 = not at all, 10 = very much). An evaluation of the educational materials was completed by the education group (table 4, items 6–9).

Table 4

Evaluation of the democratic deliberation session (items 1–8) and education materials (items 6–9) by the participants in each group

Procedures.

Surveys.

The survey was administered to each subject 3 times. Survey 1 was administered by mail prior to randomization, about 1 month before the DD date. Survey 2 was completed at the end of the DD day (for DD attendees) or around that date (by mail, for all others). Copies of the DD day presentations (slides plus notes in PowerPoint ®) were mailed to the education group with survey 2. Survey 3 was sent by mail approximately 1 month after the DD date.

DD session procedures.

On the day of the DD session, attendees were randomly assigned to tables, in groups of 5–7 persons. A trained facilitator moderated the discussion at each table; their training is described elsewhere.^¹³ The sequence and description of the session day are described in table 1.

Table 1

Sequence and content of the DD session day^a

Analyses.

Subject characteristics across the 3 arms were compared using analysis of variance or χ² tests, depending on the data. A baseline comparison of survey 1 responses across the groups was conducted using the χ² test.

Given the randomized, experimental design, the primary analysis was within-subject changes in responses from surveys 1 to 2 and 1 to 3 using the test of symmetry (which accounts for paired data) in Stata 10.0. All analyses involving the DD group conservatively included both DD attendees and nonattendees.

Although the unadjusted analysis is our primary analysis, it makes use only of data that come in complete pairs. Thus, we conducted a confirmatory analysis using a linear mixed-effects model that made use of all available data, assessing for trends over time in survey responses within group and between groups; this model used responses at surveys 2 and 3 as the dependent variable, subject as random intercepts, and as independent variables, demographic and other subject characteristics (age, gender, race, financial status, educational level, and history of relationships with persons with AD), baseline values of the response variable, DD and education group indicators, survey 3 indicator, survey 3 by DD group interaction term, and survey 3 by education group interaction term. The adjusted between-group difference estimates and the trends over time from this model were very similar to the unadjusted results; thus, only the unadjusted results are presented.

RESULTS

The demographic characteristics of the DD, education, and control groups were similar (table 2). There was no significant difference between groups at baseline in their attitudes toward a societal policy of surrogate consent for any of the 4 research scenarios (table 3; χ² test, p values 0.46–0.73 for the 4 scenarios).

Table 2

Participant characteristics at randomization (n = 503)

Table 3

Effect of democratic deliberation or education on attitudes toward societal policy of allowing surrogate consent for dementia research^a

Attitudes toward surrogate consent.

At baseline, the majority of our sample expressed support for a policy of allowing surrogate consent for dementia research for all 4 scenarios: 84% (53% probably and 31% definitely allow) for the lumbar puncture protocol, 91% (55% probably, 36% definitely) for the drug randomized controlled trial (RCT), 64% (45% probably, 19% definitely) for the vaccine protocol, and 55% (38% probably, 17% definitely) for the gene transfer protocol.

Table 3 summarizes the attitudes toward surrogate consent of the 3 study groups, over time. For the control group, there were no changes in attitudes from baseline. For the education group, there was a significant increase in support for allowing surrogate consent for the vaccine study scenario—60% at baseline, increasing to 73% at survey 2 (p < 0.05). However, this change was not sustained after a month.

For the DD group, there was a significant change in attitude toward a policy of allowing surrogate consent following the DD session that was sustained a month later, for all 4 research scenarios. For the lumbar puncture and the drug RCT protocols, which already had a high proportion of support at baseline, the change was in the strength of support. For the lumbar puncture protocol, those who would definitely allow surrogate consent rose from 33% to 76% after the DD session (p < 0.001); this declined to 51% after a month but remained significantly higher than baseline (p < 0.01). For the drug RCT protocol, the “definitely allow” response changed from 38% to 76% (p < 0.001) after the DD session, declining to 53% a month later (p < 0.01 compared to baseline). For the higher risk research protocols, there was a sustained increase in the overall proportion of those who would allow surrogate consent. The baseline support for surrogate consent for the vaccine protocol of 65% increased to 79% after the DD session (p < 0.001) and remained at 79% even after a month (p < 0.001). For the neurosurgical gene transfer protocol, the initial support of 56% for surrogate consent rose to 68% after the DD session (p < 0.001) and remained at 67% after a month (p < 0.01).

Knowledge of AD-related issues.

On the 17-item knowledge questionnaire, there was no significant difference at baseline between the groups' mean scores (11.4 ± 2.5 for controls, 11.3 ± 2.7 for education group, 11.5 ± 2.6 for DD group, analysis of variance, p = 0.86). The control group mean did not change at later surveys (11.5 ± 2.5 at survey 2, 11.5 ± 2.9 at survey 3). However, the education and DD groups both showed significant increases in mean scores at survey 2 (14.4 ± 2.6 for education group, paired t test, p < 0.001; 14.5 ± 2.3 for DD group, p < 0.001) and at survey 3 (for education group, 13.3 ± 2.9, p < 0.001; for DD group, 14.1 ± 2.6, p < 0.001). The change in the education group's mean scores is consistent with the group's self report that 90% had read “all” or “most” of the education materials (table 4, question 9).

Reactions to DD session.

DD session participants had very positive views of their experience (table 4).

The participants strongly felt their opinions were respected and the process was fair, and they were willing to abide by the policy decision put forward by their small group even if they had a different view. The participants in the DD session were more strongly affected by participating in the DD session than were the members of the education group by reading the educational materials, in terms of understanding, opinion change, and perceived usefulness.

DISCUSSION

Clinical trials are essential to advance therapeutics in AD. Yet the nature of the disease makes patients with AD highly vulnerable research subjects, as their capacity for self-determination is lost early in the illness. This ethical dilemma defies easy solutions, but an acceptable societal policy should incorporate the views of citizens, elicited in a way that provides confidence that those opinions are considered and informed—as we tried to do in this study using democratic deliberation as a vehicle for eliciting such opinions.

There are several notable findings. First, prior to deliberation, the baseline survey revealed that a majority of the respondents supported a policy of surrogate consent for dementia research (combining “probably” and “definitely” allow responses). This majority support is consistent with previous surveys, including a survey of persons at increased risk for AD,^¹² and is consistent with our national survey of older Americans in which the response categories were “yes” or “no.”^¹⁰ Second, the initial level of support for a policy of allowing surrogate consent increased when the respondents were given thorough and balanced information and a chance to deliberate with peers. Given that the information presented included a detailed history of human subject abuses (see appendix e-1), this is a significant finding. This sample of the lay public became significantly more supportive of a societal policy of surrogate consent and increased support was sustained even after 1 month, even for the higher risk research scenarios such as a first-in-human gene transfer trial. This finding is consistent with a previous study of caregivers of patients with AD using similar DD methods.^¹⁸

Third, these changes in attitude appear to be the effect of the deliberative process rather than simply of additional knowledge. Although the education group showed similar shifts in knowledge as the DD group, there was little change in their attitudes regarding surrogate consent.

Several factors support the validity of the impact of DD found in this study, including the experimental design and successful randomization. The quality of deliberative sessions was high, as reflected in participants' self report (table 4). Further, qualitative analyses of the audiotaped DD deliberations from a previous study showed equality of participation, respectful dialogue among subjects, and use of reasoned justifications—all markers of high-quality deliberation.^¹⁹ Moreover, the key materials used in this study—the experts' presentations on AD clinical research and on the ethics of surrogate consent for research—were developed and vetted in collaboration with an interdisciplinary panel of experts and laypersons. Thus, there is good reason to believe that post-DD session opinions were more informed, thoughtful, and considered than opinions solicited via traditional surveys.

The study has several potential limitations. First, because the DD sessions require a considerable time commitment from volunteer participants, there is inevitable self-selection. Based on the 52% of decliners who answered our follow-up questions, the decliners were slightly younger and less likely to know or to have cared for someone with AD. Although this does not affect the internal validity of the DD effect, it is possible that a more representative sample may show more or less change in their attitudes regarding surrogate consent following DD. Second, although there is self-report evidence from this study and published evidence of similar DD sessions that deliberations are of high quality, there may have been undetected group dynamics or subtle influences from the experts during the session that affected the deliberations. Qualitative analyses of the session transcripts are ongoing and will be reported elsewhere. Third, it is possible that some of the immediate effects found in survey 2 may have been due to nonspecific effects of the DD group completing their surveys on site, compared to the other groups completing them by mail. This may account for the decrease in the DD session effect at survey 3, although we note that an increase in support for surrogate consent by the DD group was sustained for all 4 scenarios. Fourth, because the attitudes elicited were based upon information unique to dementia research, findings may not be generalizable to other research, e.g., research involving comatose subjects or incapacitated persons with mental illness.

Finally, we note that the use of DD in bioethics research is still relatively new and further discussions of the method will be needed. For example, the educational materials were developed by an interdisciplinary panel mostly comprising persons not involved in dementia research. However, with the exception of a few laypersons, most were academics, and academics in general are likely to be favorably disposed to research. This could have led to unintended bias in the materials used (appendix e-1).

Our results may have significant policy implications. Enrolling persons in research when they are incapable of providing their own informed consent remains controversial, especially when the research involves significant risks.^⁸ Our study shows not only majority support for a policy of surrogate consent for dementia research at baseline, but as citizens become more informed and deliberate with one another, they become significantly more supportive. This is particularly striking for the higher risk research scenarios (the lower risk scenarios may not have shown dramatic changes due to ceiling effects) in which, for example, the public's support for a policy of surrogate consent for a gene transfer protocol rose from 56% to 68%, with the increase largely retained even after a month. Although there are no simple rules for translating levels of public support into policy, institutional review boards, other research oversight bodies, and future policy-making panels may find it useful to know what happens when citizens spend a day learning about and deliberating the complex issues in ethics of surrogate consent for dementia research.

Supplementary Material

Data Supplement

Click here to view.

GLOSSARY


%AD	Alzheimer disease

DD	democratic deliberation

LAR	legally authorized representative

RCT	randomized controlled trial.

Footnotes

Supplemental data at www.neurology.org

AUTHOR CONTRIBUTIONS

Dr. S. Kim was involved in the conception and design of this study, the acquisition, analysis, and interpretation of the data, and drafted and revised the manuscript. Dr. M. Kim was involved in the design of the study, led the statistical analysis, assisted in data interpretation and presentation, and contributed critical revisions related to analytical content. Dr. Knopman, Dr. De Vries, L. Damschroder, and Dr. Appelbaum were involved in the design of the study and in the acquisition, analysis, and interpretation of the data, and contributed critical revisions of the manuscript for important intellectual content.

DISCLOSURE

Dr. S. Kim serves on the editorial board of the Journal of Empirical Research on Human Research Ethics; has recently served as Associate Editor of BMC Psychiatry and on the editorial board of the American Journal of Bioethics; serves on the NIMH Data Safety and Monitoring Board and the FDA Neurological Devices Panel of the Medical Devices Advisory Committee; receives publishing royalties for Evaluation of Capacity to Consent to Treatment and Research (Oxford University Press, 2009); and receives/has received research support from the NIH (NIMH, NIA, NINDS, NINR) and the Greenwall Foundation. Dr. M. Kim is a member of the Scientific Merit Review Board for the Veterans Affairs Health Services Research and Development Field Program; and receives research support from the NIH (NIA, NIMH, NIDA) and the US Department of Veterans Affairs. Dr. Knopman serves as Deputy Editor for Neurology®; has served on a data safety monitoring board for Eli Lilly and Company; has served as a consultant for Elan/Janssen AI; is an investigator in clinical trials sponsored by Elan/Janssen AI, Baxter International Inc., and Forest Laboratories, Inc.; and receives research support from the NIH. Dr. De Vries receives research support from the National Library of Medicine, the NIH (NIA, NINDS, NIGMS, NICHD), and the Robert Wood Johnson Foundation. L. Damschroder has received funding for travel and speaker honoraria from the Robert Wood Johnson Foundation and receives research support from the US Department of Veterans Affairs, the NIH (NIA), and the European Commission/University of Geneva. Dr. Appelbaum is contributing editor to Psychiatric Services, is consulting editor for Ethics and Behavior, serves on the editorial board of the Journal of Forensic Psychiatry and Psychology (UK), Schizophrenia Bulletin, BioMed Central: Psychiatry, Psychiatry, Psychology and Law (Australia/NZ), and the American Journal of Bioethics—Primary Research, the advisory board of the International Review of Psychiatry and the editorial advisory board of Law and Human Behavior and Psychiatry and the international editorial board of the Journal of Ethics in Mental Health and the Brazilian Journal of Psychiatry; receives publishing royalties for Clinical Handbook of Psychiatry and the Law, 4 ^thed. (Lippincott Williams & Wilkins, 2007), Classification of Violence Risk (COVR): Professional Manual and Software (Psychological Assessment Resources, Inc., 2005), The MacArthur Competence Assessment Tool for Clinical Research (MacCAT-CR) (Professional Resource Press, 2001), Rethinking Risk Assessment: The MacArthur Study of Mental Disorder and Violence, Informed Consent: Legal Theory and Clinical Practice, 2^nd edition (Oxford University Press, 2001), Assessing Competence To Consent To Treatment: A Guide for Physicians and Other Health Professionals (Oxford University Press, 1998), MacArthur Competence Assessment Tool for Treatment (MacCAT-T) (Professional Resource Press, 1998), and Almost a Revolution: Mental Health Law and the Limits of Change (Oxford University Press, 1994); receives/has received research support from the NIH (NCI, NCRR, NIA, NIMH, NINR, NHGRI); and has an ownership share in COVR, Inc.

REFERENCES

1. Kim S, Caine E, Currier G, Leibovici A, Ryan J. Assessing the competence of persons with Alzheimer's disease in providing informed consent for participation in research. Am J Psychiatry 2001;158:712–717. [PubMed]

2. Okonkwo O, Griffith H, Belue K, et al. Medical decision-making capacity in patients with mild cognitive impairment. Neurology 2007;69:1528–1535. [PubMed]

3. Orgogozo J, Gilman S, Dartigues J, et al. Subacute meningoencephalitis in a subset of patients with AD after Aβ 42 immunization. Neurology 2003;61:46–54. [PubMed]

4. Tuszynski MH, Thal L, Pay M, et al. A phase 1 clinical trial of nerve growth factor gene therapy for Alzheimer disease. Nat Med 2005;11:551–555. [PubMed]

5. Kolata G. Doubt on tactic in Alzheimer's battle. The New York Times 2010. August18.

6. Kim SYH, Appelbaum PS, Jeste DV, Olin JT. Proxy and surrogate consent in geriatric neuropsychiatric research: update and recommendations. Am J Psychiatry 2004;161:797–806. [PubMed]

7. Hoffmann DE, Schwartz J. Proxy consent to participation of the decisionally impaired in medical research–Maryland's policy initiative. J Health Care Law Policy 1998;1:123–153. [PubMed]

8. Saks ER, Dunn LB, Wimer J, Gonzales M, Kim SYH. Proxy consent to research: legal landscape. Yale J Health Law Policy Ethics 2008;8:37–92. [PubMed]

9. Gutmann A, Thompson D. Deliberating about bioethics. Hastings Cent Rep 1997;27:38–41. [PubMed]

10. Kim SY, Kim HM, Langa KM, Karlawish JH, Knopman DS, Appelbaum PS. Surrogate consent for dementia research: a national survey of older Americans. Neurology 2009;72:149–155. [PMC free article] [PubMed]

11. Karlawish J, Rubright J, Casarett D, Cary M, Ten Have T, Sankar P. Older adults' attitudes toward enrollment of non-competent subjects participating in Alzheimer's research. Am J Psychiatry 2009;166:182–188. [PMC free article] [PubMed]

12. Kim SYH, Kim HM, McCallum C, Tariot PN. What do people at risk for Alzheimer's disease think about surrogate consent for research? Neurology 2005;65:1395–1401. [PubMed]

13. Kim S, Wall I, Stanczyk A, De Vries R. Assessing the public's views in research ethics controversies: deliberative democracy and bioethics as natural allies. J Empirical Res Hum Res Ethics 2009;4:3–16. [PMC free article] [PubMed]

14. Chambers S. Deliberative democratic theory. Annu Rev Pol Sci 2003;6:307–326.

15. Thompson DF. Deliberative democratic theory and empirical political science. Annu Rev Pol Sci 2008;11:497–520.

16. Gastil J, Keith WM. A nation that (sometimes) likes to talk: a brief history of public deliberation in the United States. In: Gastil J, Levine P, editors. , eds. The Deliberative Democracy Handbook San Francisco: Jossey-Bass; 2005:3–19.

17. Fishkin JS. Beyond polling alone: the quest for an informed public. Critical Review 2006;18:157–165.

18. Kim SYH, Uhlmann RA, Appelbaum PS, et al. Deliberative assessment of surrogate consent in dementia research. Alzheimer Dementia 2010;6:342–350. [PMC free article] [PubMed]

19. De Vries R, Stanczyk A, Wall IF, Uhlmann R, Damschroder LJ, Kim SY. Assessing the quality of democratic deliberation: a case study of public deliberation on the ethics of surrogate consent for research. Soc Sci Med 2010;70:1896–1903. [PMC free article] [PubMed]

Articles from Neurology are provided here courtesy of
American Academy of Neurology