PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptNIH Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Explore (NY). Author manuscript; available in PMC Jan 1, 2011.
Published in final edited form as:
PMCID: PMC2846631
NIHMSID: NIHMS171010
Attitudes and interests toward research among students at two colleges of acupuncture and Oriental medicine
PM Wayne, R Hammerschlag, JS German, and T Chapman
Corresponding author: Peter Wayne, PhD, Harvard Medical School, Division for Research and Education in Complementary and Integrative Medical Therapies, 401 Park Drive, Suite 22A, Boston, MA 02215, peter_wayne/at/hms.harvard.edu
Background
Collaborative input from clinicians of acupuncture and Oriental medicine (AOM) is required for sound AOM research, and AOM training institutions have begun to include research education into their curriculum. However, few attempts have been made to systematically evaluate AOM practitioners’ perspectives on the value of research to their profession.
Methods
We conducted surveys of AOM students at two institutions that have begun to integrate research training into their curriculum, the New England School of Acupuncture and the Oregon College of Oriental Medicine. Surveys were conducted to assess current attitudes regarding the value of research, and to serve as a reference point for documenting the impact of ongoing research training programs on these attitudes. Surveys at both institutions were independently developed and administered, but shared 7 questions that were phrased very similarly. This paper summarizes responses to these questions.
Results
Surveys at both institutions suggest interest in research among AOM students is high in first year students; students in later years showed a lower level of interest, but the cross-sectional design of this survey does not allow any temporal effects to be inferred. Results also indicate that AOM students believe research is highly relevant to how both the public and the health insurance industry view their system of health care, but not highly relevant to their own clinical practice of AOM. The belief that research is of limited relevance to clinical practice was associated with widespread belief that scientific methods may not be consistent with the principles of AOM.
Conclusions
Results of these surveys provide important preliminary information about AOM students attitudes towards research, and thus the value and future specific needs of research training programs targeting this population. Repeated implementation of validated versions of our surveys are needed to confirm the trends we report and to evaluate the impact of research training programs already in place on AOM students attitudes towards research.
Keywords: student survey, research education, complementary and alternative medicine, acupuncture
Faculty and students at complementary and alternative medicine (CAM) schools have the potential to make significant contributions to CAM research, especially toward ensuring that clinical research adequately reflects clinical practice.1 However, a significant barrier to the participation of CAM practitioners in the design as well as implementation of collaborative research is the limited research training and experience provided during CAM education.24
In recognition of this need for CAM practitioner research training, the National Center for Complementary and Alternative Medicine (NCCAM) at the National Institutes of Health (NIH) has developed several programs to build research infrastructure and enhance research literacy at CAM colleges. Two such initiatives are the U19 Developmental Centers for Research on CAM (DCRC)5 (http://grants.nih.gov/grants/guide/pa-files/PAR-03-046.html) and the R25 CAM Practitioner Research Education Project Grant Partnership (http://grants.nih.gov/grants/guide/pa-files/PAR-04-097.html).5 The program announcements for each of these grant opportunities framed research as a means of strengthening the integration of CAM and conventional medicine. To this end, both programs supported CAM colleges introducing an evidence-based healthcare perspective into their academic and clinical curricula.
Two acupuncture schools with a history of commitment to research, New England School of Acupuncture (NESA) and Oregon College of Oriental Medicine (OCOM), were awarded a DCRC (in 2003) and an R25 (in 2005), respectively. A key aim of both grants was to enrich the training of students and faculty in basic research competency, literacy and appreciation. As an integral part of programmatic evaluation, each school developed a survey to provide initial snapshots of students’ attitudes and interests toward research. The surveys were also intended for use in summative assessments of their grants. The present paper summarizes and compares baseline findings of the two surveys and discusses their implications for the infusion of research into the curriculum of acupuncture and Oriental medicine (AOM) colleges.
Research training in AOM colleges
The close to 50 nationally accredited colleges of AOM vary widely in their interest in introducing students to research as there is no national-level requirement for research coursework in their Master’s degree programs (the entry-level degree for the profession). In Fall 2004, for example, at the start of the NESA grant and prior to the start of the OCOM grant, a review of AOM college catalogs posted on the internet revealed that 27 of 48 (56%) accredited colleges offered a research course; at 23 of these institutions the course was required. In this context, NESA and OCOM are considered high-end exceptions since both colleges offer several required and/or elective research courses in addition to having formal research departments with research directors and a history of collaborative NIH/NCCAM-funded clinical research projects.
Research training efforts at NESA, supported by a DCRC grant, centered on collaboration with the Harvard Medical School (HMS) Osher Research Center. In addition to hands-on faculty and student research training via three mentored collaborative pilot studies, the program has supported the development and administration of a number of student and faculty research training classes and workshops. This program is aimed at improving basic research literacy knowledge of issues unique to AOM research, and serves as a foundation for students and faculty who wish to further pursue research training (see Wayne et al 2008 for details of specific initiatives).
OCOM’s R25 grant, Acupuncture Practitioner Research Education Enhancement (APREE), involves core collaboration with the Oregon Health & Science University School of Nursing. The major aims are to strengthen research literacy, foster a culture of research appreciation and gain recognition of research as a ‘way of knowing’ that can be integrated with clinical experience and patient preferences in the process of clinical decision-making. The grant has focused on faculty development, enhancement of a three-course research sequence for Master’s-level students, and infusion of research-related learning activities into existing non-research courses in a manner that does not increase curricular burden.68
Survey development and implementation
A first draft of NESA’s survey was developed by an interdisciplinary team of eight members including NESA’s research director and associate director, a senior NESA faculty member, one NESA student and recent graduate, NESA’s librarian, and HMS Osher’s research director and director of education. Individual members contributed lists of questions they thought relevant for gauging research interest and knowledge that were drawn from both personal experience and reviews of surveys conducted in different communities. Similar questions were combined through an iterative consensus process. A final instrument containing 38 questions was approved by NESA’s institutional review board and administered in Fall 2006 to 176 students enrolled across all three years of the Master of Acupuncture and Oriental Medicine program. A member of NESA’s research department entered mandatory 1st, 2nd and 3rd year classes, and explained to students that they were being asked to complete a survey for research purposes. The research staff person also described the voluntary nature of the process and left the room for 15 minutes, asking students to place their completed surveys in a large envelope if they chose to participate. Data from completed surveys were entered offsite by a professional data entry service.
Prior to completion of NESA’s survey instrument, an early draft was shared with OCOM. The content and wording of the questions were reviewed and revised by OCOM’s APREE evaluation committee after additional consideration of a similar survey developed by the Oregon Health & Science University-based Oregon CAM grant.9 The 34-question APREE survey was approved by the OCOM institutional review board and administered in August (graduating students) and September (new and continuing students) 2006 to a total of 227 students. Entering students were informed via their class confirmation letter of the web link to the survey. Continuing students were informed of the link via a flier placed in student mailboxes and via e-mail reminders. Fliers announcing the survey were also posted around campus. Surveys could be completed from students’ personal computers as well as from laptops set up outside the registrar’s office during the time of tuition payment appointments, which were scheduled for all students. The survey included a cover page that described the survey’s aim and a consent form that allowed students to opt out. Data from completed surveys were downloaded from the phpESP management interface into Excel spreadsheets.
The NESA and OCOM surveys both sought responses on a 5-level Likert scale. The surveys shared many questions. Seven specific questions relating to attitudes and opinions on research that were common to both surveys are presented in Table 1. These paired questions generally varied only slightly with respect to wording, but did vary in structure and format. The surveys also included questions on demographics, education and research experience.
Table 1
Table 1
Themes and Specific Questions Included in Student Research Surveys Administered at New England School of Acupuncture (NESA) and Oregon College of Oriental Medicine (OCOM)
Statistical analyses
The OCOM and NESA datasets were combined into a single dataset, and responses were standardized for consistency. Analyses on the merged dataset were conducted using SAS statistical software, version 9.1 (SAS Institute, Cary, NC). The strength of potential within-institution and between-institution differences were formally assessed using Mantel-Haenszel chi-squared and Pearson chi-squared tests.10 Because “year of enrollment” is an ordinal variable, Mantel-Haenszel tests were considered the more appropriate method for evaluating whether rates of agreement differed significantly among the three “year of enrollment” groups within each institution. Pearson chi-squared tests were also used to evaluate whether, for any particular cohort, the observed rates of agreement differed significantly between the two institutions.
Responder demographics
Sociodemographic characteristics of survey respondents at NESA and OCOM are presented in Table 2. Student enrollment at both schools was predominantly female (75.6% at NESA; 79.2% at OCOM) with a mean age in the mid 30’s. The proportion of students with bachelor’s degrees reflects the differing admission requirements at the two schools. NESA students must hold a BA, BS or RN, while OCOM requires 3 years of college at an accredited institution. In 2006, the NESA study body also included a higher proportion of students with prior postgraduate degrees. Further, approximately half of the NESA student responders reported having prior experience in research; this information was not collected in the OCOM survey.
Table 2
Table 2
Response Rate and Sociodemographic Characteristics of Responders
General survey results
Responses to each of the survey questions, summarized as percent of respondents either ‘agreeing’ or ‘strongly agreeing’, are compared between institutions in Figure 1; data with descriptive statistics are presented in Table 3. Noteworthy patterns emerging from these data are described below.
Figure 1
Figure 1
Comparisons of summed response rates (Agree plus Strongly Agree) to research-related survey questions at two schools of acupuncture and Oriental medicine. Data reflect cross-sectional assessments of student cohorts enrolled during a single calendar year, (more ...)
Table 3
Table 3
Cohort Trends in Attitudes towards Research at NESA and OCOM (Sample sizes: NESA=157 students; OCOM=192 students)
For 6 of the 7 questions, the average responses within classes as well as cross-sectional trends across classes were remarkably similar when compared between NESA and OCOM. The only major between-institution difference was in the importance students’ placed on staying current with clinical research in their field (Question 5). At both institutions, the large majority of first year students expressed interest in both learning about and participating in research (Questions 1 and 2); however, 2nd and 3rd year students were significantly less interested (Table 3). On average, the percent of students responding to these questions with ‘very interested’ or ‘somewhat interested’ (NESA) and ‘strongly agree’ or ‘agree’ (OCOM) was 75.7% (1st year) and 37.1 % (3rd year).
In contrast to this between-class disparity in self-interest in research, responses to questions 3 and 4 -- related to the perceived value of research to the AOM profession -- were positive and consistent across years. Changes in response rates between year-1 and year-3 students were not statistically significant at either school. “Agreement’ plus ‘strong agreement’ response rates at NESA and OCOM ranged from 80 – 95% concerning the impact of AOM research on public perception (question 3), and health insurance for (question 4), AOM.
Responses related to relevance of AOM research to clinical practice (questions 5 and 6) were less positive than responses to prior questions regarding relevance to public perception or insurance reimbursement. First-year NESA students appeared more skeptical than their counterparts at OCOM, although OCOM students’ agreement declined by class year on both questions, similar to NESA students on question 6.
Lastly, question 7 -- which addressed the consistency/congruence of the scientific method with AOM (phrased as a negative question) -- received generally low student endorsement at both schools, and showed an upward trend across classes. For both institutions, the percentage of respondents that either strongly or somewhat agreed that the scientific method is inconsistent with AOM was 36.6% in year 1 compared to 57.1% in year 3.
In reviewing our survey findings, we were encouraged to note that the majority of incoming students at both NESA and OCOM expressed interest in learning about and participating in research. This was also somewhat surprising as historically, like many CAM users,11,12 AOM/CAM practitioners often state they were attracted to CAM because it represents an alternative health paradigm to the research-based biomedical model. The relatively high level of interest in research we observed among incoming AOM students may reflect the fact that NESA and OCOM, as research-invested institutions, attract research-minded applicants. Interest in research may also be a reflection of the increasing numbers of students entering both AOM schools from undergraduate schools where they have had research-infused pre-med training (unpublished demographic data from NESA and OCOM).
Yet, despite this high level of interest in research training and participation among first year students, research interest was lower among students enrolled in later years of both programs. Because our data were collected as a cross-sectional survey, we cannot assume that trends across years necessarily reflect changes in cohort’s attitudes as they progress through the curriculum. However, if we assume these between-year results would hold for within-class longitudinal results, this would suggest an effect of education as students mature into clinicians. From the second year onwards in both programs, students devote an increasing amount of time to developing clinical skills and seeing patients, and thus their growing clinical interests may compete with and overshadow their initial enthusiasm and time for research. This tension between interests in clinic- and research-related activities is well known in conventional medical training, and partially underlies the widely reported difficulty of sustaining large numbers of clinician-researchers.13,14 Yet the true impact of education will only be known through longitudinal follow-up. Another possibility is that difference in interest in research between classes is related to differences in class characteristics (cohort effect) as opposed to education over time. We do see some demographic differences (gender, education) which may lend support to this theory. Nevertheless, even with the lower ratings seen in years 2 and 3 of both programs, approximately 40–50% of third-year AOM students still expressed interest in learning about and/or participating in research. With further development and administration of research training initiatives at both institutions, we anticipate that the initial interest in research may be sustained at a higher level.
A second interesting result of our survey is the striking difference in students’ beliefs in the relevance of research to those outside the profession versus those practicing within it. Students nearly unanimously agreed that research would improve public perception of acupuncture, and would positively impact insurance coverage for acupuncture treatment. In contrast, and especially for NESA students, there was markedly less support for the statements suggesting that research will inform clinical practice. These responses may be related to belief about the consistency of the scientific method with AOM principles, which was rated on average very low. Over time, we expect a greater proportion of students at NESA and OCOM to believe that well designed research is not incongruous with AOM, and that research, and knowledge of this research, can inform clinical practice.
Limitations of the study
While survey results from the two AOM institutions showed marked similarities, it is important to emphasize a number of limitations of this study. First, while the survey questions we evaluated were administered at two independent AOM schools, the generalizability of the observations to other AOM schools may be limited. NESA and OCOM are two AOM schools with a long-standing commitment to research and research education, and thus students attracted to and attending these institutions may not share research-related values with students at other AOM and CAM institutions less committed to research.
Second, the instruments used at both institutions have not yet been evaluated for validity and reliability, and thus may be biased.15 Additionally, there are important differences in the format of questions used in surveys at both institutions. For example, level of agreement to most questions at OCOM was assessed with an affirmative statement, while at NESA, it was assessed by degree of interest. While such differences in format could technically bias comparisons between institutions, the remarkable parallel in responses between institutions suggests this was unlikely. Finally, one questions (#7) was negatively worded, and it is possible that this phrasing impacted responses.
Third, the results presented reflect only one point in time. They were collected to serve as a baseline for ongoing evaluation of research-related values and, more broadly, to assess the impact of initiatives to encourage research literacy and participation. As such, it is important not to over interpret trends across cohorts within an institution as longitudinal changes in attitudes within a cohort. Repeated administration of validated versions of our surveys will address and remedy the above issues.
However, despite the preliminary nature of our results, the survey questions included in this paper address timely and relevant issues regarding the goals and value of research training at CAM schools. To our knowledge, these issues have not been systematically explored at CAM institutions. We hope that this preliminary publication of our results stimulates other CAM institutions to conduct parallel evaluations. This would enable comparisons regarding research interests to be made, and inferences to be drawn, from a much larger and diverse set of CAM institutions.
Implications for evidence based healthcare and research education
It is increasingly believed that CAM practices such as AOM must be based on sound clinical and basic research and be practiced in an evidence-based manner if they are to contribute to the future of health care. This is especially the case in settings where CAM therapies are integrated with conventional medicine.1618 It has also been argued that CAM research requires the active collaboration of experienced CAM practitioners.2,4,19,20 Knowledgeable practitioners are needed not only to provide effective treatments in the context of clinical or mechanistic studies, but are needed to inform all aspects of research–– from study conception and identification of relevant hypotheses to development of treatment interventions, choosing outcomes measures, and interpreting results. For this to happen, there is a need to enhance research training in the curriculum of CAM institutions. This need has been recognized by NCCAM/NIH and is reflected in a number of innovative initiatives to build research literacy and research infrastructure at CAM institutions, including the DCRC and R25 grants awarded to NESA and OCOM, respectively.20,21
However, the relevance of research to CAM practitioners and CAM professions as a whole has been questioned by some within the field based on perceived limitations of widely used research designs to evaluate CAM efficacy and underlying mechanisms.2225 For example, uncertainties regarding the inertness of sham needle controls, or the attempt in some trials to minimize patient-practitioner interaction bring into question whether these methods grossly bias the intervention being evaluated and lead to artificially reduced effects sizes. Additionally, resistance to research may result from the association of research with more mainstream and reductionist medicine as a whole, and the political desires of some practitioners to maintain autonomy/independence from it. Surprisingly, few attempts have been made to systematically evaluate CAM practitioners’ perspectives on the value of research to their profession.26 To the best of our knowledge, this paper reflects the first attempt to directly and systematically assess AOM students’ perspectives on the potential value of integrating research education into their AOM training.
A central aim of NCCAM’s DCRC and R25 programs is to more fully engage CAM practitioners in research. Based on the results of this survey, and more broadly, on our experience developing research programs and research education initiatives at NESA and OCOM, we believe that it may be useful to classify general levels of research interest among our students into three broad categories. First, there is a large and growing number of students who appreciate that sound research is essential for the development of the AOM profession. Consequently, as eventual practitioners, they feel it is essential to develop AOM-relevant research literacy competencies, e.g., a basic knowledge of the principles of clinical and basic research, an ability to search for and evaluate AOM research findings, and an appreciation of issues unique to studying complex interventions such as AOM. These competencies will position them to communicate more effectively with both patients and conventional practitioners, and will inform some aspects of their clinical practice. Second, there is a small number of students, including those with prior research experience, who seek to include research as part of their careers. As clinician-researchers, they may choose to engage in research in a number of ways including collaboratively developing treatment protocols and providing treatments in clinical trials, helping in the overall design of studies, and contributing to evidence-based reviews of the literature. A smaller proportion of these clinician-researchers with appropriate levels of training might even serve as study principal investigators. Finally, we believe there will always be a third group of students who have minimal interest in research and believe it is largely irrelevant to their practice. A common view among this group of students is that, because some aspects of AOM are based on energy medicine (and more intuitive processes), the current emphasis on reductionist scientific methods is of little value for developing clinically relevant research. Right or wrong, we believe this voice adds important diversity to the student body and to the profession. Nevertheless, it is possible that as AOM research methodology develops, and more readily adopts experimental designs and interventions that more accurately reflect clinical practice (e.g. pragmatic clinical trials), and as these approaches are highlighted in research courses, the size of this third group of students may decrease.
With these distinctions in mind, NESA and OCOM’s research training programs are aimed predominantly at impacting the first two groups of students. Both AOM schools include required courses that provide training in basic research principles, as well as practical skills such as identifying, retrieving, reading, and evaluating original literature. For NESA students interested in additional training, research electives that teach more advanced research principles and skills are now offered.2 Additionally, a key component of more advanced research training at both institutions occurs via hands-on participation in clinical trials (e.g. research assistants), surveys and case series research, and the writing of peer-reviewed scientific papers. In conventional medicine, there is a well-established positive relationship between participation in research during medical school and post-graduate research.14,27 Finally, both NESA and OCOM have also developed parallel research training initiatives for faculty and staff.2,7 Initial outcomes from these faculty initiatives have been very positive, and are playing a central role in catalyzing the infusion of research into broad aspects of the curriculum as well as the culture of our AOM institutions.
Summary
This survey represent the first attempt to systematically assess AOM students’ perspectives on the potential value of integrating research education into their AOM training. Preliminary results suggest interest in research among AOM students is high, early on in training. Results also indicate that AOM students believe research is highly relevant to how both the public and the health insurance industry view their system of health care, but not highly relevant to their own clinical practice of AOM. Repeated implementation of validated versions of our surveys are needed to confirm the trends we report and to evaluate the impact of research training programs already in place on AOM students attitudes towards research.
Acknowledgments
This project was supported by grant numbers 5 U19 AT002022 (NESA) and R25 AT002879 (OCOM) from the National Center for Complementary and Alternative Medicine (NCCAM), NIH. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the NCCAM, or the National Institutes of Health. We thank Belinda Anderson, Steve Schachter, Della Lawhon, and Monica Shields for assistance in the development and administration of the NESA survey, and Shauna McCuaig, Catherine Salveson and Sonya Salanti for assistance in the development and administration of the OCOM survey.
Footnotes
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
1. Lao L, Sherman K, Bovey M. The role of acupuncture schools and individual practitioners in acupuncture research. Clinical Acupuncture and Oriental Medicine. 2002;3:32–38.
2. Wayne PM, Buring JE, Davis RB, et al. Increasing research capacity at the New England School of Acupuncture through faculty and student research training initiatives. Altern Ther Health Med. 2008;14:52–58. [PubMed]
3. Wayne PM, Pensack LM, Connors EM, et al. Increasing research capacity at the New England School of Acupuncture: building grants management infrastructure. Altern Ther Health Med. 2008;14:56–64. [PubMed]
4. Wayne PM, Sherman KJ, Bovey M. Engaging acupuncturists in research. In: MacPherson H, Lewith G, editors. Acupuncture research: strategies for building an evidence base. London: Elsevier Science; 2008. pp. 219–237.
6. Sierpina V, Bulik R, Baldwin C, et al. Creating sustainable curricular change: lessons learned from an alternative therapies educational initiative. Acad Med. 2007;82:341–350. [PubMed]
7. Hammerschlag R, Lasater K, Salanti S, Fleishman S. Research scholars program: a faculty development initiative at the Oregon College of Oriental Medicine. J Altern Complement Med. 2008;14:437–443. [PubMed]
8. Kreitzer MJ, Sierpina VS. Explore. Vol. 4. NY: 2008. NCCAM awards grants to CAM institutions to enhance research education; pp. 74–76. [PubMed]
9. Nedrow AR, Heitkemper M, Frenkel M, Mann D, Wayne P, Hughes E. Collaborations between allopathic and complementary and alternative medicine health professionals: four initiatives. Acad Med. 2007;82:962–966. [PubMed]
10. Agresti A. Introduction to Categorical Data Analysis. New York: John Wiley and Sons; 1996.
11. Astin JA. Why patients use alternative medicine: results of a national study. Jama. 1998;279:1548–1553. [PubMed]
12. Kaptchuk TJ, Eisenberg DM. The persuasive appeal of alternative medicine. Ann Intern Med. 1998;129:1061–1065. [PubMed]
13. Wyngaarden JB. The clinical investigator as an endangered species. N Engl J Med. 1979;301:1254–1259. [PubMed]
14. Fang D, Meyer RE. Effect of two Howard Hughes Medical Institute research training programs for medical students on the likelihood of pursuing research careers. Acad Med. 2003;78:1271–1280. [PubMed]
15. Neuman WL. Social Resaerch Methods: Qualitative and Quantitative Appraoches. 5. Boston: Allyn and Bacon; 2003.
16. Maha N, Shaw A. Academic doctors’ views of complementary and alternative medicine (CAM) and its role within the NHS: an exploratory qualitative study. BMC Complement Altern Med. 2007;7:17. [PMC free article] [PubMed]
17. Sagar SM. Integrative oncology in North America. J Soc Integr Oncol. 2006;4:27–39. [PubMed]
18. Ruggie M. Health Aff. Vol. 24. Millwood: 2005. Mainstreaming complementary therapies: new directions in health care; pp. 980–990. [PubMed]
19. Hammerschlag R. Acupuncture: on what should its evidence base be based? Altern Ther Health Med. 2003;9:34–35. [PubMed]
20. Pearson NJ, Chesney MA. The CAM Education Program of the National Center for Complementary and Alternative Medicine: an overview. Acad Med. 2007;82:921–926. [PubMed]
21. Khalsa PS, Pearson NJ. Financial support for research training and career development in complementary and alternative medicine from the National Institutes of Health. J Manipulative Physiol Ther. 2007;30:483–490. [PubMed]
22. Fonnebo V, Grimsgaard S, Walach H, et al. Researching complementary and alternative treatments--the gatekeepers are not at home. BMC Med Res Methodol. 2007;7:7. [PMC free article] [PubMed]
23. Jackson S, Scambler G. Perceptions of evidence-based medicine: traditional acupuncturists in the UK and resistance to biomedical modes of evaluation. Sociol Health Illn. 2007;29:412–429. [PubMed]
24. McManus CA, Schnyer RN, Kong J, et al. Sham acupuncture devices--practical advice for researchers. Acupunct Med. 2007;25:36–40. [PubMed]
25. Colbert AP, Hammerschlag R, Aickin M, McNames J. Reliability of the Prognos electrodermal device for measurements of electrical skin resistance at acupuncture points. J Altern Complement Med. 2004;10:610–616. [PubMed]
26. Claraco AE, Fargas-Babjak A, Hanna SE. The reporting of clinical acupuncture research: what do clinicians need to know? J Altern Complement Med. 2003;9:143–149. [PubMed]
27. Segal S, Lloyd T, Houts PS, Stillman PL, Jungas RL, Greer RB., 3rd The association between students’ research involvement in medical school and their postgraduate medical activities. Acad Med. 1990;65:530–533. [PubMed]