Search tips
Search criteria

Results 1-25 (1001181)

Clipboard (0)

Related Articles

1.  Sicily statement on classification and development of evidence-based practice learning assessment tools 
BMC Medical Education  2011;11:78.
Teaching the steps of evidence-based practice (EBP) has become standard curriculum for health professions at both student and professional levels. Determining the best methods for evaluating EBP learning is hampered by a dearth of valid and practical assessment tools and by the absence of guidelines for classifying the purpose of those that exist. Conceived and developed by delegates of the Fifth International Conference of Evidence-Based Health Care Teachers and Developers, the aim of this statement is to provide guidance for purposeful classification and development of tools to assess EBP learning.
This paper identifies key principles for designing EBP learning assessment tools, recommends a common taxonomy for new and existing tools, and presents the Classification Rubric for EBP Assessment Tools in Education (CREATE) framework for classifying such tools. Recommendations are provided for developers of EBP learning assessments and priorities are suggested for the types of assessments that are needed. Examples place existing EBP assessments into the CREATE framework to demonstrate how a common taxonomy might facilitate purposeful development and use of EBP learning assessment tools.
The widespread adoption of EBP into professional education requires valid and reliable measures of learning. Limited tools exist with established psychometrics. This international consensus statement strives to provide direction for developers of new EBP learning assessment tools and a framework for classifying the purposes of such tools.
PMCID: PMC3221624  PMID: 21970731
2.  Assessing competency in Evidence Based Practice: strengths and limitations of current tools in practice 
Evidence Based Practice (EBP) involves making clinical decisions informed by the most relevant and valid evidence available. Competence can broadly be defined as a concept that incorporates a variety of domains including knowledge, skills and attitudes. Adopting an evidence-based approach to practice requires differing competencies across various domains including literature searching, critical appraisal and communication. This paper examines the current tools available to assess EBP competence and compares their applicability to existing assessment techniques used in medicine, nursing and health sciences.
Only two validated assessment tools have been developed to specifically assess all aspects of EBP competence. Of the two tools (Berlin and Fresno tools), only the Fresno tool comprehensively assesses EBP competency across all relevant domains. However, both tools focus on assessing EBP competency in medical students; therefore neither can be used for assessing EBP competency across different health disciplines. The Objective Structured Clinical Exam (OSCE) has been demonstrated as a reliable and versatile tool to assess clinical competencies, practical and communication skills. The OSCE has scope as an alternate method for assessing EBP competency, since it combines assessment of cognitive skills including knowledge, reasoning and communication. However, further research is needed to develop the OSCE as a viable method for assessing EBP competency.
Demonstrating EBP competence is a complex task – therefore no single assessment method can adequately provide all of the necessary data to assess complete EBP competence. There is a need for further research to explore how EBP competence is best assessed; be it in written formats, such as the Fresno tool, or another format, such as the OSCE. Future tools must also incorporate measures of assessing how EBP competence affects clinician behaviour and attitudes as well as clinical outcomes in real-time situations. This research should also be conducted across a variety of health disciplines to best inform practice.
PMCID: PMC2728711  PMID: 19656412
3.  Effectiveness of the tailored EBP training program for Filipino physiotherapists: A randomised controlled trial 
BMC Medical Education  2011;11:14.
Evidence implementation continues to challenge health professionals most especially those from developing countries. Filipino physiotherapists represent a group of health professionals in a developing country who by tradition and historical practice, take direction from a doctor, on treatment options. Lack of autonomy in decision-making challenges their capacity to deliver evidence-based care. However, this scenario should not limit them from updating and up-skilling themselves on evidence- based practice (EBP). EBP training tailored to their needs and practice was developed to address this gap. This study will be conducted to assess the effectiveness of a tailored EBP-training program for Filipino physiotherapists, in improving knowledge, skills, attitudes and behaviour to EBP. Participation in this program aims to improve capacity to EBP and engage with referring doctors to determine the most effective treatments for their patients.
A double blind randomised controlled trial, assessing the effectiveness of the EBP training intervention, compared with a waitlist control, will be conducted. An adequately powered sample of 54 physiotherapists from the Philippines will be recruited and randomly allocated to EBP intervention or waitlist control.
Intervention: The EBP program for Filipino physiotherapists is a one-day program on EBP principles and techniques, delivered using effective adult education strategies. It consists of lectures and practical workshops. A novel component in this program is the specially-developed recommendation form, which participants can use after completing their training, to assist them to negotiate with referring doctors regarding evidence-based treatment choices for their patients.
Pre and post measures of EBP knowledge, skills and attitudes will be assessed in both groups using the Adapted Fresno Test and the Questions to EBP attitudes. Behaviour to EBP will be measured using activity diaries for a period of three months.
Should the EBP-training program be found to be effective in improving EBP-uptake in Filipino physiotherapists, it will form the basis of a much needed national intervention which is contextually appropriate to Filipino physiotherapists. It will therefore form the genesis for a model for building EBP capacity of other health professionals in the Philippines as well as physiotherapists in developing countries.
Trial Registration
Current Controlled Trials: ISRCTN74485061
PMCID: PMC3100239  PMID: 21489229
4.  Development and psychometric testing of an instrument to evaluate cognitive skills of evidence based practice in student health professionals 
BMC Medical Education  2011;11:77.
Health educators need rigorously developed instruments to evaluate cognitive skills relating to evidence based practice (EBP). Previous EBP evaluation instruments have focused on the acquisition and appraisal of the evidence and are largely based in the medical profession. The aim of this study was to develop and validate an EBP evaluation instrument to assess EBP cognitive skills for entry-level health professional disciplines.
The Fresno test of competence in evidence based medicine was considered in the development of the 'Knowledge of Research Evidence Competencies' instrument (K-REC). The K-REC was reviewed for content validity. Two cohorts of entry-level students were recruited for the pilot study, those who had been exposed to EBP training (physiotherapy students, n = 24), and who had not been exposed to EBP training (human movement students, n = 76). The K-REC was administered to one cohort of students (n = 24) on two testing occasions to evaluate test-retest reliability. Two raters independently scored the first test occasion (n = 24) to evaluate the inter-rater reliability of the marking guidelines. Construct validity was assessed by comparison of the two groups, 'exposed' and 'non-exposed', and the percentage of students achieving a 'pass' score in each of these groups. Item difficulty was established.
Among the 100 participants (24 EBP 'exposed', and 76 EBP 'non-exposed' students), there was a statistically significant (p < 0.0001) difference in the total K-REC scores. The test-retest and inter-rater reliability of the individual items and total scores ranged from moderate to excellent (measured by Cohen's Kappa and ICC, range: 0.62 to perfect agreement).
The K-REC instrument is a valid and reliable evaluation instrument of cognitive skills of EBP in entry-level student health professionals. The instrument is quick to disseminate and easy to score, making it a suitable instrument for health educators to employ to evaluate students' knowledge of EBP or in the evaluation of entry-level EBP training.
PMCID: PMC3196731  PMID: 21967728
5.  Evidence-based practice training for health professionals in the Philippines 
Evidence-based practice (EBP) is integral in the health care system whether in developed or developing countries. Thus, all health professionals need to be trained in EBP. An EBP training program was conducted to health professionals in a developing country, the Philippines. The health professionals (medical doctors and allied health professionals [physical therapists and occupational therapists]) were working in hospitals in Manila, Philippines. The program aimed to build capacity in EBP in terms of knowledge and skills. The EBP training program was conducted as a 1-day face-to-face training. Pre- and post-test measures of EBP knowledge and skills were taken prior to and immediately after the 1-day training, using the Fresno test of evidence-based medicine for the medical doctors and the Adapted Fresno test for the allied health professionals. The EBP training program resulted in significant improvements in knowledge and skills for both the medical doctors (change in pre- and post-Fresno test measures, 95% confidence interval [CI]: 14.6–23.5; P≤0.05) and the allied health professionals (change in pre- and post-Adapted Fresno test measures, 95% CI: 32.7–38.5; P≤0.05). The EBP training conducted amongst the health professionals is an effective and tested undertaking in introducing EBP in developing countries such as the Philippines.
PMCID: PMC3998855  PMID: 24790472
continuing education; developing countries physical therapy; medicine; knowledge and skills
6.  Validation of the modified Fresno Test: assessing physical therapists' evidence based practice knowledge and skills 
BMC Medical Education  2010;10:38.
Health care educators need valid and reliable tools to assess evidence based practice (EBP) knowledge and skills. Such instruments have yet to be developed for use among physical therapists. The Fresno Test (FT) has been validated only among general practitioners and occupational therapists and does not assess integration of research evidence with patient perspectives and clinical expertise. The purpose of this study was to develop and validate a modified FT to assess EBP knowledge and skills relevant to physical therapist (PT) practice.
The FT was modified to include PT-specific content and two new questions to assess integration of patient perspectives and clinical expertise with research evidence. An expert panel reviewed the test for content validity. A cross-sectional cohort representing three training levels (EBP-novice students, EBP-trained students, EBP-expert faculty) completed the test. Two blinded raters, not involved in test development, independently scored each test. Construct validity was assessed through analysis of variance for linear trends among known groups. Inter and intra-rater reliability, internal consistency, item discrimination index, item total correlation, and difficulty were analyzed.
Among 108 participants (31 EBP-novice students, 50 EBP-trained students, and 27 EBP-expert faculty), there was a statistically significant (p < 0.0001) difference in total score corresponding to training level. Total score reliability and psychometric properties of items modified for discipline-specific content were excellent [inter-rater (ICC (2,1)] = 0.91); intra-rater (ICC (2,1)] = 0.95, 0.96)]. Cronbach's α was 0.78. Of the two new items, only one had strong psychometric properties.
The 13-item modified FT presented here is a valid, reliable assessment of physical therapists' EBP knowledge and skills. One new item assesses integration of patient perspective as part of the EBP model. Educators and researchers may use the 13-item modified FT to evaluate PT EBP curricula and physical therapists' EBP knowledge and skills.
PMCID: PMC2896364  PMID: 20500871
7.  Implementation of evidence-based practice across medical, nursing, pharmacological and allied healthcare professionals: a questionnaire survey in nationwide hospital settings 
Implementation of evidence-based practice (EBP) is regarded as core competence to improve healthcare quality. In the current study, we investigated the EBP of six groups of professionals: physicians, nurses, pharmacists, physical therapists, technicians, and other allied healthcare personnel.
A structured questionnaire survey of regional hospitals throughout Taiwan was conducted by post in 2011. Questionnaires were mailed to all healthcare workers of 11 randomly selected hospitals. Linear and logistic regression models were used to examine predictors for implementing EBP.
In total, 6,160 returned questionnaires, including 645 from physicians, 4,206 from nurses, 430 from pharmacists, 179 from physical therapists, 537 from technicians, and 163 from other allied healthcare professionals, were valid for the analysis. Physicians and pharmacists were more aware of EBP than were the other professional groups (p < 0.001). Positive attitudes toward and beliefs in EBP were significantly lower among nurses than in the other groups (p < 0.001). Physicians had more sufficient knowledge and skills of EBP than did the other professionals (p < 0.001); in addition, they implemented EBP for clinical decision-making more often and perceived fewer personal barriers to EBP (p < 0.001). Multivariate logistic regression analyses showed that EBP implementation was associated with the following characteristics of participants: EBP training, having a faculty position, academic degree, one's profession, and perceptions (beliefs, attitudes, knowledge, skills and barriers).
This study depicts various levels of EBP implementation among medical, nursing, pharmacological, and allied healthcare personnel. There were significant differences in their implementation of EBP. We observed that certain factors were associated with EBP implementation, including personal backgrounds and perceptions toward EBP. The data suggest that strategies for enhancing EBP implementation should differ for various groups of professionals.
PMCID: PMC3849261  PMID: 24063756
8.  International web survey of chiropractic students about evidence-based practice: a pilot study 
Positive attitude toward evidence-based practice (EBP) principles in healthcare education may be one of the first steps for motivating a healthcare professional student to later apply EBP principles in clinical decision-making. The objectives for this project were to pilot an international web-based survey of chiropractic students and to describe student attitudes, behaviors, and knowledge about EBP principles.
We used SurveyMonkey™ to develop our survey based on an existing questionnaire used to measure basic knowledge, skills and beliefs about EBP among allied healthcare professionals and CAM practitioners. We invited 26 chiropractic educational institutions teaching in English and accredited by official organizations to participate. Academic officials and registrars at participating institutions forwarded an invitation email and two reminders to students between July and September 2010. The invitation contained a link to the 38-item web-based questionnaire. Descriptive statistics were performed for analysis.
Fourteen institutions from Australia, Canada, US, Denmark and New Zealand participated. Among an estimated 7,142 student recipients of invitation letters, 674 participated in the survey for an estimated response rate of 9.4%. Most respondents reported having access to medical/healthcare literature through the internet, but only 11% read literature every week and 21% did not read literature at all. Respondents generally agreed that the use of research evidence in chiropractic was important. Although 76% of respondents found it easy to understand research evidence and 81% had some level of confidence assessing the general worth of research articles, 71% felt they needed more training in EBP to be able to apply evidence in chiropractic care. Respondents without previous training in research methods had lower confidence in assessing published papers. While more than 60% marked the correct answer for two knowledge items, the mean number of correct answers to the five knowledge questions was 1.3 (SD 0.9).
Although it is feasible to conduct an international web survey of chiropractic students, significant stakeholder participation is important to improve response rates. Students had relatively positive attitudes toward EBP. However, participants felt they needed more training in EBP and based on the knowledge questions they may need further training about basic research concepts.
PMCID: PMC3058064  PMID: 21371309
9.  A modified evidence-based practice- knowledge, attitudes, behaviour and decisions/outcomes questionnaire is valid across multiple professions involved in pain management 
BMC Medical Education  2014;14(1):263.
A validated and reliable instrument was developed to knowledge, attitudes and behaviours with respect to evidence-based practice (EBB-KABQ) in medical trainees but requires further adaptation and validation to be applied across different health professionals.
A modified 33-item evidence-based practice scale (EBP-KABQ) was developed to evaluate EBP perceptions and behaviors in clinicians. An international sample of 673 clinicians interested in treatment of pain (mean age = 45 years, 48% occupational therapists/physical therapists, 25% had more than 5 years of clinical training) completed an online English version of the questionnaire and demographics. Scaling properties (internal consistency, floor/ceiling effects) and construct validity (association with EBP activities, comparator constructs) were examined. A confirmatory factor analysis was used to assess the 4-domain structure EBP knowledge, attitudes, behavior, outcomes/decisions).
The EBP-KABQ scale demonstrated high internal consistency (Cronbach’s alpha = 0.85), no evident floor/ceiling effects, and support for a priori construct validation hypotheses. A 4-factor structure provided the best fit statistics (CFI =0.89, TLI =0.86, and RMSEA = 0.06).
The EBP-KABQ scale demonstrates promising psychometric properties in this sample. Areas for improvement are described.
Electronic supplementary material
The online version of this article (doi:10.1186/s12909-014-0263-4) contains supplementary material, which is available to authorized users.
PMCID: PMC4272818  PMID: 25495467
Evidence-based; Scale; Self-reported; Validation; Clinician
10.  Teaching evidence based practice in physical therapy in a developing country: a national survey of Philippine schools 
BMC Medical Education  2013;13:154.
Early education on the foundations of evidence based practice (EBP) is advocated as a potent intervention toward enhancing EBP uptake among physical therapists. Little is known about the extent to which EBP is integrated in educational curricula in developing countries where the benefits of EBP are more acutely needed. This study sought to describe EBP education in Philippine physical therapy schools, including the challenges encountered by educators in teaching EBP.
A national survey of higher education institutions offering an undergraduate degree program in physical therapy was conducted from August 2011 through January 2012. A 35-item questionnaire was developed to gather data on whether or not EBP was taught, specific EBP content covered and courses in which content was covered, teaching and evaluation methods, and challenges in teaching EBP. Data were analyzed descriptively.
The study had a response rate of 55.7% (34/61). Majority of the participating educational institutions (82%, 28/34) reported teaching EBP by incorporating EBP content in the professional courses. Among those that did not teach EBP, inadequate educator competence was the leading barrier. Courses commonly used to teach EBP were those on research (78.6%, 22/28), therapy planning (71.4%, 20/28), treatment skills (57.1-64.3%, 16-18/28), and undergraduate thesis (60.7%, 17/28). Various EBP contents were covered, with statistical concepts more frequently taught compared with critical EBP content. Lectures and journal reports were the usual teaching methods (96.4%, 27/28 and 89.3%, 25/28, respectively) while written examinations, completion of an undergraduate thesis, and oral reports (82.1%, 23/28, 78.6%, 22/28, and 78.6%, 22/28, respectively) were often used in evaluation. Students’ inadequate knowledge of statistics and lack of curricular structure for EBP were identified as leading challenges to teaching (75%, 21/28 and 50%, 14/28, respectively).
Many physical therapy faculties across the Philippines are incorporating EBP content in teaching. However, there is arbitrary and fragmented coverage of EBP content and inadequate emphasis on clinically oriented teaching-learning and assessment methods. These findings suggest the need to design appropriate entry-level educational programs on EBP. Effective ‘educating the educators’ strategies are urgently needed and can have far-reaching positive repercussions on EBP uptake in physical therapist practice.
PMCID: PMC3874799  PMID: 24267512
Allied health; Evidence based health care; Medical education; Teaching methods; Under-developed countries
11.  Evidence-based practice profiles of physiotherapists transitioning into the workforce: a study of two cohorts 
BMC Medical Education  2011;11:100.
Training in the five steps of evidence-based practice (EBP) has been recommended for inclusion in entry-level health professional training. The effectiveness of EBP education has been explored predominantly in the medical and nursing professions and more commonly in post-graduate than entry-level students. Few studies have investigated longitudinal changes in EBP attitudes and behaviours. This study aimed to assess the changes in EBP knowledge, attitudes and behaviours in entry-level physiotherapy students transitioning into the workforce.
A prospective, observational, longitudinal design was used, with two cohorts. From 2008, 29 participants were tested in their final year in a physiotherapy program, and after the first and second workforce years. From 2009, 76 participants were tested in their final entry-level and first workforce years. Participants completed an Evidence-Based Practice Profile questionnaire (EBP2), which includes self-report EBP domains [Relevance, Terminology (knowledge of EBP concepts), Confidence, Practice (EBP implementation), Sympathy (disposition towards EBP)]. Mixed model analysis with sequential Bonferroni adjustment was used to analyse the matched data. Effect sizes (ES) (95% CI) were calculated for all changes.
Effect sizes of the changes in EBP domains were small (ES range 0.02 to 0.42). While most changes were not significant there was a consistent pattern of decline in scores for Relevance in the first workforce year (ES -0.42 to -0.29) followed by an improvement in the second year (ES +0.27). Scores in Terminology improved (ES +0.19 to +0.26) in each of the first two workforce years, while Practice scores declined (ES -0.23 to -0.19) in the first year and improved minimally in the second year (ES +0.04). Confidence scores improved during the second workforce year (ES +0.27). Scores for Sympathy showed little change.
During the first two years in the workforce, there was a transitory decline in the self-reported practice and sense of relevance of EBP, despite increases in confidence and knowledge. The pattern of progression of EBP skills beyond these early professional working years is unknown.
PMCID: PMC3248363  PMID: 22126299
12.  Knowledge and beliefs concerning evidence-based practice amongst complementary and alternative medicine health care practitioners and allied health care professionals: A questionnaire survey 
Evidence-based practice (EBP) has become an important competency in many allied and complementary and alternative medicine (CAM) health care practitioners' professional standards of proficiency.
To compliment an EBP course for allied health care professionals and CAM practitioners, we undertook a questionnaire survey to assess learning needs. We developed a questionnaire to measure allied health care professionals and CAM practitioners' basic knowledge, skills and beliefs concerning the main principles of EBP. The questionnaires were administered to all attendees of one-day EBP workshops.
During 2004–5 we surveyed 193 allied health care professionals and CAM practitioners who attended one-day EBP courses prior to commencement of teaching. Of the respondents 121 (62.7%) were allied health care professionals and 65 (33.7%) practitioners stated that they work in the CAM field Our survey found that the majority of the respondents had not previously attended a literature appraisal skills workshop (87.3%) or received formal training in research methods (69.9%), epidemiology (91.2%) or statistics (80.8%). Furthermore, 67.1% of practitioners specified that they felt that they had not had adequate training in EBM and they identified that they needed more training and education in the principles of EBM (86.7%). Differences in knowledge and beliefs concerning EBP amongst allied and CAM practitioners were found and length of time since qualification was also found to be an important factor in practitioner's beliefs. More CAM practitioners compared to allied health professionals accessed educational literature via the Internet (95.3% v 68.1%, p = 0.008). Whilst, practitioners with more than 11 years experience felt that original research papers were far more confusing (p = 0.02) than their less experienced colleagues.
The results demonstrate that practitioner's learning needs do vary according to the type of profession, time since graduation and prior research experience. Our survey findings are exploratory and will benefit from further replication, however, we do believe that they warrant consideration by allied health care and CAM tutors and trainers when planning EBP teaching curricula as it is important to tailor teaching to meet the needs of specific subgroups of trainees to ensure that specific learning needs are met.
PMCID: PMC2533291  PMID: 18651937
13.  Implementation of a blended learning approach to teaching evidence based practice: a protocol for a mixed methods study 
BMC Medical Education  2013;13:170.
Evidence based practice (EBP) requires that health professionals are competent in integrating the best evidence in their decision making. Being 'evidence-based’ requires skills and knowledge in epidemiology, biostatistics and information literacy. EBP is commonly taught in medical and health sciences degrees, yet there is little evidence to guide educators as to the best teaching modality to increase learner competency in EBP.
This study is mixed methods in design. A randomised controlled trial will examine the effectiveness of blended learning versus didactic approach of teaching EBP to medical students. The primary outcome of the RCT is EBP competency as assessed by the Berlin tool. Focus groups will be conducted to explore student perceptions and attitudes towards implementing a blended learning approach in teaching EBP. A concurrent triangulation design will be implemented, permitting quantitative data to inform the effectiveness of the intervention and qualitative data to contextualise the results.
This study will provide novel evidence on the effectiveness of blended learning in teaching EBP to a cohort of undergraduate and graduate-entry medical students.
PMCID: PMC3878342  PMID: 24351113
14.  A KT intervention including the evidence alert system to improve clinician’s evidence-based practice behavior—a cluster randomized controlled trial 
It is difficult to foster research utilization among allied health professionals (AHPs). Tailored, multifaceted knowledge translation (KT) strategies are now recommended but are resource intensive to implement. Employers need effective KT solutions but little is known about; the impact and viability of multifaceted KT strategies using an online KT tool, their effectiveness with AHPs and their effect on evidence-based practice (EBP) decision-making behavior. The study aim was to measure the effectiveness of a multifaceted KT intervention including a customized KT tool, to change EBP behavior, knowledge, and attitudes of AHPs.
This is an evaluator-blinded, cluster randomized controlled trial conducted in an Australian community-based cerebral palsy service. 135 AHPs (physiotherapists, occupational therapists, speech pathologists, psychologists and social workers) from four regions were cluster randomized (n = 4), to either the KT intervention group (n = 73 AHPs) or the control group (n = 62 AHPs), using computer-generated random numbers, concealed in opaque envelopes, by an independent officer. The KT intervention included three-day skills training workshop and multifaceted workplace supports to redress barriers (paid EBP time, mentoring, system changes and access to an online research synthesis tool). Primary outcome (self- and peer-rated EBP behavior) was measured using the Goal Attainment Scale (individual level). Secondary outcomes (knowledge and attitudes) were measured using exams and the Evidence Based Practice Attitude Scale.
The intervention group’s primary outcome scores improved relative to the control group, however when clustering was taken into account, the findings were non-significant: self-rated EBP behavior [effect size 4.97 (95% CI -10.47, 20.41) (p = 0.52)]; peer-rated EBP behavior [effect size 5.86 (95% CI -17.77, 29.50) (p = 0.62)]. Statistically significant improvements in EBP knowledge were detected [effect size 2.97 (95% CI 1.97, 3.97 (p < 0.0001)]. Change in EBP attitudes was not statistically significant.
Improvement in EBP behavior was not statistically significant after adjusting for cluster effect, however similar improvements from peer-ratings suggest behaviorally meaningful gains. The large variability in behavior observed between clusters suggests barrier assessments and subsequent KT interventions may need to target subgroups within an organization.
Trial registration
Registered on the Australian New Zealand Clinical Trials Registry (ACTRN12611000529943).
PMCID: PMC3831589  PMID: 24220660
KT; Allied health; Evidence-based practice; Online KT tool
15.  Approved Clinical Instructors' Perspectives on Implementation Strategies in Evidence-Based Practice for Athletic Training Students 
Journal of Athletic Training  2011;46(6):655-664.
Understanding implementation strategies of Approved Clinical Instructors (ACIs) who use evidence-based practice (EBP) in clinical instruction will help promote the use of EBP in clinical practice.
To examine the perspectives and experiences of ACIs using EBP concepts in undergraduate athletic training education programs to determine the importance of using these concepts in clinical practice, clinical EBP implementation strategies for students, and challenges of implementing EBP into clinical practice while mentoring and teaching their students.
Qualitative study.
Telephone interviews.
Patients or Other Participants:
Sixteen ACIs (11 men, 5 women; experience as a certified athletic trainer = 10 ± 4.7 years, experience as an ACI = 6.8 ± 3.9 years) were interviewed.
Data Collection and Analysis:
We interviewed each participant by telephone. Interview transcripts were analyzed and coded for common themes and subthemes regarding implementation strategies. Established themes were triangulated through peer review and member checking to verify the data.
The ACIs identified EBP implementation as important for validation of the profession, changing paradigm shift, improving patient care, and improving student educational experiences. They promoted 3 methods of implementing EBP concepts with their students: self-discovery, promoting critical thinking, and sharing information. They assisted students with the steps of EBP and often faced challenges in implementation of the first 3 steps of EBP: defining a clinical question, literature searching, and literature appraisal. Finally, ACIs indicated that modeling the behavior of making clinical decisions based on evidence was the best way to encourage students to continue using EBP.
Athletic training education program directors should encourage and recommend specific techniques for EBP implementation in the clinical setting. The ACIs believed that role modeling is a strategy that can be used to promote the use of EBP with students. Training of ACIs should include methods by which to address the steps of the EBP process while still promoting critical thinking.
PMCID: PMC3418944  PMID: 22488192
clinical education; role modeling; qualitative research
16.  Application of the principles of evidence-based practice in decision making among senior management in Nova Scotia’s addiction services agencies 
Competency profiles for occupational clusters within Canada’s substance abuse workforce (SAW) define the need for skill and knowledge in evidence-based practice (EBP) across all its members. Members of the Senior Management occupational cluster hold ultimate responsibility for decisions made within addiction services agencies and therefore must possess the highest level of proficiency in EBP. The objective of this study was to assess the knowledge of the principles of EBP, and use of the components of the evidence-based decision making (EBDM) process in members of this occupational cluster from selected addiction services agencies in Nova Scotia.
A convenience sampling method was used to recruit participants from addiction services agencies. Semi-structured qualitative interviews were conducted with eighteen Senior Management. The interviews were audio-recorded, transcribed verbatim and checked by the participants. Interview transcripts were coded and analyzed for themes using content analysis and assisted by qualitative data analysis software (NVivo 9.0).
Data analysis revealed four main themes: 1) Senior Management believe that addictions services agencies are evidence-based; 2) Consensus-based decision making is the norm; 3) Senior Management understand the principles of EBP and; 4) Senior Management do not themselves use all components of the EBDM process when making decisions, oftentimes delegating components of this process to decision support staff.
Senior Management possess an understanding of the principles of EBP, however, when making decisions they often delegate components of the EBDM process to decision support staff. Decision support staff are not defined as an occupational cluster in Canada’s SAW and have not been ascribed a competency profile. As such, there is no guarantee that this group possesses competency in EBDM. There is a need to advocate for the development of a defined occupational cluster and associated competency profile for this critical group.
Electronic supplementary material
The online version of this article (doi:10.1186/1747-597X-9-47) contains supplementary material, which is available to authorized users.
PMCID: PMC4320476  PMID: 25479733
Evidence-based decision-making; Evidence-based medicine; Substance abuse; Addiction; Treatment; Prevention; Senior Administrative Personnel; Qualitative research
17.  Survey of knowledge and perception on the access to evidence-based practice and clinical practice change among maternal and infant health practitioners in South East Asia 
Evidence-based practice (EBP) can provide appropriate care for women and their babies; however implementation of EBP requires health professionals to have access to knowledge, the ability to interpret health care information and then strategies to apply care. The aim of this survey was to assess current knowledge of evidence-based practice, information seeking practices, perceptions and potential enablers and barriers to clinical practice change among maternal and infant health practitioners in South East Asia.
Questionnaires about IT access for health information and evidence-based practice were administered during August to December 2005 to health care professionals working at the nine hospitals participating in the South East Asia Optimising Reproductive and Child Health in Developing countries (SEA-ORCHID) project in Indonesia, Malaysia, Thailand and The Philippines.
The survey was completed by 660 staff from six health professional groups. Overall, easy IT access for health care information was available to 46% of participants. However, over a fifth reported no IT access was available and over half of nurses and midwives never used IT health information. Evidence-based practice had been heard of by 58% but the majority did not understand the concept. The most frequent sites accessed were Google and PubMed. The Cochrane Library had been heard of by 47% of whom 51% had access although the majority did not use it or used it less than monthly. Only 27% had heard of the WHO Reproductive Health Library and 35% had been involved in a clinical practice change and were able to identify enablers and barriers to change. Only a third of participants had been actively involved in practice change with wide variation between the countries. Willingness to participate in professional development workshops on evidence-based practice was high.
This survey has identified the need to improve IT access to health care information and health professionals' knowledge of evidence-based health care to assist in employing evidence base practice effectively.
PMCID: PMC2533287  PMID: 18680603
18.  Development and validation of the competence in evidence based practice questionnaire (EBP-COQ) among nursing students 
BMC Medical Education  2013;13:19.
Nursing educators need rigorously developed instruments to assess competency in evidence based practice (EBP) at undergraduate level. This concept is defined as the capability to choose and use an integrated combination of knowledge, skills and attitudes with the intention to develop a task in a certain context. Also, we understand that EBP is gaining knowledge and skills, as well as increasing positive attitudes toward EBP that will promote a change in behaviour to implement EBP in practice. This study aims to develop a psychometric test of the Evidence Based Practice Evaluation Competence Questionnaire (EBP-COQ) among undergraduate nursing students.
The questionnaire was developed by item generation through a review of scientific literature and focus groups. The instrument was validated in terms of content validity through an expert review. The EBP-COQ was administered to a cohort of nursing students (n =100) to evaluate test reliability and select the best items. Psychometric properties of the final instrument were assessed in a sample of 261 nursing students.
The EBP-COQ consisted of 25 items. A factorial analysis grouped the items into the three categories that define competence relating to EBP: attitude, knowledge and skills. Cronbach’s alpha was 0.888 for the entire questionnaire. The factor solution explained 55.55% of the variance.
EBP-COQ appears to measure with adequate reliability the attributes of undergraduate nursing students’ competence in EBP. The instrument is quick to disseminate and easy to score, making it a suitable instrument for nursing educators to evaluate students’ self-perceived competence in EBP.
PMCID: PMC3598337  PMID: 23391040
Competence; Evidence-based practice; Undergraduate nursing; Questionnaire; Validation
19.  Towards evidence‐based medicine for paediatricians 
To give the best care to patients and families, paediatricians need to integrate the highest quality scientific evidence with clinical expertise and the opinions of the family.1Archimedes seeks to assist practising clinicians by providing “evidence‐based” answers to common questions that are not at the forefront of research but are at the core of practice. In doing this, we are adapting a format that has been successfully developed by Kevin Mackway‐Jones and the group at the Emergency Medicine Journal—“BestBets”.
A word of warning. The topic summaries are not systematic reviews, although they are as exhaustive as a practising clinician can produce. They make no attempt to statistically aggregate the data, nor to search the grey, unpublished literature. What Archimedes offers is practical, best evidence‐based answers to practical, clinical questions.
The format of Archimedes may be familiar. A description of the clinical setting is followed by a structured clinical question. (These aid in focusing the mind, assisting searching2 and obtaining answers.3) A brief report of the search used follows—this has been performed in a hierarchical way, to search for the best quality evidence to answer the question ( A table provides a summary of the evidence and key points of the critical appraisal. For further information on critical appraisal, and the measures of effect (such as the number needed to treat), books by Sackett4 and Moyer5 may help. To pull the information together, a commentary is provided, but to make it all much more accessible, a box provides the clinical bottom lines.
Electronics‐only topics that have been published on the BestBets site ( and may be of interest to paediatricians include the following.
Can steroids be used to reduce post tonsillectomy pain?
Readers wishing to submit their own questions—with best evidence answers—are encouraged to review those already proposed at If your question still hasn't been answered, feel free to submit your summary according to the instructions for authors at Three topics are covered in this issue of the journal:
Is teething the cause of minor ailments?
Should steroid creams be used in cases of labial fusion?
Does erythromycin cause pyloric stenosis?
1 Moyer VA, Ellior EJ. Preface. In: Moyer VA, Elliott EJ, Davis RL, et al, eds. Evidence based pediatrics and child health. Issue 1. London: BMJ Books, 2000.
2 Richardson WS, Wilson MC, Nishikawa J, et al. The well‐built clinical question: a key to evidence‐based decisions. ACP J Club 1995;123:A12–13.
3 Bergus GR, Randall CS, Sinift SD, et al. Does the structure of clinical questions affect the outcome of curbside consultations with specialty colleagues? Arch Fam Med 2000;9:541–7.
4 Sackett DL, Starus S, Richardson WS, et al. Evidence‐based medicine. How to practice and teach EBM. San Diego: Harcourt‐Brace, 2000.
5 Moyer VA, Elliott EJ, Davis RL, et al, eds. Evidence based pediatrics and child health. Issue 1. London: BMJ Books, 2000.
Can: doing, using and replicating evidence‐based child health
The practice of evidence‐based child health is said to be the five‐step way of asking questions, acquiring information, appraising the evidence, applying the results and assessing our performance.
If the truth be known, for the vast majority of the time, most of us perform our clinical practice replicating what we have done previously. Most of the time this is based on the combination of excellent education, skilled interpretation of clinical findings, and good discussions with children and families. We hope that the education we rely on was (and remains) based on the best available scientific evidence. If it is, we are practising a form of “micro‐evidence‐based healthcare (EBHC)” (doing just step 4).
Sometimes, we question our knowledge (or more uncomfortably, someone does this for us), and will head off to top up our understanding of an area. This “using” mode, if we use well‐appraised resources to supply our thirst for information, will also promote the practice of evidence‐based care. This midi‐EBHC asks us to go through steps 1, 2 and 4.
Occasionally, we also actually need to go through the entire process of getting “down and dirty” with the primary research and appraising it to influence our practice. Maxi‐EBHC is considerably more demanding in time, but largely more satisfying intellectually.
If we reframe the practice of EBHC as using the family and child values, the best evidence, and our clinical expertise, then we can do it by micro‐methods, midi‐methods or maxi‐methods, and choose the most appropriate approach for the situation we confront.
I thank Dr Sharon Straus, Director of the Center for Evidence‐based Medicine, University of Toronto, Toronto, Ontario, Canada.
PMCID: PMC2083440
20.  A survey of Australian chiropractors’ attitudes and beliefs about evidence-based practice and their use of research literature and clinical practice guidelines 
Research into chiropractors’ use of evidence in clinical practice appears limited to a single small qualitative study. The paucity of research in this area suggests that it is timely to undertake a more extensive study to build a more detailed understanding of the factors that influence chiropractors’ adoption of evidence-based practice (EBP) principles. This study aimed to identify Australian chiropractors’ attitudes and beliefs towards EBP in clinical practice, and also examine their use of research literature and clinical practice guidelines.
We used an online questionnaire about attitudes, beliefs and behaviours towards the use of EBP in clinical practice that had been developed to survey physiotherapists and modified it to ensure that it was relevant to chiropractic practice. We endeavoured to survey all registered Australian chiropractors (n = 4378) via email invitation distributed by Australian chiropractic professional organisations and the Chiropractic Board of Australia. Logistic regression analyses were conducted to examine univariate associations between responses to items measuring attitudes and beliefs with items measuring: age; years since registration; attention to literature; and use of clinical practice guidelines.
Questionnaires were returned by 584 respondents (response rate approximately 13%). The respondents’ perceptions of EBP were generally positive: most agreed that the application of EBP is necessary (77.9%), literature and research findings are useful (80.2%), EBP helps them make decisions about patient care (66.5%), and expressed an interest in learning or improving EBP skills (74.9%). Almost half of the respondents (45.1%) read between two to five articles a month. Close to half of the respondents (44.7%) used literature in the process of clinical decision making two to five times each month. About half of the respondents (52.4%) agreed that they used clinical practice guidelines, and around half (54.4%) agreed that they were able to incorporate patient preferences with clinical practice guidelines. The most common factor associated with increased research uptake was the perception that EBP helps make decisions about patient care.
Most Australian chiropractors hold positive attitudes towards EBP, thought EBP was useful, and were interested in improving EBP skills. However, despite the favourable inclination towards EBP, many Australian chiropractors did not use clinical practice guidelines. Our findings should be interpreted cautiously due to the low response rate.
PMCID: PMC3878410  PMID: 24345082
21.  Does journal club membership improve research evidence uptake in different allied health disciplines: a pre-post study 
BMC Research Notes  2012;5:588.
Although allied health is considered to be one 'unit' of healthcare providers, it comprises a range of disciplines which have different training and ways of thinking, and different tasks and methods of patient care. Very few empirical studies on evidence-based practice (EBP) have directly compared allied health professionals. The objective of this study was to examine the impact of a structured model of journal club (JC), known as iCAHE (International Centre for Allied Health Evidence) JC, on the EBP knowledge, skills and behaviour of the different allied health disciplines.
A pilot, pre-post study design using maximum variation sampling was undertaken. Recruitment was conducted in groups and practitioners such as physiotherapists, occupational therapists, speech pathologists, social workers, psychologists, nutritionists/dieticians and podiatrists were invited to participate. All participating groups received the iCAHE JC for six months. Quantitative data using the Adapted Fresno Test (McCluskey & Bishop) and Evidence-based Practice Questionnaire (Upton & Upton) were collected prior to the implementation of the JC, with follow-up measurements six months later. Mean percentage change and confidence intervals were calculated to compare baseline and post JC scores for all outcome measures.
The results of this study demonstrate variability in EBP outcomes across disciplines after receiving the iCAHE JC. Only physiotherapists showed statistically significant improvements in all outcomes; speech pathologists and occupational therapists demonstrated a statistically significant increase in knowledge but not for attitude and evidence uptake; social workers and dieticians/nutritionists showed statistically significant positive changes in their knowledge, and evidence uptake but not for attitude.
There is evidence to suggest that a JC such as the iCAHE model is an effective method for improving the EBP knowledge and skills of allied health practitioners. It may be used as a single intervention to facilitate evidence uptake in some allied health disciplines but may need to be integrated with other strategies to influence practice behaviour in other practitioners. An in-depth analysis of other factors (e.g. individual, contextual, organisational), or the relative contribution of these variables is required to better understand the determinants of evidence uptake in allied health.
PMCID: PMC3517354  PMID: 23106851
Journal club; Evidence-based practice; Allied health; Evidence uptake
22.  Adopting evidence-based practice in clinical decision making: nurses' perceptions, knowledge, and barriers 
Evidence-based practice (EBP) provides nurses with a method to use critically appraised and scientifically proven evidence for delivering quality health care to a specific population. The objective of this study was to explore nurses' awareness of, knowledge of, and attitude toward EBP and factors likely to encourage or create barriers to adoption. In addition, information sources used by nurses and their literature searching skills were also investigated.
A total of 2,100 copies of the questionnaire were distributed to registered nurses in 2 public hospitals in Singapore, and 1,486 completed forms were returned, resulting in a response rate of 70.8%.
More than 64% of the nurses expressed a positive attitude toward EBP. However, they pointed out that due to heavy workload, they cannot keep up to date with new evidence. Regarding self-efficacy of EBP-related abilities, the nurses perceived themselves to possess moderate levels of skills. The nurses also felt that EBP training, time availability, and mentoring by nurses with EBP experience would encourage them to implement EBP. The top three barriers to adopting EBP were lack of time, inability to understand statistical terms, and inadequate understanding of the jargon used in research articles. For literature searching, nurses were using basic search features and less than one-quarter of them were familiar with Boolean and proximity operators.
Although nurses showed a positive attitude toward EBP, certain barriers were hindering their smooth adoption. It is, therefore, desirable that hospital management in Southeast Asia, particularly in Singapore, develop a comprehensive strategy for building EBP competencies through proper training. Moreover, hospital libraries should also play an active role in developing adequate information literacy skills among the nurses.
PMCID: PMC3133901  PMID: 21753915
23.  Framework of policy recommendations for implementation of evidence-based practice: a systematic scoping review 
BMJ Open  2013;3(1):e001881.
Evidence-based practice (EBP) may help improve healthcare quality. However, not all healthcare professionals and managers use EBP in their daily practice. We systematically reviewed the literature to summarise self-reported appreciation of EBP and organisational infrastructure solutions proposed to promote EBP.
Systematic review. Two investigators independently performed the systematic reviewing process.
Information sources
MEDLINE, EMBASE and Cochrane Library were searched for publications between 2000 and 2011.
Eligibility criteria for included studies
Reviews and surveys of EBP attitude, knowledge, awareness, skills, barriers and facilitators among managers, doctors and nurses in clinical settings.
We found 31 surveys of fairly good quality. General attitude towards EBP was welcoming. Respondents perceived several barriers, but also many facilitators for EBP implementation. Solutions were proposed at various organisational levels, including (inter)national associations and hospital management promoting EBP, pregraduate and postgraduate education, as well as individual support by EBP mentors on the wards to move EBP from the classroom to the bedside.
More than 20 years after its introduction, the EBP paradigm has been embraced by healthcare professionals as an important means to improve quality of patient care, but its implementation is still deficient. Policy exerted at microlevel , middlelevel and macrolevel, and supported by professional, educational and managerial role models, may further facilitate EBP.
PMCID: PMC3563143  PMID: 23355664
Medical Education & Training
24.  Professional Uncertainty and Disempowerment Responding to Ethnic Diversity in Health Care: A Qualitative Study 
PLoS Medicine  2007;4(11):e323.
While ethnic disparities in health and health care are increasing, evidence on how to enhance quality of care and reduce inequalities remains limited. Despite growth in the scope and application of guidelines on “cultural competence,” remarkably little is known about how practising health professionals experience and perceive their work with patients from diverse ethnic communities. Using cancer care as a clinical context, we aimed to explore this with a range of health professionals to inform interventions to enhance quality of care.
Methods and Findings
We conducted a qualitative study involving 18 focus groups with a purposeful sample of 106 health professionals of differing disciplines, in primary and secondary care settings, working with patient populations of varying ethnic diversity in the Midlands of the UK. Data were analysed by constant comparison and we undertook processes for validation of analysis. We found that, as they sought to offer appropriate care, health professionals wrestled with considerable uncertainty and apprehension in responding to the needs of patients of ethnicities different from their own. They emphasised their perceived ignorance about cultural difference and were anxious about being culturally inappropriate, causing affront, or appearing discriminatory or racist. Professionals' ability to think and act flexibly or creatively faltered. Although trying to do their best, professionals' uncertainty was disempowering, creating a disabling hesitancy and inertia in their practice. Most professionals sought and applied a knowledge-based cultural expertise approach to patients, though some identified the risk of engendering stereotypical expectations of patients. Professionals' uncertainty and disempowerment had the potential to perpetuate each other, to the detriment of patient care.
This study suggests potential mechanisms by which health professionals may inadvertently contribute to ethnic disparities in health care. It identifies critical opportunities to empower health professionals to respond more effectively. Interventions should help professionals acknowledge their uncertainty and its potential to create inertia in their practice. A shift away from a cultural expertise model toward a greater focus on each patient as an individual may help.
From a qualitative study, Joe Kai and colleagues have identified opportunities to empower health professionals to respond more effectively to challenges in their work with patients from diverse ethnic communities.
Editors' Summary
Communities are increasingly diverse in terms of ethnicity (belonging to a group of people defined by social characteristics such as cultural tradition or national origin) and race (belonging to a group identified by inherited physical characteristics). Although health professionals and governments are striving to ensure that everybody has the same access to health care, there is increasing evidence of ethnic inequalities in health-care outcomes. Some of these inequalities reflect intrinsic differences between groups of people—Ashkenazi Jews, for example, often carry an altered gene that increases their chance of developing aggressive breast cancer. Often, however, these differences reflect inequalities in the health care received by different ethnic groups. To improve this situation, “cultural competence” has been promoted over recent years. Cultural competence is the development of skills by individuals and organizations that allow them to work effectively with people from different cultures. Health professionals are now taught about ethnic differences in health beliefs and practices, religion, and communication styles to help them provide the best service to all their patients.
Why Was This Study Done?
Numerous guidelines aim to improve cultural competency but little is known about how health professionals experience and perceive their work with patients from diverse ethnic groups. Is their behavior influenced by ethnicity in ways that might contribute to health care disparities? For example, do doctors sometimes avoid medical examinations for fear of causing offence because of cultural differences? If more were known about how health professionals handle ethnic diversity (a term used here to include both ethnicity and race) it might be possible to reduce ethnic inequalities in health care. In this qualitative study, the researchers have explored how health professionals involved in cancer care are affected by working with ethnically diverse patients. A qualitative study is one that collects nonquantitative data such as how doctors “feel” about treating people of different ethnic backgrounds; a quantitative study might compare clinical outcomes in different ethnic groups.
What Did the Researchers Do and Find?
The researchers enrolled 106 doctors, nurses, and other health-related professionals from different health-service settings in the Midlands, an ethnically diverse region of the UK. They organized 18 focus groups in which the health professionals described their experiences of caring for people from ethnic minority backgrounds. The participants were encouraged to recall actual cases and to identify what they saw as problems and strengths in their interactions with these patients. The researchers found that the health professionals wrestled with many challenges when providing health care for patients from diverse ethnic backgrounds. These challenges included problems with language and with general communication (for example, deciding when it was acceptable to touch a patient to show empathy). Health professionals also worried they did not know enough about cultural differences. As a result, they said they often felt uncertain of their ability to avoid causing affront or appearing racist. This uncertainty, the researchers report, disempowered the health professionals, sometimes making them hesitate or fail to do what was best for their patient.
What Do These Findings Mean?
These findings reveal that health professionals often experience considerable uncertainty when caring for ethnically diverse patients, even after training in cultural competency. They also show that this uncertainty can lead to hesitancy and inertia, which might contribute to ethnic health care inequalities. Because the study participants were probably already interested in ethnic diversity and health care, interviews with other health professionals (and investigations of patient experiences) are needed to confirm these findings. Nevertheless, the researchers suggest several interventions that might reduce health care inequalities caused by ethnic diversity. For example, health professionals should be encouraged to recognize their uncertainty and should have access to more information and training about ethnic differences. In addition, there should be a shift in emphasis away from relying on knowledge-based cultural information towards taking an “ethnographic” approach. In other words, health professionals should be helped to feel able to ask their patients about what matters most to them as individuals about their illness and treatment.
Additional Information.
Please access these Web sites via the online version of this summary at
Information on cultural competence and health care is available from the US National Center for Cultural Competence (in English and Spanish) and DiversityRx
PROCEED (Professionals Responding to Cancer in Ethnic Diversity) is a multimedia training tool for educators within the health and allied professions developed from the results of this study; a press release on PROCEED is available from the University of Nottingham
Transcultural Health Care Practice: An educational resource for nurses and health care practitioners is available on the web site of the UK Royal College of Nursing
PMCID: PMC2071935  PMID: 18001148
25.  Institutionalizing evidence-based practice: an organizational case study using a model of strategic change 
There is a general expectation within healthcare that organizations should use evidence-based practice (EBP) as an approach to improving the quality of care. However, challenges exist regarding how to make EBP a reality, particularly at an organizational level and as a routine, sustained aspect of professional practice.
A mixed method explanatory case study was conducted to study context; i.e., in terms of the presence or absence of multiple, inter-related contextual elements and associated strategic approaches required for integrated, routine use of EBP ('institutionalization'). The Pettigrew et al. Content, Context, and Process model was used as the theoretical framework. Two sites in the US were purposively sampled to provide contrasting cases: i.e., a 'role model' site, widely recognized as demonstrating capacity to successfully implement and sustain EBP to a greater degree than others; and a 'beginner' site, self-perceived as early in the journey towards institutionalization.
The two sites were clearly different in terms of their organizational context, level of EBP activity, and degree of institutionalization. For example, the role model site had a pervasive, integrated presence of EBP versus a sporadic, isolated presence in the beginner site. Within the inner context of the role model site, there was also a combination of the Pettigrew and colleagues' receptive elements that, together, appeared to enhance its ability to effectively implement EBP-related change at multiple levels. In contrast, the beginner site, which had been involved for a few years in EBP-related efforts, had primarily non-receptive conditions in several contextual elements and a fairly low overall level of EBP receptivity. The beginner site thus appeared, at the time of data collection, to lack an integrated context to either support or facilitate the institutionalization of EBP.
Our findings provide evidence of some of the key contextual elements that may require attention if institutionalization of EBP is to be realized. They also suggest the need for an integrated set of receptive contextual elements to achieve EBP institutionalization; and they further support the importance of specific interactions among these elements, including ways in which leadership affects other contextual elements positively or negatively.
PMCID: PMC2795741  PMID: 19948064

Results 1-25 (1001181)