Objective. To assess course instructors’ and students’ perceptions of the Educating Pharmacy Students and Pharmacists to Improve Quality (EPIQ) curriculum.
Methods. Seven colleges and schools of pharmacy that were using the EPIQ program in their curricula agreed to participate in the study. Five of the 7 collected student retrospective pre- and post-intervention questionnaires. Changes in students’ perceptions were evaluated to assess their relationships with demographics and course variables. Instructors who implemented the EPIQ program at each of the 7 colleges and schools were also asked to complete a questionnaire.
Results. Scores on all questionnaire items indicated improvement in students’ perceived knowledge of quality improvement. The university the students attended, completion of a class project, and length of coverage of material were significantly related to improvement in the students’ scores. Instructors at all colleges and schools felt the EPIQ curriculum was a strong program that fulfilled the criteria for quality improvement and medication error reduction education.
Conclusion The EPIQ program is a viable, turnkey option for colleges and schools of pharmacy to use in teaching students about quality improvement.
quality improvement; medication error; pharmacy education; pharmacy student; assessment; curriculum
To compare student and faculty perceptions of the delivery and achievement of professional competencies in a doctor of pharmacy program in order to provide data for both accountability and curricular improvement purposes.
A survey instrument was designed based on current learning theory, and 76 specific competency statements generated from mission and goal statements of The Ohio State University College of Pharmacy and the Center for the Advancement of Pharmaceutical Education. This instrument was administered to PharmD program students and faculty.
The number of competencies by program year that are delivered in the curriculum, the percent of students and faculty reporting individual competency delivery and achievement, and differences between student and faculty perceptions of competency delivery and achievement are reported.
The faculty and student opinions provided an in-depth view of curricular outcomes. Gathering perception data from faculty and students about the delivery and achievement of competencies in a PharmD program can be used to both meet accreditation requirements (accountability) and to improve the curriculum (improvement).
outcomes assessment; assessment; survey research; curriculum reform; competency
Objective. To assess pharmacy faculty trainers’ perceptions of a Web-based train-the-trainer program for PharmGenEd, a shared pharmacogenomics curriculum for health professional students and licensed clinicians.
Methods. Pharmacy faculty trainers (n=58, representing 39 colleges and schools of pharmacy in the United States and 1 school from Canada) participated in a train-the-trainer program consisting of up to 9 pharmacogenomics topics. Posttraining survey instruments assessed faculty trainers’ perceptions toward the training program and the likelihood of their adopting the educational materials as part of their institution’s curriculum.
Results. Fifty-five percent of faculty trainers reported no prior formal training in pharmacogenomics. There was a significant increase (p<0.001) in self-reported ability to teach pharmacogenomics to pharmacy students after participants viewed the webinar and obtained educational materials. Nearly two-thirds (64%) indicated at least a “good” likelihood of adopting PharmGenEd materials at their institution during the upcoming academic year. More than two-thirds of respondents indicated interest in using PharmGenEd materials to train licensed health professionals, and 95% indicated that they would recommend the program to other pharmacy faculty members.
Conclusion. As a result of participating in the train-the-trainer program in pharmacogenomics, faculty member participants gained confidence in teaching pharmacogenomics to their students, and the majority of participants indicated a high likelihood of adopting the program at their institution. A Web-based train-the-trainer model appears to be a feasible strategy for training pharmacy faculty in pharmacogenomics.
pharmacogenomics; curriculum; pharmacy colleges and schools; faculty development; train-the-trainer
To examine the validity and reliability of a web-based version of the Epidemiology of Prolapse and Incontinence Questionnaire (EPIQ).
Participants included 876 women ages 38 to 65 attending primary care clinics in the Salt Lake Valley. Women completed a single web or paper based version of the symptom bother questions from EPIQ, and a subset repeated the same or opposite method at 2 separate time points. To assess subscales for the web-based version factor analysis of the 22 EPIQ items related to pelvic floor disorder (PFD) symptoms was performed using principal components analysis and varimax rotation. Internal consistency was assessed using coefficient alpha. Test-retest and inter-method reliability were assessed using intraclass correlation coefficients (ICC) for domain scores. Correlations above 0.70 were considered acceptable.
Overall, 384 and 492 women completed at least 1 web and 1 paper EPIQ and 93% were Caucasian with mean age of 50±7 years. Of these, 63 completed web-web, 57 web-paper, 47 paper-web and 109 paper-paper test-retest. Overall, factor analyses were consistent with the 7 domains of the original EPIQ. Cronbach’s alpha for the 4 symptomatic PFD domains and range of test-retest reliability for the various administration methods were similar to the original EPIQ instrument. Correlations for domain scores were above 0.70, except the anal incontinence scale (ICC=0.68.)
Web administration of the EPIQ has similar psychometric properties with comparable internal consistency and test-retest reliability when administered in the same modality. Reliability between both methods of administration is acceptable.
Epidemiology; internet survey; pelvic floor disorders; questionnaire; web
To engage pharmacy students at the McWhorter School of Pharmacy in an authentic discussion of professionalism early in their education.
A booklet was prepared that included several classic short stories and essays that dealt with professionalism. This booklet was sent to all entering students in the class of 2008 and 2009 during the summer prior to their first-professional year of the PharmD program. The stories and essays were discussed in small groups with faculty facilitation during orientation when the students first arrived on campus. A survey instrument was created and administered to assess the impact of this innovative approach to enhancing professionalism.
The program was well received and engaged our pharmacy students in a productive discussion on professionalism. Both classes' mean scores on survey items related that the students were engaged in the discussion of professionalism. Survey results pertaining to professional behavior also indicated increased awareness of the importance of professionalism.
Enhancing professionalism requires a culture change that necessitates addressing professionalism at its core, a calling to serve, in a persistent and continual manner. Requiring students to read and think about professionalism in a novel way, before they even begin their first-professional year of pharmacy school, appears to be an effective approach to nurturing/encouraging professionalism.
professionalism; literature; humanities; vocation
In 2013, the Director’s Forum published our assessment of issues facing pharmacy leaders to assist pharmacy directors in planning for the year ahead. The issues include health care reform and the Affordable Care Act, the American Society of Health-System Pharmacists Pharmacy Practice Model Initiative, the health care workforce, patients’ perceptions of pharmacists, and the changing landscape of pharmacy education. Based on our environmental scan, the issues addressed in 2013 are pertinent to a department’s plan for 2014. The goal of this article is to provide practical approaches to each of these issues to help pharmacy directors focus their department’s goals for 2014 to support the development of patient-centered pharmacy services. This column will address (1) strategies to reduce medication costs and generate new pharmacy revenue streams, (2) innovative approaches to improving medication safety and quality, (3) steps to advance the clinical practice model, and (4) ways to create mutually beneficial student experiences.
Objectives. To describe the development and validation of an instrument designed to assess student perceptions of physician-pharmacist interprofessional clinical education (SPICE).
Methods. Faculty members from pharmacy and medical schools developed items for the instrument, and 179 medical and pharmacy students completed the scale. Psychometric properties, including reliability and construct validity, were assessed using confirmatory factor analysis.
Results. The final instrument consisted of 10 items with 3 subscales measuring student perceptions of interprofessional teamwork and team-based practice, roles/responsibilities for collaborative practice, and patient outcomes from collaborative practice. Validity and reliability of the instrument were demonstrated.
Conclusion. The SPICE instrument demonstrated promise as a valid and reliable measure of pharmacy and medical student perceptions of interprofessional clinical education. SPICE may serve as a useful instrument for educational researchers in assessing the impact of interprofessional educational experiences.
interprofessional education; interdisciplinary education; instrument validation; confirmatory factor analysis
This paper provides baseline information on integrating the science of safety into the professional degree curriculum at colleges and schools of pharmacy. A multi-method examination was conducted that included a literature review, key informant interviews of 30 individuals, and in-depth case studies of 5 colleges and schools of pharmacy. Educators believe that they are devoting adequate time to science of safety topics and doing a good job teaching students to identify, understand, report, manage, and communicate medication risk. Areas perceived to be in need of improvement include educating pharmacy students about the Food and Drug Administration's (FDA's) role in product safety, how to work with the FDA in post-marketing surveillance and other FDA safety initiatives, teaching students methods to improve safety, and educating students to practice in interprofessional teams. The report makes 10 recommendations to help pharmacy school graduates be more effective in protecting patients from preventable drug-related problems.
safety; curriculum; pharmacy education; FDA; quality
Formal guidelines for mentoring faculty members in pharmacy practice divisions of colleges and schools of pharmacy do not exist in the literature. This paper addresses the background literature on mentoring programs, explores the current state of mentoring programs used in pharmacy practice departments, and provides guidelines for colleges and schools instituting formal mentoring programs. As the number of pharmacy colleges and schools has grown, the demand for quality pharmacy faculty members has dramatically increased. While some faculty members gain teaching experience during postgraduate residency training, new pharmacy practice faculty members often need professional development to meet the demands of their academic responsibilities. A mentoring program can be 1 means of improving faculty success and retention. Many US colleges and schools of pharmacy have developed formal mentoring programs, whereas several others have informal processes in place. This paper discusses those programs and the literature available, and makes recommendations on the structure of mentoring programs.
mentoring; faculty development; mentor; pharmacy practice; faculty
To assess the train-the-trainer component of an initiative (Partners in D) to train pharmacy students to facilitate patient enrollment in the best Medicare Part D prescription drug plan (Part D).
Faculty members from 6 California colleges or schools of pharmacy were taught how to train pharmacy students about Medicare Part D and how to conduct outreach events targeting underserved patient populations. A preintervention and postintervention survey instrument was administered to determine participants' (1) knowledge of the Part D program; (2) skill using the Medicare Prescription Drug Plan Finder tool; and (3) confidence in their ability to train pharmacy students. Implementation of the Partners in D curriculum in faculty members' colleges or schools of pharmacy was also determined.
Participants' knowledge of Part D, mastery of the Plan Finder, and confidence in teaching the material to pharmacy students all significantly improved. Within 8 weeks following the program, 5 of 6 colleges or schools of pharmacy adopted Partners in D coursework and initiated teaching the Partners-in-D curriculum. Four months afterwards, 21 outreach events reaching 186 Medicare beneficiaries had been completed.
The train-the-trainer component of the Partners in D program is practical and effective, and merits serious consideration as a national model for educating patients about Medicare Part D.
Medicare Part D; Medicare Prescription Drug Plan Finder; train-the-trainer; faculty development
In France, to practice as a pharmacist, one needs a “diplome d'état de Docteur en Pharmacie” This degree is awarded after 6 or 9 years of pharmacy studies, depending on the option chosen by the student. The degree is offered only at universities and is recognized in France as well as throughout the European Union.
Each university in France is divided into faculties called Unité de Formation et de Recherche (UFR). There are 24 faculties of pharmacy or UFRs de pharmacie. A national committee develops a pharmacy education program at the national level and each faculty adapts this program according to its specific features and means (eg, faculty, buildings). The number of students accepted in the second year is determined each year by a Government decree (numerus clausus).
Successive placements, totalling 62 weeks, progressively familiarize the student with professional practice, and enable him/her to acquire the required competencies, such as drug monitoring and educating and counselling patients. Challenges facing community pharmacies in the next 10 years are patient education, home health care, and orthopaedics; in hospital pharmacies, empowering pharmacists to supervise and validate all prescriptions; and finally, research in pharmacy practice.
international pharmacy education; France
To improve pharmacy education through integrating theory and practice, coherent constructively aligned course entities, and enhanced deep-level learning.
The reform was conducted collaboratively with faculty and staff members, students, and stakeholders in pharmacy. The curriculum, syllabus, and teaching methods were assessed through evaluations and research, conducting core content analyses, and measuring the workload of pharmacy education courses. The new curriculum, launched in August 2005, consists of 6 strands, comprised of different courses which run through the entire program.
Three years after the introduction of the reformed curriculum, the results of the reform are being evaluated. Ongoing assessments of teaching and learning will reveal how the education at the faculty level has developed since the reform. These assessment procedures are an integral part of the faculty's quality assurance program.
The integration of practical training and theoretical studies was improved with personal study plans introduced to enhance students’ learning.
curriculum; pharmacy education; Bologna process; Finland; curriculum reform
To implement a medication education project and assess the competencies students learned and implemented in professional practice after graduation.
Fourth-year pharmacy students planned, carried out, and reported on a real-life project during 1 study year. Outside experts and 2 faculty members facilitated the work. The aim of the medication education project was to create material that schoolteachers could use to teach children about rational use of medicines.
All students who had participated in the medication education program during its 3 years were contacted (n = 31). A questionnaire was sent to the 21 students who had graduated (18 responded), and a focus group was conducted with the 10 students completing their final year of pharmacy school (9 participants). The competencies that the students reported learning most were teamwork and social interaction skills. They considered the project motivating but also found it challenging and the deadlines frustrating.
Through participation in a medication education project, students learned interpersonal skills, time management, conflict resolution, and other skills that many of them already were finding valuable in their professional practice.
competencies; project-based learning; team; assessment; social science
Pharmacy education programs in Vietnam are complex and offer various career pathways. All include theory and laboratory modules in general, foundation, and pharmaceutical knowledge; placements in health facilities; and a final examination. The various pharmacy degree programs allow specialization in 1 or more of 5 main fields: (1) drug management and supply, (2) drug development and production, (3) pharmacology and clinical pharmacy, (4) traditional medicine and pharmacognosy, and (5) drug quality control, which are offered as main specialization options during the reformed undergraduate and postgraduate programs. However, pharmacy education in Vietnam in general remains product oriented and clinical pharmacy training has not received adequate attention. Only students who have obtained the bachelor of pharmacy degree, which requires a minimum of 5 years of study, are considered as fully qualified pharmacists. In contrast, an elementary diploma in pharmacy awarded after 1 year of pharmacy study permits entry into more junior pharmacy positions. Since the 2000s, there has been a surge in the number and types of schools offering pharmacy qualifications at various levels.
pharmacy education; pharmacy practice; Vietnam
To assess prepharmacy students' perceptions of the professional role of pharmacists prior to enrollment in pharmacy school, and the association between perceptions and student demographics.
A 58-question survey instrument regarding pharmacists' roles, work experiences, and demographics was developed and administered to students (N = 127) enrolled in an organic chemistry laboratory experience at Purdue University.
Theory of planned behavior subscales (attitude, subjective norm, perceived behavioral control) were influenced by students' grade point average, gender, and application to pharmacy school, while unpaid work experience affected professional commitment. Students evaluated work experience related to their pharmacy studies more positively than non-pharmacy-related areas in the theory of planned behavior subscales.
Evaluating students' perceptions may be beneficial in helping pharmacy educators design their curricula, as well as allowing admissions committees to select the most qualified students to promote the development of positive perceptions toward the professional role of pharmacists. Grade point average (GPA) and application to pharmacy school were associated with significant differences for the theory of planned behavior and professional commitment subscales.
Perceptions; professional role; theory of planned behavior; prepharmacy; pharmacist
Objectives. To determine projected growth in pharmacy education and research from 2010 to 2015 and to relate findings to external and internal factors.
Methods. An e-mail survey instrument was sent to all US pharmacy deans, and responses were used to estimate growth in the number of first-professional-degree doctor of pharmacy (PharmD) graduates, residents/fellows, graduate students, faculty members, graduate research faculty members, and postdoctoral fellows. Results were related to the national economy, trends in faculty vacancies, growth trends in other health professions, pharmacist roles, and healthcare reform.
Results. Five-year growth projections were: 58% increase in the number of residents/fellows, 23% in postdoctoral fellows, 21% in entry-level PharmD graduates, 19% in graduate/research faculty members, 17% in graduate students, and 13% in total pharmacy faculty members. Residencies/fellowships showed the highest projected growth rates (58%). Graduate education and research data suggest a growing research enterprise. Faculty vacancy trends were downward and this suggests better faculty availability in coming years.
Conclusions. Substantial growth is expected from 2010 to 2015 in all areas of pharmacy education. External factors and how well the profession is able to demonstrate its contribution to resolving healthcare problems may influence the actual growth rates achieved.
pharmacy education; pharmacy faculty members; residents; fellows; graduate students; growth; research
Objective. To determine the perceptions of junior pharmacy faculty members with US doctor of pharmacy (PharmD) degrees regarding their exposure to residency, fellowship, and graduate school training options in pharmacy school. Perceptions of exposure to career options and research were also sought.
Methods. A mixed-mode survey instrument was developed and sent to assistant professors at US colleges and schools of pharmacy.
Results. Usable responses were received from 735 pharmacy faculty members. Faculty members perceived decreased exposure to and awareness of fellowship and graduate education training as compared to residency training. Awareness of and exposure to academic careers and research-related fields was low from a faculty recruitment perspective.
Conclusions. Ensuring adequate exposure of pharmacy students to career paths and postgraduate training opportunities could increase the number of PharmD graduates who choose academic careers or other pharmacy careers resulting from postgraduate training.
pharmacy faculty members; residency programs; fellowships; graduate education; careers
Academic entitlement and student consumerism have been described as a cause for unprofessional behavior in higher education. Colleges and schools of pharmacy may inadvertently encourage student consumerism and academic entitlement by misunderstanding who is the primary customer of pharmacy education. Pharmacy colleges and schools who view students as the primary customer can unintentionally pressure faculty members to relax expectations for professionalism and academic performance and thereby cause a general downward spiral in the quality of pharmacy graduates. In contrast, this paper argues that the primary customer of pharmacy education is the patient. Placing the patient at the center of the educational process is consistent with the concepts of pharmaceutical care, medication therapy management, the patient-centered home, and the oath of the pharmacist. Emphasizing the patient as the primary customer discourages academic entitlement and student consumerism and encourages an emphasis on learning how to serve the medication-related needs of the patient.
academic entitlement; pharmacy students; student consumerism; higher education; pharmacy
The lessons learned from a collaboration between a faculty of pharmacy and a practice site that involved implementation of an innovative experiential placement model are described, as well as the broader impact of the project on other practice sites, the faculty of pharmacy’s experiential education program, and experiential placement capacity. The partnerships and collaborative strategies formed were key to the implementation and evaluation of a pharmacy student clinical teaching unit pilot program and integration of concepts used in the unit into the advanced pharmacy practice experience (APPE) program to enhance capacity and quality. The university-practice partnerships have made it possible to promote the delegation of responsibility and accountability for patient care to students, challenge the anticipated workload burden for preceptors, question the optimal length of an APPE placement, and highlight the value of higher student-to-preceptor ratios that facilitate peer-assisted learning (PAL) and optimize the practice learning experiences for preceptors and students. Collaboration in experiential education between universities and practice sites can provide opportunities to address challenges faced by practitioners and academics alike.
collaboration; peer assisted learning; pharmacy; capacity; experiential education
To determine the content and extent, design, and relative importance of patient assessment courses in the professional pharmacy curriculum.
A 20-item questionnaire was developed to gather information pertaining to patient assessment. Pharmacy practice department chairs were mailed a letter with an Internet link to an online survey instrument.
Ninety-six percent of the programs indicated that patient assessment skills were taught. Forty-five percent of respondents indicated their course was a standalone course. The most common topics covered in assessment courses were pulmonary examination, vital signs, and cardiovascular assessment.
There is significant variability in the topics covered, depth of content, types of instruction, and evaluation methods used in patient assessment courses in US colleges of pharmacy. This survey was an initial assessment of what is being done regarding education of student pharmacists on patient assessment.
curriculum design; laboratory instruction; patient assessment; physical assessment
Objective. To examine how hidden and informal curricula shaped pharmacy students’ learning about patient safety.
Methods. A preliminary study exploring planned patient safety content in pharmacy curricula at 3 UK schools of pharmacy was conducted. In-depth case studies were then carried out at 2 schools of pharmacy to examine patient safety education as delivered.
Results. Informal learning from teaching practitioners was assigned high levels of credibility by the students, indicating the importance of role models in practice. Students felt that the hidden lessons received in the form of voluntary work experience compensated for limited practice exposure and elements of patient safety not adequately addressed in the formal curriculum, such as learning about safe systems, errors, and professionalism.
Conclusions. Patient safety is a multifaceted concept and the findings from this study highlight the importance of pharmacy students learning in a variety of settings to gain an appreciation of these different facets.
patient safety; curriculum; pharmacy education
Children warrant access to care from clinical pharmacists trained in pediatrics. The American College of Clinical Pharmacy Pediatrics Practice and Research Network (ACCP Pediatrics PRN) released an opinion paper in 2005 with recommendations for improving the quality and quantity of pediatric pharmacy education in colleges of pharmacy, residency programs, and fellowships. While progress has been made in increasing the availability of pediatric residencies, there is still much to be done to meet the direct care needs of pediatric patients. The purpose of this Joint Opinion paper is to outline strategies and recommendations for expanding the quality and capacity of pediatric clinical pharmacy practitioners by 1) elevating the minimum expectations for pharmacists entering practice to provide pediatric care; 2) standardizing pediatric pharmacy education; 3) expanding the current number of pediatric clinical pharmacists; and 4) creating an infrastructure for development of pediatric clinical pharmacists and clinical scientists. These recommendations may be used to provide both a conceptual framework and action items for schools of pharmacy, health care systems, and policymakers to work together to increase the quality and quantity of pediatric training, practice, or research initiatives.
fellowship; pediatric pharmacy; pharmacy education; standardization; residency
Objectives. To develop and pilot test a multiple mini-interview (MMI) to select students for admission to a pharmacy degree program.
Methods. A nominal group process was used to identify 8 important nonacademic attributes of pharmacists, with relative importance determined by means of a paired-comparison survey of pharmacy stakeholders (ie, university-affiliated individuals with a vested interest in the quality of student admitted to the pharmacy program, such as faculty members, students, admissions staff members, and practitioners). A 10-station MMI based on the weighted-attribute blueprint was pilot tested with 30 incoming pharmacy students. MMI score reliability (intraclass correlation coefficient [ICC]) and correlation with other admissions tool scores were determined.
Results. Station scores provided by student interviewers were slightly higher than those of faculty member or practitioner interviewers. While most interviewers judged a 6-minute interview as “just right” and an 8-minute interview “a bit long,” candidates had the opposite opinion. Station scenarios had face validity for candidates and interviewers. The ICC for the MMI was 0.77 and correlations with prepharmacy average (PPA) and Pharmacy College Admission Test (PCAT) composite were negligible.
Conclusions. MMI feasibility was confirmed, based on the finding that interview scores were reliable and that this admissions tool measures different attributes than do the PCAT and PPA.
pharmacy admissions; multiple mini interviews; pilot test
To describe the extent of psychiatric pharmacy instruction in US pharmacy curricula, including course and faculty characteristics and mental health topics taught in clinical therapeutics-based courses.
An 11-item survey instrument (54% response) was developed and mailed to 91 colleges and schools of pharmacy.
Over 75% of colleges and schools employed a psychiatric pharmacist; however, less than 50% of faculty teaching psychiatric pharmacy content were psychiatric pharmacy specialists as defined in the study. All colleges and schools included psychiatric topics as part of a therapeutics-based course with an average of 9.5% of course content devoted to these topics. About 25% of colleges and schools offered elective didactic courses in psychiatric pharmacy. Only 2 schools required a psychiatric pharmacy advanced pharmacy practice experience (APPE), but about 92% offered elective APPEs. The mean number of hours spent on lecture- and case-based instruction across all colleges and schools was highest for depression and lowest for personality disorders.
There is a need for colleges and schools of pharmacy to better identify and standardize the minimal acceptable level of didactic instruction in psychiatric pharmacy as well as the minimal level of specialty qualifications for faculty members who teach this subject.
psychiatric pharmacy; pharmacy education; curriculum; mental health
The purpose of this article is to describe the experiential program created at the newly formed University of Hawai‘i at Hilo College of Pharmacy (UHH CoP). The Introductory Pharmacy Practice Experience (IPPE) rotations were developed to prepare student pharmacists for their final year of Advanced Pharmacy Practice Experience (APPE) rotations by improving clinical skills and patient interactions. In partnership with the John A. Burns School of Medicine (JABSOM) Department of Family Practice, physician and pharmacist teams collaborate to deliver patient care for chronic diseases and elevate educational opportunities provided by UHH CoP. Another goal of the experiential program is to determine whether the investment of pharmacist faculty and adjunct physician/nurse preceptors prepares students for the final year of APPE rotations. A survey was administered to non-faculty pharmacist preceptors who taught the third IPPE rotation during the summer of 2009. Twenty-nine surveys were received from six facilities on O‘ahu and the Big Island. Initial survey results revealed an overall rating average of 3.72 (Likert scale: 1-lowest to 5-highest), an average of 4.14 for professionalism, an average of 3.41 for overall clinical skills, and an average of 3.45 for overall readiness for experiential rotations. Average ratings when compared with fourth-year students from several mainland colleges ranged from 1.7 to 2.2 (1-worse than, 2-same, 3-better). This data demonstrates that UHH CoP is investing faculty and preceptor resources wisely to enhance the preparation of students for APPE rotations.