The Singapore-adapted GLF was well-received and proved a useful educational and development tool that demonstrated improvement in performance over time. The majority of pharmacists who undertook an evaluation using the tool at the study site felt it made a positive contribution to their learning experience. Feedback from the super trainers was encouraging and indicated that the whole GLF process was well-received as a way to evaluate many different aspects of practice (ie, attitudes, knowledge, and skills), provide inspiring and practical feedback, prompt reflection on practice and thus provide a platform for needs-based lifelong adult learning (). The few perceived negative aspects included feeling uncomfortable being observed and providing feedback to a peer. Others found the process taxing, a sentiment echoed by colleagues in the Queensland study.24
Both of these aspects are expected to become easier with time as the GLF is integrated into daily practice and a more open learning culture develops.
Many pharmacists found the process of self-reflection on their own practice almost as important as the observed evaluation and feedback process. The process of self-reflection, which is a significant component of the GLF, is important to facilitate a greater understanding of defined and accepted expectations and is consistent with the adult learning principles of self-reflection, feedback, and needs-based learning. Once learning objectives were identified using a combination of self-reflection, evaluation, and feedback, different strategies were used to meet these objectives. Some simply involved reading and discussion of findings (where possible in relation to a live case), maintaining and discussing care plans, or reviewing intervention records. Others consisted of giving case presentations, conducting literature reviews and presenting the findings, writing summary documents on new research findings pertinent to clinical area, or demonstrating improvements in practice by training others in a task and being observed performing activities such as taking medication histories. Some cases involved more formal training, such as an in-house intensive-care training course or certification using outside providers (eg, the fundamentals-of-critical-care support course offered by the Society of Critical Care Medicine).
Of the 63 behaviors analyzed, all but 8 demonstrated a significant improvement between baseline and repeat observations. The 8 cases in which no significant improvement was observed were a result of pharmacists being perceived to already be practicing at the highest performance level during the baseline evaluation. The competencies that this applied to related to tasks that were widely accepted as comprising fundamental roles of a hospital pharmacist in Singapore, an observation that has also been made in other studies.23,24
The behaviors in the problem-solving cluster all demonstrated significant improvement as well as the largest change in performance over time out of the 3 clusters (). This finding reflects research results from other countries using this tool and emphasizes the large growth in problem-solving skills that takes place in the first years of practice.23,24
This cluster measures knowledge-based behaviors, such as describing the pharmacology of drugs, pathophysiology of disease, and mechanisms of interactions, which require a more complex understanding and continual learning, compared with the process-based behaviors that comprise the majority of the other 2 clusters. Process-based behaviors that involve the learning and refining of processes related to daily activities appear to be mastered more quickly than knowledge-based behaviors, which are subject to continual improvement and can be fully refined only over time in conjunction with appropriate mentoring and continued professional development.15
The lower median scores demonstrated in the problem-solving cluster, compared with those in the other 2, suggest that this cluster may show the greatest improvement from continual mentoring and guidance. They also highlight the gap between theory learned at university and during the preregistration year and the application of knowledge and skills in practice. Parts of the United Kingdom have addressed this gap with the introduction of a Diploma in General Level Pharmacy Practice, which incorporates the GLF to support the development of junior pharmacists using work-based self-directed learning and case-based assessments under the mentorship of a more experienced practitioner.34
In response to the learning needs identified by the GLF in this study, pharmacists at the study site were divided into clinical teams, each of which was led by a designated senior pharmacist. Teams met each week to discuss cases, share learning experiences, and review GLF training needs.
Eighteen of the 24 behaviors in the patient-care cluster demonstrated significant improvement in performance, including the behaviors around medication history-taking and allergy documentation (). When carried out by pharmacists, these activities have been demonstrated to be more complete and to result in reduced mortality.7,35-37
Therefore, they are essential components of a clinical pharmacy service that must be performed at the highest level to ensure optimal patient outcomes.
Six of the behaviors that failed to demonstrate significant improvement were in the patient-care competency cluster. Five of these related to the provision of medication and included ensuring that prescriptions were unambiguous and legal, medication labels contained the required information, the correct drugs, patients, and labels were provided, and supplies of medication were documented. These supply-related behaviors are traditionally seen as some of the most well-established roles of the pharmacist and were being performed at the highest level at the baseline evaluation. Interestingly, “ensuring medication availability,” which was 1 of the 5 behaviors within the provision of medication competency, did show significant improvement, suggesting that this was perhaps not as obvious a role among general pharmacist practitioners as the other 4 supply-related behaviors, and highlighting how the GLF can be a useful tool to set standards and ensure uniform provision of services.
A similar trend was seen in the behaviors relating to the patient-education competency. In this competency, provision of appropriate oral/written information did not show significant improvement over time because of the high level of baseline performance, but the advice on non-pharmacotherapy treatments and assessing the patients’ understanding of the information they had been given did show improvement. “Advice on non-drug therapy” scored a median of 3 at both time points; although significant improvement was demonstrated, it was the lowest-scoring behavior in the patient-education competency. This finding raises the issue of whether it is necessary for pharmacists to aim for the top performance level in all behaviors or if it might be more advantageous for a department to prioritize by setting minimum standards of performance based on staffing, expectations, and targets.
The professional competency cluster was the highest scoring of the 3 on repeat evaluation and contained the final 2 behaviors that failed to demonstrate significant improvement. These behaviors related to confidentiality and recognizing the value of team members (). Again, performance was already considered to be at the highest level at the time of the baseline evaluation, suggesting that these values were instilled early in the pharmacists’ careers. The 2 behaviors in this cluster that ranked lowest at both time points were “demonstrates confidence” and “active in educating and training healthcare professionals.” It is possible that these 2 behaviors are linked, and pharmacists will become more involved in education and training as they gain experience and, therefore, confidence.
The GLF demonstrated improvements in performance over a median of 9 months, demonstrating that it is a valid tool for measuring performance over this timeframe. There was also potential for further performance improvement after 9 months, suggesting that this tool would be a valid development aid beyond this time period.
This study is the first analysis of the use of the GLF in an Asian setting. The introduction of this tool was received with generally positive feelings in a culture where knowledge-based assessment is traditionally favored over competency-based programs. However, a shift in thinking is occurring, in which the value of individualized competency-based development in addition to (rather than instead of) academic merit is being recognized by many of the most influential clinical pharmacy leaders in Singapore. The GLF has now been adopted by a significant number of public healthcare institutions, with others expected to follow in the near future.
Pharmacists were initially nervous at the thought of being observed in practice but also quite excited at the prospect of having help from a more experienced mentor to develop their practice. Pharmacists are traditionally accustomed to working alone, and it is rare to have someone observe their practice after the initial training period. Although hands-on training and observation are key components to the postgraduate development and training of medical staff members and other healthcare professionals in Singapore, clinical pharmacists have largely been left to develop and, in many cases, pioneer their own practices. Some thrive on this challenge, but many others require more support, empowerment, and instruction.
Another argument in support of observing professional practice is that all practitioners should be open to having their practice reviewed by peers. Such interaction provides the opportunity to ensure practitioners are providing safe and effective care to patients and to congratulate those who achieve identified goals and perform tasks according to standards. The GLF plays a pivotal role in clinical governance in that it can be used as an effective tool by heads of pharmacy to measure how pharmacists are performing according to defined and accepted standards.
The current study demonstrated significant improvements in 87% of all behaviors (n=63) evaluated over a median of 9 months. This compares with 95% (n=58; p
< 0.05) at 6 months (sustained at 12 months) in the original London study, and 57% (n=61; p
< 0.05) over a median of 14 months in the Queensland study.23,24
Performance improvements are comparable between the London and Singapore cohorts, but there is an obvious disparity in the Queensland results. Although the competency frameworks used in the 3 studies contained different behaviors so that a direct comparison is not feasible, some observations can be made.
The weighting of behaviors in the London and Singapore frameworks were comparable, whereas the Queensland framework had a larger focus on the patient-care cluster (). The patient-care behaviors in the Queensland framework were more detailed and included extra behaviors, such as relevant patient background, patient’s understanding of illness, and patient’s experience of medication use. These behaviors generally had lower baseline and repeat scores, perhaps indicating that other behaviors were prioritized over these.
Comparison of Performance Improvement, by Practice Site and Competency Cluster
Nine behaviors in the Queensland study (compared to 8 in the Singapore study) failed to show significant improvement due to pharmacists already performing at the maximum level upon initial evaluation. These behaviors mainly related to the professional cluster and the discharge-facilitation competency. The remaining 17 behaviors, which did not significantly improve (although most demonstrated a trend toward improved performance) were primarily in the patient-care cluster. The explanation provided for this finding was that these behaviors were associated with a deeper understanding of medication-related consultation. If the Singapore GLF had been as detailed, perhaps a similar result would have been demonstrated.
Two of the behaviors that failed to demonstrate improved performance in the Queensland study which were also included in the Singapore GLF were “medication reconciliation” and “mechanisms of interactions.” These showed a significant improvement in 1 country but not in the other, a difference that could be explained by variations in expectations and accepted standards of practice between the 2 countries. For example, the Singapore GLF states that medication reconciliation should be done “when appropriate,” whereas in Queensland, it is a standard procedure for all patients admitted to hospital. Perhaps this disparity is attributable to time pressures, staffing levels, an understanding of the importance of this process, or the level of development of clinical pharmacy practice.
Only the Singapore GLF contained a competency related to provision of medication and, on initial evaluation, performance of most of the associated behaviors was maximal. Now that technicians have largely assumed the supply role in Singapore (as in London and Queensland), perhaps these behaviors could be transferred to a technician-level framework.38,39
The validity, sensitivity, and reliability of the GLF evaluation process has previously been evaluated.22,23
However, using such a tool is always open to variations in the expectations of individual assessors. To reduce inter-rater variability, the intent was that the same facilitator should complete all evaluations for an individual pharmacist throughout the study. However, in some instances, this was not possible; 20% of pharmacists had 2 facilitators. The GLF can be used as a developmental tool for individual pharmacists, independent of such variations, but the comparison of scores between individuals should be done loosely, taking this potential limitation into account. In response to feedback from this study, a handbook was produced as a reference for trainers and trainees to provide more detailed descriptions of the competencies to facilitate standardization of the process.33
Feedback was obtained at regular intervals from the general pharmacist practitioners, and 81% of those surveyed indicated that the GLF added value to their leaning experience. Ideally, general pharmacist practitioners will echo the positive feedback received from the super trainers, but this outcome cannot be confirmed without completion of similar feedback questionnaires.