|Home | About | Journals | Submit | Contact Us | Français|
Pediatric transfusion medicine (PTM) is a subspecialty of transfusion medicine (TM) with no formal training program and few specialists. The Pediatric Transfusion Medicine Academic Awardees (PedsTMAA) group surveyed PTM content experts to identify relevant objectives for the first formal PTM curriculum.
Eight North American PTM experts were invited to participate in a two-step consensus process. PTM-related objectives compiled from a review of existing training documents were organized into a survey. Experts were asked to rate each objective for relevancy for a clinical pathology trainee. Content validity indices (CVIs) and asymmetric confidence intervals (ACIs) of expert ratings and analysis of respondents’ comments were used to identify relevant objectives.
Six experts participated and reviewed 117 objectives. Based on content validity criteria (CVI ≥ 0.83 and lower-limit 95% ACI ≥ 3), a total of 65 objectives were considered relevant. Twenty-three objectives were rated “very relevant” by all the experts while some proposed objectives were determined to be not relevant, out-of-date, or inappropriate for a resident trainee level.
The PedsTMAA group identified 65 objectives for a PTM curriculum. Twenty-three represent a clear core set of objectives and should be considered for clinical pathology training. The next step is to consider the teaching strategies and evaluation methods that will be employed to best deliver this content addressing competency in medical knowledge.
Transfusion therapy of fetal, neonatal, and pediatric patients requires special considerations because the indications, techniques, and relative risks and benefits differ from those of adults. For example, in some instances, specific pediatric populations are at risk for transfusion associated volume overload or metabolic complications and specialized approaches are needed to prevent these complications. Additionally, the immaturity of a neonate’s immune system reduces the risk of alloimmunization to transfused RBC antigens, but can simultaneously increase other risks such as transfusion associated graft versus host disease. Pediatric patients with congenital red blood cell disorders that require chronic transfusion therapy are at increased risk for iron toxicity and alloimmunization to RBC antigens and foreign plasma proteins. Transfusion complications have an added importance in pediatric populations since transfused pediatric patients have longer post-transfusion survival rates compared to transfused adults, and therefore, are at increased susceptibility to long-term consequences of transfusion (i.e., viral infection, immune modulation, and iron overload).1, 2
Clinical pathology training programs should cover relevant PTM content during the transfusion medicine/blood bank core curriculum and rotation to provide the trainee with sufficient knowledge and skills for pediatric transfusion therapy that occurs in most hospitals. A clinical pathologist should be in a position to offer consultation in the evaluation and management of surgical and medical pediatric patients requiring blood products.3 The need for pediatric-specific training in areas of clinical pathology has been recognized by others.4 The members of the Children’s Health Initiative through Laboratory Diagnostics (CHILDx) were the first to convene a task force to develop a specific list of needed training elements, including curriculum topics, key resources, and training experiences in pediatric laboratory medicine.4 However, many trainees have insufficient exposure to PTM because there may be inadequate numbers of neonatal or pediatric patients in training program facilities, infrequent clinical PTM-related consultations, and lack of exposure to pertinent didactic/educational materials. A defined PTM curriculum would serve to enhance professional training in preparation for the pathology board exam and future practice.
The Pediatric Transfusion Medicine Academic Awardees5 (PedsTMAA) group sought to identify relevant PTM content for a clinical pathology residency training program which already has many topics to address in its curriculum. The group used a two-stage process; the first stage or “developmental stage” involves the identification of objectives through a comprehensive literature review, and the second stage or “judgment/quantification stage”, involves surveying PTM experts to evaluate and rate each objective for relevance for the group of learners in question.6, 7 Outside PTM experts were invited to evaluate the content identified by the PedsTMAA group to establish content validity. Given the small number of PTM specialists and lack of formal training programs, dissemination of a PTM curriculum would serve as a resource for program directors, educators, trainees, and others involved in transfusion of pediatric patients.
As shown in Table 1, a total of 171 PTM-related objectives were identified from existing training documents in the fields of pediatrics,8 pediatric hematology-oncology,9 clinical pathology,4, 10 and transfusion medicine.11–13 After review and discussion among members of the PedsTMAA group for redundancy, 78 objectives were subject to expert review. We proposed that this set of objectives could be further narrowed into a core set of objectives using a process in which PTM experts judge relevance for a PTM curriculum.7
Nationally, few transfusion medicine specialists are recognized for their expertise in PTM. The PedsTMAA group identified potential PTM content experts based on knowledge of their publications, research, and/or clinical expertise in the field.7 Additionally, a content expert should have a “broad/comprehensive” working knowledge in PTM to be able to address all or most of the topic areas in the survey for a clinical pathology trainee. We did not include individuals whose training or experience limited their expertise to one or a few topics within PTM. To evaluate their current PTM expertise, we performed a PubMed search to quantify relevant publications in the past fifteen years (2007-1993). If an expert only had a limited number of PTM-related publications, we further evaluated their relevant experience, taking into account their clinical training in pediatrics, their experience directing pathology residency programs, their contributions as authors or editors to books on PTM, and their experience in pediatric blood banks, especially those at large academic medical centers. Selection of experts was limited to North America because differences in standards, training, and/or practices of PTM in other countries would influence the content of the curriculum. Experts were individually invited via email to participate in a two-step consensus project to identify relevant PTM objectives for a clinical pathology residency training program. The University of California San Francisco Institutional Review Board approved the study protocol.
The 78 objectives identified through a comprehensive literature review were organized into a survey format. To avoid bias, the wording of the each objective was left unchanged from the original source. The experts were asked to rate each objective on a four-point scale for relevancy for a clinical pathology residency training program: 1 = not relevant, 2 = somewhat relevant, 3 = relevant, but needs revision, and 4 = very relevant.6 A four-point rating scale has been recommended to minimize the ambivalent middle rating common in odd number rating scales.6 In addition, for purposes of data analyses, ratings can be collapsed into dichotomous categories of relevance; a rating of 3 or 4 = relevant and a rating of 2 or 1 = not relevant. Additional space was available after each objective for respondents’ comments or suggestions for revision. At the end of the survey, respondents were asked to list additional PTM topics and/or objectives not covered in the survey. The survey was sent via email to each expert with a cover letter invitation and description of the study.
Based on the expert ratings and feedback, we developed a second survey composed of 13 revised items and 26 new objectives. The same reviewers were asked to rate the survey items using the same four-point scale as described above.
Two weeks after each invitation, the experts were sent an email reminder. Respondents had the choice to send completed surveys by fax or email attachment, and therefore, were not anonymous.
A widely accepted method used to quantify the relevancy of an objective is the content validity index (CVI).6 On a four-point rating scale (1=not relevant, 2=somewhat relevant, 3=relevant, but needs revision, and 4=very relevant), the CVI is the proportion of experts who assign the objective a rating of 3 (relevant) or 4 (very relevant). For example, if five out of six experts rate an objective as 3 or 4, the item’s CVI is 0.83 (i.e., 83% of experts agree that the objective is relevant or very relevant). A CVI of 0.78 or greater has been recognized in the literature as sufficient agreement to identify an item to have content validity.6, 14 However, this number is likely to be high by the nature of the task. Because of the possibility of chance inflation of the CVI, Penfield and Miller proposed the use of asymmetric confidence interval (ACI) around each objective’s mean rating to better identify an item as content valid which has been used in other studies.15–18 This confidence interval is similar to other confidence intervals in that it determines how confident the researcher is that the data represents truth and is not just due to an artifact of sampling. The typical confidence interval also assumes a normal distribution of the variable which is not an appropriate assumption with CVI.19, 20 In surveys of curriculum content in which the number of experts surveyed is often low (often fewer than the six that participated in this study) and the distribution is likely to be skewed, a different approach is used to determine how likely the survey reflects the “truth”. This alternative approach is known as the ACI, The ACI improves the accuracy of content validity studies by providing information concerning the expected agreement of the sample mean with the population mean; i.e., it measures how the chance that the average found in the study accurately reflects the average of the general population.18 Therefore, an improved way of estimating content validity is to use two criterions, CVI and ACI. For this study, ACIs were computed following methods described by Penfield and Miller.18 In addition to a CVI of 0.83 or greater, a lower-limit 95% ACI of 3 or greater was the cutoff used to consider the objective relevant for a clinical pathology training program.18 CVIs and ACIs were calculated in Excel (2003 v.11.8237.8221, Microsoft Corp., Redmond, WA). Objectives that met both criteria were classified as “accept” and ones that met neither or one criterion were rejected.
Six of the eight invited PTM experts completed and returned both surveys, yielding a response rate of 75%. Two experts declined participation because of other commitments and unable to participate at the time the survey was sent. All of the reviewers were transfusion medicine specialists. Four of the six experts had additional formal training in pediatrics and/or pediatric hematology-oncology. A summary of expert qualifications is displayed in Table 2.
Experts provided comments related to “datedness” and appropriateness of objectives for a clinical pathology resident trainee (“general overview” versus “too detailed for trainee level”). Some of the objectives under the topic heading of Hemolytic Disease of the Newborn from the proposed Transfusion Medicine Academic Award (TMAA) curriculum13 were considered out-of-date, given current technology, knowledge, and practice. Several of the reviewers suggested changing the topic heading to Hemolytic Disease of the Fetus and Newborn (HDFN). The objectives on Iron Overload were noted to be unnecessarily detailed for a clinical pathology trainee. Hemostasis and Ethical Issues were suggested as additional topics for a PTM curriculum.
Seventy-seven of the 78 objectives identified in the literature and existing training documents (survey I) met the CVI criterion of 0.83 or greater, and of these, 69 met the criterion of achieving a lower-limit 95% ACI of 3 or greater. However, only one of the 26 new and six of the 13 revised objectives composed based on expert feedback (survey II) achieved content validity criteria. The summary of results is displayed in Table 3. Overall, a total of 76 objectives were accepted. Eleven objectives were excluded because of redundancy, resulting in 65 objectives. The final topics and objectives accepted for a PTM curriculum along with their mean rating, CVI, and lower-limit 95% ACI are reported in Table 4. Appendix I reports the same information for the rejected items.
Although there are several books on PTM,21–23 the development of a validated PTM curriculum serves to refine the content for teaching clinical pathology trainees and potentially other target learner groups (e.g., pediatricians, pediatric hematologist-oncologists, other pediatric subspecialists, participants in continuing medical education, and trainees outside the USA). Residency programs may find it easier to include this set of objectives to complement available text resources. In addition, a defined curriculum can be periodically up-dated by interested parties as the field advances through new studies and technologies. Given the limited number of PTM experts, a curriculum would serve as an educational resource to teach the important issues of the field where local PTM expertise is unavailable and/or where there are few pediatric cases in a hospital transfusion service.
Recently, several groups have produced training documents related to transfusion medicine. The Academy of Clinical Laboratory Physicians and Scientists (ACLPS) published a comprehensive clinical pathology residency training document of which transfusion medicine was a subsection.10 Fung et al. further expanded on the objectives from the ACLPS document and assigned the Accreditation Council for Graduate Medical Education (ACGME) general competencies (patient care, medical knowledge, practice-based learning and improvement, interpersonal and communication skills, professionalism, and systems-based practice) fulfilled by each objective.11 Wu et al. provided a blueprint on how to design and implement a TM residency and fellowship training program in the context of the ACGME competencies.12 Together, these resources provide a comprehensive scope of all areas of transfusion medicine. However, the content of a PTM curriculum for teaching clinical pathology trainees has received little attention.
This study identified a clear core set of 23 objectives unanimously considered “very relevant”, and a second tier set of 42 “relevant” objectives. The information derived in this study sets a firm foundation for the development of a curriculum of a magnitude that can be incorporated into a demanding residency program. The methodology used provided a clear direction for the next steps in curriculum development, and both narrowed the objectives considered a priority and eliminated those that might otherwise have been debated. Of the “very relevant” objectives, nine are focused on the topic of Pre-transfusion testing, Product selection, and Indications for special patient populations, and six are related to HDFN (Table 3). These topics are clearly important and frequently common reasons for consultation of a TM specialist. The second tier of objectives accepted are related to the following topics: Ethical Issues, Hematopoietic Stem Cell Transplantation (HSCT), Neonatal Alloimmune Thrombocytopenia (NAIT), Neonatal Alloimmune Neutropenia (NAIN), Indications and controversies surrounding use of “special” blood products, and Sickle Cell Disease and Thalassemia. Additional topics did not have broad enough consensus among the experts for content validity and hence, we do not have evidence to include these as part of a core curriculum. However, the curriculum is meant to provide a minimum set of PTM topics and should not be used to preclude individual residency program directors from including the rejected topics as part of their Transfusion Medicine training.
Although 26 new objectives related to the topics of Ethical Issues and Hemostasis were presented in the second survey, only one new objective was accepted by content validity criteria. The purpose of this survey was to further refine the content of a PTM curriculum based on expert feedback. An additional column was provided on the survey for the respondent to mark if the objective was “relevant in advanced curriculum”. Twenty-five of the 39 objectives were marked “advanced” by at least one respondent. As a result, some experts did not rate the objective and thereby this had an impact on the quantitative measurements of content validity. To avoid missing data from respondents, the instructions could have been more explicit. A reason none of the Hemostasis objectives were accepted could be related to the wording or content of the objectives. The content may have been oriented for practitioners who most frequently provide direct patient care and considered not relevant for a clinical pathology trainee. In addition, Hemostasis is a topic that is commonly taught during the coagulation core curriculum or rotation. Training programs should consider evaluating the adequacy of pediatric-related hemostasis objectives in the coagulation core curriculum. Finally, experts tend to have biases towards content that they are more familiar with, but the use of ACIs should have minimized the influences of such biases.
There are several potential limitations in this study. The authors of this paper might have created biases in the way the comprehensive review of existing training documents was synthesized in preparing the survey for expert review. While the comprehensive review included documents from pediatrics, clinical pathology, and transfusion medicine, and the wording of the objectives selected for the survey was left unchanged from the original source, members of the PedsTMAA discussed and removed redundant objectives. Although the survey instructions asked reviewers to rate the objective for a clinical pathology trainee, it is possible that expert’s training background (pathology and/or medical specialty) could have influenced their ratings. The relatively small number of recognized experts in the field limited the survey size. Although Hemostasis and Ethical Issues were suggested by the experts as additional topics for a PTM curriculum, the objectives presented were not widely accepted (Appendix 1). The wording of the objectives may have influenced the expert ratings. The objectives were written by members of the PedsTMAA and may have been unnecessarily detailed for a pathology trainee. Despite these limitations, the methodology used in this study has identified a relevant set of objectives to use in further development of a PTM curriculum. The PedsTMAA group will organize the objectives presented here into several levels of training (e.g., residents, transfusion medicine fellows, and other subspecialists) and write them using appropriate language for objectives so that specific medical knowledge needs of target learners are addressed. With a well developed curriculum, more than knowledge can be addressed and the PedsTMAA group will designate the ACGME competencies fulfilled by each objective once they are embedded in learning materials.
We thank Margaret Clark, PhD, for editorial assistance, and Dr. Elizabeth Read for helpful discussion of the manuscript. A special thanks to our Program Officer, Traci Heath Mondoro, PhD, for her support and contributions to PedsTMAA. We also thank Eileen Everret for technical assistance.
This work was supported by Grant 5K07HL88923-4 from the National Heart, Lung, and Blood Institute, NIH.
REPRINTS WILL NOT BE AVAILABLE
Conflict of Interest: None