|Home | About | Journals | Submit | Contact Us | Français|
The ability to assess quality of care is a necessary component of continuous quality improvement. The assessment typically is accomplished by determination of compliance with a defined set of quality measures (QMs). The objective of this effort was to establish a set of QM for the assessment of the process of care in JIA.
A 12 member working group (WG) composed of representatives from the ACR, AAP, ABP, and ARHP was assembled to guide the project. Delphi questionnaires were sent to 237 health professionals involved in the care of children with juvenile idiopathic arthritis (JIA). A total of 471 items in 27 domains were identified. The WG met via four live e-meetings during which results from the Delphi’s were distilled to a reduced draft set. Each WG member selected a proposed QM to investigate and present evidence from the literature as to its attributes and appropriateness for inclusion into the set. Nominal group technique was used to come to consensus on a proposed set of QMs.
The proposed set contains 12 QMs within four health care domains. Each QM consists of a statement of (i) the assessment to be completed, (ii) when the first assessment should be completed and a suggested frequency of assessment during follow-up, (iii) recommendations of appropriate tools or methods of assessment, and (iv) initial performance goals.
Implementation of the proposed QM set will improve the process of care, facilitate continuous QI, and eventuate in improved health outcomes of children with JIA.
Pediatric rheumatology centers are engaged in local quality improvement (QI) efforts in their centers to varying degrees. Systematic efforts have been pursued at some centers for years, but for most the efforts are just beginning. Central to this process is the identification of ‘quality measures’ (QMs, also commonly referred to as ‘quality indicators’). QMs must be developed in consideration of the evidence that (i) their performance results in improved health outcomes and evidence and (ii) they are operationally feasible in a variety of clinic infrastructures. QMs focused on the process of care typically contain a statement of the assessment to be completed, method for conducting the assessment, frequency of reassessments during follow-up and initial performance goals. Although some individual pediatric rheumatology centers have identified QMs and are using them to perform QI, there is not a uniform QM set relevant to the process of care of children and adolescents with Juvenile Idiopathic Arthritis (JIA) in widespread use. In this project the focus was on process of care to facilitate development of a standardized method of assessment of QI across centers. Once this method is established, then centers will be able to use these data in the future to develop validated QI measures that focus on patient status and outcomes.
In 2008 the American College of Rheumatology (ACR) provided funding to develop a set of QMs for use by health care professionals (HCPs) involved in the care of patients with JIA. The chief objective of the project described herein was to convene representatives from major North American organizations involved in pediatric rheumatology to work together to develop a standard set of QMs, based upon existing evidence and clinical experience focused on the process of care for patients with JIA. The development of this standard set of QMs was done to (i) facilitate the development of a network of pediatric rheumatology centers working in a coordinated fashion to improve the outcome of children with JIA, (ii) to serve as a basis for QI projects for the ACR and other organizations and, (iii) to be used by the American Board of Pediatrics (ABP), as Part 4 of the requirements for Maintenance of Certification in Pediatric Rheumatology1.
We used a methodology similar to that used by MacLean et al to develop QM for rheumatic diseases in adults2. A Quality Measures Working Group (WG) was assembled that was composed of 12 individuals, each selected by their respective organization from the pediatric rheumatology sections of the American College of Rheumatology (ACR) and the American Academy of Pediatrics (AAP), the Association of Rheumatology Health Professionals (ARHP), the American Board of Pediatrics (ABP), the Childhood Arthritis and Rheumatology Research Alliance (CARRA) Consolidated Registry Committee, the Pediatric Rheumatology Improvement Network for Clinical Effectiveness (PRINCES), and two facilitators experienced in group process and consensus formation techniques. The WG’s overall charge was to use consensus techniques to develop a QM set, based upon available evidence and clinical expertise, for the process of care for JIA.
In order to obtain broad-based input regarding health care domains and assessments in those domains, the WG sent an online Delphi-type3 electronic questionnaire (Delphi 1) to various stakeholders. The Delphi packets included an explanation of the rationale for need, the intended purpose of the survey, explanations as to what QMs represent and their intended use. Recipients were instructed to state what they believed to be the top five measures of outcome or care in JIA. A total of 237 individuals received the survey, including 170 pediatric rheumatologists, eight advanced practice nurses (APNs) involved in JIA care, and 59 patients and parents, representing 27 states and Prince Edward Island, Canada. Results of Delphi 1 were reviewed by the WG, which distilled the assessments offered by respondents including grouping them into domains, during a series of teleconferences employing nominal group technique (NGT) consensus formation methodology4. Replies that were redundant or represented the same concept, but stated in a different manner, were combined. Domains and assessments in the domains were then retained or eliminated based largely upon feasibility (practicality, ease of use), and face validity (clinical sensibility)5-6.
A second questionnaire survey (Delphi 2) containing a reduced set of domains and assessments was developed and sent to the same 237 stakeholders, regardless of whether or not they had replied to Delphi 1. Participants were asked to rank order the reduced number of domains and assessments within a domain in terms of their perceived relative importance in assessing quality of care.
Working from these results, a series of four web-based interactive live meetings using iLinc (available at www.iLinc.com) were held among WG members and facilitators during 2009. The purposes of these meetings were to come to consensus on the following questions: (1) should the assessment be retained in the proposed set; (2) do validated methods exist for conducting the assessment and, if so, should one method be recommended over the others; (3) when should the first assessment occur, and how frequently should reassessment be done during follow-up; (4) what proportion of patients with JIA in a given clinical practice should meet the QM initially, referred to as the ‘initial performance goal’.
Prior to convening the series of live meetings, each WG member chose a specific QM to investigate by searching the literature and relying on his/her personal clinical expertise in order to answer the four questions shown above. The intent was to familiarize the WG with existing evidence of the utility and clinical relevance of each QM, thereby facilitating consensus-formation and synthesis of a proposed set of recommendations. (The specific references used to obtain evidence on which to base recommendations made to the group by individual presenters are given as an Appendix to this manuscript.)
Format of the four live meetings was as follows. A facilitator introduced the topics for discussion and deliverables (e.g. questions to be addressed) for the call. This was followed by a presentation of the evidence for each QM by the member who had undertaken the review. The presenter stated his or her suggested answers to the questions above, and provided the evidence that formed the basis for their opinions. Following the presentation, NGT was used to come to consensus about the answers to the questions stated above. In general, meetings lasted approximately 3 hours and two to three QM were discussed during each meeting. Facilitators reviewed digital recordings of each meeting, developed a working document containing QM statements, and circulated it to the entire WG for revision. A finalized set of recommendations was then produced.
Response to Delphi 1: Overall response rate to Delphi 1 was 40% (N = 95); 35% for pediatric rheumatologists, 50% for APNs, and 52% for parents and patients. Among physician-respondents, 85% were currently treating more than 50 patients with JIA per year.
Responses to Delphi 1 resulted in 471 proposed JIA QMs sorted into 23 domains (Table 1). Using conference calls, Delphi and NGT, the WG reviewed the 471 QMs for feasibility, face validity, and redundancy. Item reduction resulted in five domains with a total of 19 quality measures: (1) disease control, 11 QMs, (2) safety monitoring, 4 QMs, (3) education, 2 QMs, (4) access/equity/timeliness, one QM and (5) relationship/patient-family satisfaction with care, one QM.
These five domains and 19 assessments formed the basis for Delphi 2 in which participants were asked to rank order the domains and, for the three domains that contained more than one assessment (disease control, safety monitoring, and education), rank order the assessments within the respective domain.
Response to Delphi 2: Overall response rate was 57% (N = 135): 57% for pediatric rheumatologists, 100% for APNs and 50% for patients and parents. Of these respondents, 69% had participated in the Delphi 1.
Results from Delphi 2 showed the highest ranking assessments in domains with more than one assessment were: (1) disease control: (tied) number of joints with active arthritis and physician global assessment of disease activity; (2) safety monitoring: patients undergo eye screening according to published guidelines; (3) access/equity/timeliness: time to first available appointment with a pediatric rheumatologist.
The WG reviewed the Delphi 2 responses, and used NGT to come to consensus on a working set of QMs. The original five domains and 12 assessments within the domains were maintained in the working set (Table 2). This working set formed the basis for the four live web-based consensus conferences aimed at reviewing the evidence for maintaining the proposed domains and assessments, as well as tools for assessment, frequency of assessment, and initial and final quality performance goals.
Results of the four live web-based meetings were as follows. During the initial meeting, the WG decided that five assumptions were necessary in order to specify what categories of JIA the QMs would be applicable to, and to provide working definitions of the terminology used in QM statements. These assumptions are shown at the top of Table 3. Consensus was reached during the live meetings on eight of the resulting QM statements. The remaining four QMs were re-circulated to WG members in survey format in an attempt to arrive at consensus. When it became apparent that no consensus among the WG members was possible, outside opinion was sought from five external board-certified pediatric rheumatologists chosen by the WG, and with extensive experience in caring for children with JIA. The proposed set of QMs resulting from the WG meetings and follow-up surveys are shown in Table 3. A total of 12 QM distributed among four health care domains are included in the proposed set.
Healthcare providers strive to provide good care to their patients by virtue of their training, and professional ethics; however, this does not necessarily translate into quality outcomes for the patients. In 1999, the Institute of Medicine (IOM) reported that an estimated 98,000 patients die from medical errors per year in the United States and mandated reformation in the healthcare system to reduce the error and improve safety for patients7. The IOM further recommended that all organizations improve their performance by incorporating care process and outcome measures into their daily work for improvement and accountability. The QI movement grew from this and subsequent efforts of the IOM and is now incorporated into a broad, complex and incompletely coordinated landscape of agencies, institutions, healthcare insurers, healthcare facilities, etc. Furthermore, the importance of developing quality measures derives from the contemporary era of mandated quality of care as set forth in virtually all arenas of medical environment: patient care, education, and accreditation.
The JIA QM Workgroup was organized with the purpose of developing a core set of QM for JIA that would be adopted broadly. Moreover, it was thought to be critical that these QM be developed by those most directly involved in the care of children with JIA-parents, patients and pediatric rheumatology health care providers. The organizations which the WG members were drawn from represent the key professional, scientific and accreditation organizations in pediatric rheumatology. The ABP, in accord with the recommendations of the Task Force on Competence of the American Board of Medical Specialties (ABMS), has mandated that all pediatric rheumatologists seeking ABP recertification demonstrate competency through active participation in quality improvement. Representatives from the ABP were actively involved in the development of these JIA QMs and will incorporate them in Maintenance of Certification training and materials. The JIA QM Workgroup chose to focus on measures of the system or process of care initially to establish regular assessments of the common important measures of care in a standardized fashion. This is a necessary and critical first step in QI efforts. Once centers have adopted these process measures we will be able to move forward as a group with addressing patient outcomes. This work will be informed and grounded in the results obtained from the current proposed core set of QI measures for the process of care in JIA.
The domains developed in this project will assess dimensions of care which the IOM identified as needing improvement. The IOM specified dimensions of quality care include safety, efficiency, efficacy, patient-centered, equity, and timeliness8. The domains in this JIA QM set include: 1) Disease Control which reflects effectiveness; 2) Safety Monitoring, which addresses Safety; 3) Assessment of Self-Efficacy and Patient/Parent Satisfaction which are central to Patient Centered Care; and lastly 4) Access which incorporates Timeliness. All of the selected JIA QMs are measurable and were subject to evidence in the literature. Equity in closing racial and ethnic gaps in health status, recognized as a key area needing improvement, is an outcome-focused measure, and therefore not included in the current set of QMs. ‘Initial performance goals’ given in the QM set are based largely on what the QMWG concluded were reasonable and feasible for most practices to achieve. Conversely, no ‘ideal performance goal’ is given because continuous QI implies that ever higher performance goals are to be set, achieved, and then improved upon further.
The JIA QMs developed in this project hopefully will serve as a format for registry development and initiation of serial data acquisition. This will provide a baseline of data for the development of quality improvement projects, both for the individual practitioner or as a collaborative for a group of centers. Many pediatric subspecialty groups, including neonatology, pulmonology, and critical care medicine have had strategies to conduct quality improvement through national and regional collaboratives. Benchmarking can be established as collaborative members are transparent in the data and share best practices. The optimal result of the quality improvement work done in either setting, individual or collaborative, will be the improvement of the processes of care which ultimately affects the outcome of the patient’s disease status. The view of the WG members is that this improvement will be achieved in a much more efficient and timely manner for JIA if the effort is driven by the use of a standardized set of widely adopted QMs. QMs developed in this project provide parameters but leave latitude for choosing the specific tools or instruments to measure the assessment. The aim is to have relatively uniform data collection for optimizing the outcome of quality improvement projects. Broad based efforts in pediatric rheumatology are ongoing at this time to develop a common JIA registry for longitudinal data collection in JIA patients in many pediatric rheumatology centers and to develop a sophisticated QI network. The incorporation of these JIA QMs in both these efforts will enhance the effectiveness of our shared goal of improving the systems of care for children with JIA so as to improve their disease, function, and quality of life outcomes.
The authors wish to thank all health care professionals, parents and patients who participated in the Delphi surveys, and the five external experts who provided further consensus voting on the final set of quality measures: Gloria Higgins, PhD, MD; Suzanne Li, MD, PhD; Paula Morris, MD; Karen Onel, MD; Sarah Ringold, MD, MS.
Grant Support: This work was funded through grants from the American College of Rheumatology and by NIH Grant No. (MCRC) P60-AR047884
Pain should be assessed in all patients at the first visit and at each subsequent visit that occurs at least 7 days apart. (The QMWG recognized two types of pain: (i) acute pain intensity at a point in time and, (ii) average pain over some period. Here, average pain over a period of 7 days is what is to be assessed.)
A full joint count of all 75 joints should be done on all patients at the first visit and at 6 month intervals. (Reduced or weighted joint counts may be acceptable in the future after these methods have been more fully validated)
A PGA should be completed on all patients at the initial visit and at each subsequent visit.
All patients should receive an assessment of functional ability at the initial visit and at a minimum of 6 month intervals thereafter.
All patients should receive an assessment of HRQOL at the initial visit and at 6 month intervals.
Either the AAP13 or modified Heiligenhaus14 guidelines for eye examinations should be followed for patients with any category of JIA. Documentation of compliance to the guidelines should be performed at every visit at least 3 months apart.
All patients with JIA will undergo TB screening no longer than 3 months prior to the start of any biologic therapy and yearly thereafter as long as the patient remains on biologic therapy.
All JIA patients receiving DMARD or biologic therapy will be monitored for toxicity by clinical laboratory methods. The minimal frequency of laboratory monitoring for DMARDs is every 2-4 weeks for the first 3 months of therapy, every 8-12 weeks after 3-6 months, and every 12 weeks after 6 months of therapy. Guidelines for laboratory monitoring of biologic therapy are evolving and recommendations in this quality measure will be added in the future.
Patients engaging in risk-taking behaviors which could have health consequences related to their medication or disease should receive counseling at each visit. Patients not engaging in risk-taking behaviors which may have health consequences related to their medication or disease should receive counseling at least annually.
(self efficacy is the belief that one is capable of performing in a certain manner to attain certain goals15)
Patients should be assessed for self-efficacy at periodic intervals. The first assessment of self-efficacy should be ≤ 6 months after the initial visit and every 6 months thereafter.
Patients and/or parents should be assessed for their level of satisfaction with the quality of care provided to them/their child. The first assessment should be done no longer than 1 year after the initial visit, and annually thereafter. (Because the individual physician is often not able to influence the patient’s satisfaction with the overall health care delivery system or insurance company, this QM focuses upon satisfaction with the health care provided by the physician.)
As an initial goal, all patients with signs and/or symptoms of JIA should receive an appointment, based upon the third-next-available20, an average of ≤ 60 calendar days for a new patient visit from the time of referral. (Timing of follow-up visits is dependent upon numerous factors, and no recommendation is made here.)
Daniel J. Lovell, Cincinnati Children’s Hospital Medical Center.
Murray H. Passo, Medical University of South Carolina.
Timothy Beukelman, University of Alabama at Birmingham.
Suzanne L. Bowyer, Indiana University, School of Medicine.
Beth S. Gottlieb, Steven & Alexandra Cohen Children’s Medical Center of New York.
Michael Henrickson, Cincinnati Children’s Hospital Medical Center.
Norman T. Ilowite, Children’s Hospital at Montefiore.
Yukiko Kimura, Hackensack University Medical Center.
Esi Morgan DeWitt, Cincinnati Children’s Hospital Medical Center.
Jill Segerman, Cincinnati Children’s Hospital Medical Center.
Leonard D. Stein, University of North Carolina.
Janalee Taylor, Cincinnati Children’s Hospital Medical Center.
Richard K. Vehe, University of Minnesota.
Edward H. Giannini, Cincinnati Children’s Hospital Medical Center.