|Home | About | Journals | Submit | Contact Us | Français|
Practical and objective instruments to assess pediatric Crohn’s disease (CD) activity are required for observational research and quality improvement.
1) To determine the feasibility of completing the Pediatric Crohn’s Disease Activity Index (PCDAI) and the Abbreviated PCDAI (APCDAI), and 2) To create a short PCDAI by retaining and re-weighting the most practical and informative components.
Physicians in the ImproveCareNow Collaborative for pediatric inflammatory bowel disease (IBD) were asked to record components of the PCDAI and assign a Physician Global Assessment (PGA) of disease severity at each patient encounter. We assessed the feasibility of the PCDAI, the APCDAI, and the individual index components by determining the proportion of visits in which data were recorded. We created a short index by retaining and reweighting components of the PCDAI completed in ≥80% of visits. The feasibility of the short PCDAI and its ability to discriminate between PGA categories were evaluated using descriptive statistics.
This study population included 1355 subjects with CD (6373 visits). The PCDAI and APCDAI were complete in 16.7% and 44.1% of visits respectively. A short PCDAI, including general well-being, abdominal pain, stools, weight, abdominal exam, and extra-intestinal manifestations could be completed in 66.5% of visits. The correlation between the Short PCDAI and PGA was similar to that of the PCDAI (r= 0.60, p<0.001 vs 0.61, p<0.001).
The Short PCDAI is a practical and valid tool to measure pediatric CD activity disease activity. Its use should facilitate quality improvement and observational research.
Crohn’s disease (CD) is a chronic, relapsing disorder of the gastrointestinal tract which affects nearly 600,000 Americans, including 50,000 children . Nearly 25% of patients experience onset of their disease during the childhood or adolescent years. Because childhood-onset disease is associated with a high degree of patient morbidity including missed work  and school, decreased quality of life[5, 6], and psychosocial disturbance[7, 8], both clinical research and quality improvement approaches are necessary to improve outcomes for affected patients.
A significant impediment to research in pediatric Crohn’s disease has been the difficulty in selecting an appropriate outcome measure [9, 10]. Disease activity indices, such as the Pediatric Crohn’s Disease Activity Index (PCDAI) have been developed and validated as endpoints for clinical trials. However, due to problems inherent in both the funding and recruitment for clinical trials (i.e. interventional studies), much of the research in pediatric Crohn’s disease has relied on observational study designs such as patient registries and retrospective chart reviews. Because the PCDAI requires determination of height velocity, analysis of laboratory tests, and components of the physical exam (perirectal examination) that are not always completed during patient encounters, the feasibility of this index in observational studies has been called into question. Consequently, a disease activity index that can be easily completed in the context of routine clinical care would greatly facilitate clinical research in pediatric Crohn’s disease. In addition, a more practical index would also be helpful in measuring outcomes for quality improvement purposes, in establishing performance benchmarks, and in assisting individual clinicians at the point of care in making decisions about whether or not to step-up or adjust medical therapy. Shortened activity indices have been previously developed for adult Crohn’s disease and ulcerative colitis assessments, with good feasibility and overall accuracy. Although an abbreviated PCDAI (APCDAI) has been previously proposed and tested in a single center study, this instrument has not been evalulated in a large, multi-center setting. Furthermore, use of the APCDAI in observational research may be limited by the requirement of a perirectal examination which is not always performed during routine clinic visits.
The aims of this study were 1) To determine the feasibility of completing the PCDAI, APCDAI, and their individual components in the context of routine clinical care, and 2) To create a Short PCDAI by retaining and re-weighting the most practical and informative components. To accomplish this, we performed an analysis of clinical and laboratory data prospectively collected by centers participating in the ImproveCareNow Collaborative, a quality improvement project for Crohn’s disease and ulcerative colitis (formerly known as the PIBDNet Trailblazer Improvement Collaborative).
The ImproveCareNow Collaborative for pediatric inflammatory bowel disease (IBD) was formed in April 2007 by 9 participating pediatric gastroenterology programs. Subsequently, an additional 6 centers joined the network. In this quality improvement program, children with previously and newly diagnosed IBD were enrolled during clinic visits, and followed prospectively at each encounter. At the onset of the collaborative, a case report form was designed to capture clinical and demographic information about each patient, including all of the components of the PCDAI as well as a Physician Global Assessment (PGA) of disease severity. Physicians at participating sites were asked to complete each of these items at each clinic visit.
For each patient visit, we scored each component of the PCDAI if sufficient data were available. If the required data were not available, we considered the component as incomplete and a score was not assigned. PCDAI component scores were assigned using the methods described in the published user’s guide. We calculated height velocity Z scores based on Centers for Disease Control (CDC) height velocity growth charts, only if a prior height and date were previously recorded 6–12 months prior to that visit. We based all Z scores on chronological age, as bone age and Tanner staging were not available for all patient visits. For female patients age 15 years or greater, or male patients age 17 years or greater, the height velocity PCDAI component was assigned as score of “0” because even a height velocity of 0 cm/year was greater than 1 SD below the mean. For the weight component, we used a historic weight recorded 4–6 months prior to the visit in order to assign the PCDAI weight score. We did not take into account weight loss which may have occurred due to corticosteroid withdrawal.
We used standard descriptive statistics to summarize the demographic characteristics of included patients. We then assessed the feasibility of measuring the PCDAI, the APCDAI, and the individual components of the PCDAI in the context of routine clinical care by determining the proportion of visits for which each index (or individual component) could be assigned a score. Next, we evaluated the relationship between the PCDAI and Physician Global Assessment (PGA) in our study population. For visits in which each of the components of the PCDAI and the PGA were completed, we used box and whisker plots and descriptive statistics to analyze the distribution of PCDAI scores across the following categories of PGA: inactive, mild, moderate, and severe.
In order to create a short index with improved feasibility, we examined the distribution of responses for each component of the PCDAI and retained only those components that were completed in ≥80% of visits. This led to removal of the items that were more often incomplete including perirectal exam, height velocity, and laboratory values. Thus, our final short index, included the following 6 components of the original PCDAI: abdominal pain, patient functioning/general well-being, stools, weight, abdominal exam, and extra-intestinal manifestations.
Next, to determine whether re- weighting of any of the individual components would improve the overall accuracy of the short index, we randomly split the visits into a training set and a validation set. In the training set, we created linear regression models using each of the component scores as independent variables and the log-transformed full PCDAI score as the dependent variable. Log transformation of the full PCDAI score was necessary as these scores were not normally distributed. A value of 0.5 was added to the full PCDAI score values prior to log transformation to account for values of 0. The components of the short index that were most strongly associated with the log-transformed total PCDAI score were identified from the size of the standardized regression parameters in the fitted multivariable model. Standardized regression parameters, rather than unstandardized β coefficients, were utilized to account for differences in standard errors between each of the components. These components were then weighted according to the magnitude of the standardized regression coefficients.
We then validated this component weighting strategy in the validation set using a two step process. First, we confirmed that the three items identified for weighting in the training set had similar standardized regression parameters in the validation set. Next, we compared the weighting schema selected in the training set with alternative weighting schema using Pearson’s correlation to measure the association between each schema and the log-transformed PCDAI in the validation set. We decided, a. priori, that if the correlation of the weighting schema identified in the training set was equal to or greater than that of the alternative weighting schema in the validation set, we would retained the original weighting strategy assigned during the training phase. To confirm that the selected weighting schema was not overly influenced by repeated measures within individual subjects, we repeated the training and validation procedures using only the first visit for each eligible subject and obtained similar results.
Finally, we evaluated the feasibility and performance of the Short PCDAI in the full set of visits. We assessed the overall feasibility of the Short PCDAI by determining the proportion of total visits for which the Short PCDAI could be assigned a score. We then measured the correlation between the Short PCDAI and a number of other indicators of disease severity including PCDAI, abbreviated PCDAI, PGA, hemoglobin, ESR, and albumin. Pearson’s correlation was used to assess correlation with log transformed continuous measures (PCDAI and abbreviated PCDAI), and Spearman correlation was used for ordinal variables (PGA, lab parameters) We further characterized the distribution of Short PCDAI scores across different PGA categories using means (s.d.), medians (IQ range), and box and whisker plots. In addition, we used receiver operating characteristic (ROC) curves to test the discriminant ability of the Short PCDAI and other indices to distinguish between inactive and mild disease and between mild and moderate-severe disease. Finally, we calculated the sensitivity and specificity of the Short PCDAI for discriminating between inactive and mild disease and between mild and moderate-severe disease at different cut-points.
All statistical analyses were performed using SAS version 9.2 (SAS Institute: Cary, NC). The study protocol was approved by all sites participating in the collaborative, and informed consent was obtained if required by institutional IRB.
This study included a total of 1355 subjects with Crohn’s disease, comprising a total of 6373 visits. The mean age was 14.2 years (SD. 3.3). Fifty-six percent were male, and the racial/ethnic distribution was as follows: 74% white, 10% black, 2 % Hispanic, and 14% not identified or other. The mean number of visits per patient was 4.5 (s.d. 3.6).
The PCDAI scores for each component of the index, as well as the proportion of visits for which each component could be assigned a score, is displayed in Table 1. All 11 components of the PCDAI were complete in only 1067 of the 6373 visits (16.7%). The abbreviated PCDAI was completed in 2811 out of 6373 visits (44.1%). PCDAI components completed in less than 80% of visits included albumin, hemoglobin, sedimentation rate, height velocity, and perirectal examination.
The distribution of PCDAI scores across the following categories of PGA: inactive, mild, moderate, and severe are displayed as box and whisker plots in Figure 1a. The mean (s.d.) PCDAI scores (0–100 scale) for patients with inactive, mild, moderate, and severe disease were 7.0 (7.1), 18.3 (10.8), 26.2 (11.4), and 40.5 (14.2), respectively. As shown in the figure and consistent with prior literature, there is overlap of PCDAI scores across PGA categories, but the PCDAI is a reasonable instrument to discriminate between PGA categories.
Retaining all elements of the abbreviated PCDAI with a feasibility ≥80% resulted in a short index containing the following components: abdominal pain, patient functioning/general well-being, stools, weight, abdominal exam, and extra-intestinal manifestations. In the training set, three items (patient functioning/general well-being, abdominal pain, and weight) were most strongly associated with the log-transformed total PCDAI score (standardized regression parameters approximately twice that of the other components), and considered as candidate items to receive a higher weighting. Based on this, a value of 20 points was selected for these three items (twice the weight of the other components) (Table 2). To validate this weighting strategy, we confirmed that 1) these components had the highest standardized regression parameters in the validation set, and 2) that the Pearson correlation between the re-weighted Short index and the log-transformed full PCDAI was equal to or greater than several alternative weighting strategies (data not shown). After repeating the training and validation procedures using only the first visit for each subject, we confirmed that the optimal component weighting did not change, suggesting that the weighting schema selected was not overly influenced by repeated measures within individual subjects.
Overall, the Short PCDAI could be completed in 4241 out of 6373 visits, resulting in feasibility 66.5% (compared with 16.7% for the full index and 44.1% in the abbreviated index). The correlation between the Short PCDAI and disease severity assessed by PGA was similar to that of the full PCDAI (r= 0.60, p<0.001 vs 0.61, p < 0.001). The correlation between the Short PCDAI and the log transformed PCDAI and Abbreviated PCDAI were 0.66 and 0.84 respectively. Correlations with laboratory indicators of disease activity are shown in Table 3. Box and whisker plots demonstrating the relationship between the Short PCDAI and PGA are shown in figure 1b. The mean (s.d.) Short PCDAI scores for patients with inactive, mild, and moderate to severe disease are 6.1 (8.7), 19.5 (13.3), 31.6 (17.1), and 47.3 (16.0), respectively. Additionally, the area under the curve values generated from receiver operating characteristic (ROC) curves demonstrating the discriminant ability of the Short PCDAI , PCDAI, and ABPCDAI in distinguishing between inactive and mild disease and between mild and moderate-severe disease are displayed in Table 4. Discriminatory capacity for the Short PCDAI is approximately the same as for the two expanded indices. Finally, Table 5 shows the sensitivity and specificity of the Short PCDAI in discriminating between inactive and mild disease and between mild and moderate-severe disease at different cut-points. For discrimination between inactive and mild, we recommend a cut-point of less than 15. For discrimination between moderate-severe and mild, we recommend a cut-point of ≥ 30.
In this large, multi-center study, we found that the completion of the Pediatric Crohn’s Disease Activity Index and the Abbreviated PCDAI in the context of routine clinical care was quite limited. The components of height velocity, laboratory data, and perirectal examination were the major barriers to completion of these indices. We therefore created a Short PCDAI by eliminating these components, and re-weighting the remaining components to maximize the correlation of the Short PCDAI and the PCDAI. As compared to the full index, this short index is more feasible to complete in the context of routine clinical care and has similar correlation to physician global assessment. The ability to discriminate between inactive and mild disease and between mild and moderate-severe disease is also similar to the full index.
Based on these properties, we believe the Short PCDAI is a promising tool for quality improvement and observational research. It should allow providers to track their patient outcomes over time in a more objective fashion, study how changes in their practices impact patient outcomes, and even allow comparisons to be made to national performance standards (i.e. benchmarking). In addition, this instrument should facilitate retrospective chart reviews and prospective registry studies. Given the myriad of financial and ethical issues, along with the practical issue of difficult recruitment, registries and other observational studies have been a mainstay of clinical research in pediatric IBD for decades. Until now, a significant limitation of such observational studies is the large amount of missing data resulting in possible informational bias when PCDAI was used as an outcome measure, or the inherent subjectivity incurred when Physician Global Assessment was the primary outcome. Hence, the Short PCDAI may optimize the balance between feasibly and objectivity that is necessary for observational research.
Our results regarding the limited ability to complete the PCDAI in the context of routine clinical care are consistent with prior published studies. Turner et al recently published a study indicating that the PCDAI could be completed in less than 50% of visits. The feasibility of the PCDAI reported in that study was higher than we report, perhaps owing to differences in study design and setting. Turner’s study included patients in the Pediatric IBD Collaborative Research Group, a select group of closely followed newly diagnosed children with IBD. In contrast, rather than a “research study” where careful procedures were used to optimize the completeness of data collected on a limited number of selected patients at selected center, this was a study of actual clinical practice. Participating centers sought to enroll 100% of their patients with Crohn’s disease, and the data in the ImproveCareNow database reflect that captured during routine clinic visits. Therefore, it is not surprising that the completeness of the different clinical indices reported in this study are lower than that reported in other studies. In fact, this underscores the difference between the feasibility of completing an index for a specific research study and the feasibility of completing an index in actual clinical practice.
In our study, we also observed variation in the completeness of PCDAI components across centers. This study included 15 centers, with the largest center contributing approximately 25% of included patients. Therefore, it is possible that this and other large centers might have heavily influenced our results. However, the diversity of sites in the ImproveCareNow collaborative reflect the settings in which pediatric IBD care is delivered in the real world (large and small centers, private and academic practices, etc.)
Our short index differs from the APCDAI in that it does not require the perirectal examination. Prior work has suggested that this component helps discriminate between remission and recurrence with a sensitivity of 32% and a specificity of 98%. We chose to eliminate this item from our short index because this component of the physical examination is neither indicated nor routinely completed at all outpatient visits. Furthermore, inclusion of this element did not substantially improve the correlation with the full index. Thus, we believe the overall benefit of increased feasibility achieved by eliminating the perirectal exam component is worth this minor trade-off.
It was not possible to select an a. priori threshold for which items to retain in the Short PCDAI. Rather, this threshold was decided after examining the distributions of responses for each component item. If we included completed at less than 80% of visits, the ability to complete the Short PCDAI during routine clinical care would fall to less than two-thirds, greatly limiting the utility of the index. Had we required higher completion rates, this would have eliminated items which are clinically very important and components of most other indices for pediatric and adult IBD (stools, general well-being). Hence, this threshold was selected in order to balance completeness and clinical relevance.
As with the parent PCDAI, the ability of the Short PCDAI to discriminate between inactive and mild disease and between mild and moderate-severe disease is somewhat limited (Figure 1A and B). This is in part due to the fact that Physician Global Assessment is not a true “gold standard”. Although used as a gold standard in this and other studies evaluating clinical indices in IBD, the subjectivity of this reference limits the ability to actually measure sensitivity and specificity. In a recent evaluation of the PCDAI, the sensitivity and specificity in distinguishing inactive from mild disease was 0.81 and 0.68 respectively, and in distinguishing between mild and moderate-severe was 0.71 and 0.83 respectively. In the present study, the discriminatory capability of the Short PCDAI was similar, though slightly decreased. This is perhaps due to the fact that PGA was obtained from a large number of practicing clinicians across 15 centers. Although this is a major strength of this study, to the extent that assignment of PGA varies across providers, the sensitivity and specificity of both the full and short index will be diminished. Therefore, the values reported here likely represent a worst-case scenario. In summary, existing pediatric Crohn’s disease activity indices including the PCDAI and APCDAI are of limited value during routine clinical care, as many of the required elements are not always collected and recorded during patient encounters. By retaining and re-weighting the most frequently collected items, we have developed a Short PCDAI that is easier to complete at the point of care, has excellent correlation to the full index, and appears to have a similar ability as the PCDAI to discriminate between different levels of disease activity as measured by Physician Global Assessment. We expect this standardized classification tool will facilitate observational research, quality improvement, and clinical care.
Study Support: Dr. Kappelman was supported in part by the National Center for Research Resources (NCRR) Grant KL2 RR025746 and the National Institute for Diabetes and Digestive and Kidney Diseases Grant P30 DK034987. Support for this analysis was also provided by the the Agency for Healthcare Research and Quality grant HS 016957.
Conflict of Interest: No authors have a conflict of interest to disclose