|Home | About | Journals | Submit | Contact Us | Français|
Factors contributing to patient-reported experiences of diabetes self-management support are not understood well, particularly over time.
To identify the contribution of patient characteristics to patient-reported quality of SMS.
Using secondary data from a prospective clinical trial (n = 339) comparing three approaches of providing diabetes self-management support (Group Medical Visits, Automated Telephone Support, and Usual Care) in a diverse, underserved population, the influence of patient characteristics (e.g., age, gender, income, health status) was examined on Patient Assessment of Chronic Illness Care ratings.
At baseline, older age (p = .014), being female (p = .038), and having lower income (p = .001) were associated with lower ratings. Income and interactions involving income combined explained 12% of the variance in baseline ratings. Compared to White patients, African American and Asian patients tended to have higher baseline ratings (p = .076 and p = .045, respectively). Race or ethnicity influenced perceptions throughout the trial, explaining 5% of the variance at baseline and 2% of the variance in one-year changes in Patient Assessment of Chronic Illness Care (PACIC) ratings. As expected, over 1 year, ratings increased more for patients in both intervention groups compared to the control group (p < .001).
Ratings of health care quality are influenced by patient characteristics independent of the nature of the care provided. Understanding more precisely how these differences are associated with differences in clinical processes will be particularly important for efforts aiming to integrate patient-reported measures into assessments of health care quality during routine clinical care and clinical trials.
Despite advances in the treatment of diabetes, few patients reach outcomes recommended in clinical guidelines (American Diabetes Association [ADA], 2012). Racial minority status, less education, lower literacy skills, and lack of insurance are associated with both a higher prevalence of diabetes as well as with poorer diabetes-related outcomes (Golden et al., 2012; Miech, Kim, McConnell, & Hamman, 2009; Trivedi, Grebla, Wright, & Washington, 2011). In recent years, it has been proposed that a key to understanding diabetes-related health disparities may lie in recognizing that, like many chronic illnesses, diabetes requires patients to integrate numerous actions (e.g., medication management, glucose testing, regular appointments, diet and exercise) into their daily lives. However, patients are often not taught the skills important to self-management of diabetes such as interpreting medical information, applying practical information, setting realistic goals, and problem-solving (Vaccaro et al., 2012); it is estimated that less than 50% of patients with diabetes have attended even one formal diabetes self-management class (Centers for Disease Control and Prevention, 2009). The same vulnerable populations who experience the poorest diabetes-related outcomes are also those who experience access barriers to diabetes self-management education (Anderson, 2008; Nwasuruba, Osuagwu, Bae, Singh, & Egede, 2009).
Efforts to close gaps in diabetes outcomes have been focused largely on redesigning healthcare delivery systems to improve the quality of clinical care and better serve the needs of those with chronic illness. There are multiple models informing such efforts (e.g., the adoption of Patient Centered Medical Homes); however, most are related to the Chronic Care Model (CCM), in which chronic illness care can be improved through proactive, coordinated service delivery to patients with chronic illness. The CCM outlines methods of providing support to both healthcare providers through delivery system redesign (decision support and clinical information systems) and to patients through self-management support (goal-setting, close follow-up, and links to community resources; Bodenheimer, Wagner, & Grumback, 2002; Wagner, Glasgow, et al., 2001). The CCM, as well as a wide array of similar disease management programs have resulted in improved outcomes for those with chronic conditions, including those with diabetes (McEvoy & Barnes, 2007; Patient-Centered Primary Care Collaborative, 2010; Yu & Beresford, 2010). In addition, there is evidence that self-management support and associated system changes modeled after the CCM result in improved outcomes for vulnerable populations, particularly racial or ethnic minorities and those with limited literacy skills (Rothman et al., 2004).
Because measuring how well practices align their efforts with those articulated in the CCM is imperative to understanding the effects of the CCM on patient health outcomes, researchers have created two instruments, the Assessment of Chronic Illness Care (ACIC) and the Patient Assessment of Chronic Illness Care (PACIC). Most studies on the effectiveness of the CCM in improving patient outcomes have used the ACIC, an organizational-level measure completed by healthcare providers assessing how well their clinical services align with key aspects of the CCM (ADA, 2012; Bonomi, Wagner, Glasgow, & VonKorff, 2002). In contrast, the PACIC is a patient-reported measure of whether a number of activities seen as particularly supportive of disease self-management were completed during healthcare visits, including being given adequate information, experiencing open communication, and setting behavioral goals (Glasgow et al., 2005). In validation studies of those with diabetes, PACIC scores were associated positively with the quality of diabetes care (receipt of diabetes-related laboratory tests and behavioral counseling), and were unassociated with patient characteristics (gender, race or ethnicity, income, and condition; Glasgow, Whitesides, Nelson, & King, 2005; Schmittdiel et al., 2008). These findings suggest the utility of the PACIC in measuring the quality of self-management support offered to patients during clinical encounters.
The PACIC has been adapted in multiple languages, and has been incorporated in a wide range of studies examining the quality of chronic illness care delivered in clinical settings. While the focus on these studies has been on using the PACIC to document practice improvements related to the quality of patient self-management support, variability in PACIC scores by race or ethnicity (higher among non-Whites) and education (higher among those with less education) has been documented suggesting that, in the particular system studied, non-Whites and those with less education receive more self-management support (Jackson, Weinberger, Hamilton, & Edelman, 2008). These findings correspond with others who have found that patient satisfaction varies by age, race, and gender (Nicolucci et al., 2009; Xiao & Barber, 2008).
In contrast to others’ experiences with the PACIC, a 2010 study concerning a cross-sectional sample of 208 diabetes patients seen in an academic internal medicine practice demonstrated that the PACIC was unassociated with objective measures of clinical intensity (i.e. involvement by a multidisciplinary team, care coordination and navigation, telephone follow-up). Rather, patients’ health literacy was the only variable associated with PACIC ratings (Wallace, Carlson, Malone, Joyner, & Dewalt, 2010). Similarly, another 2010 study found that the measurement properties proposed by PACIC developers could not be confirmed with objective clinical laboratory information (Gugiu, Coryn, & Applegate, 2010). This study is a secondary analysis of data collected during a 3-arm randomized clinical trial testing the effectiveness of two different approaches, group medical visits (GMV) and Automated Telephone Self-Management (ATSM), for supporting diabetes self-management in a vulnerable and diverse patient population (Handley, Hammer, & Schillinger, 2006; Schillinger et al., 2008; Schillinger, Handley, Wang, & Hammer, 2009). In the GMV approach, a group process is used to provide support, education, and patient activation and has been shown to improve self-efficacy and functional status among selected patients with chronic diseases (Lorig et al., 2001; Sadur et al., 1999). In the ATSM approach, technology is used to provide surveillance, education, and patient activation and has been linked with improvements in satisfaction and functional status (Piette, Weinberger, & McPhee, 2000). Both approaches are rooted in self-efficacy theory, share objectives characteristic of patient-oriented care (Fisher et al., 2005), and promote the use of short-term self-management goals (Fisher et al., 2005; Peek, Cargill, & Huang, 2007). Both intervention models were delivered in English, Spanish, and Cantonese.
The intervention study has been described previously (Handley, Hammer, & Schillinger, 2006; Schillinger et al., 2008; Schillinger, Handley, Wang, & Hammer, 2009), and was part of an initiative to improve diabetes care by the University of California, San Francisco Collaborative Research Network, a primary care practice-based research organization. Key findings of the trial included improvements in the PACIC and in reported self-management behaviors for both intervention groups, and improvements in interpersonal processes of care for the ATSM group. The ATSM group also had fewer reported bed days per month than the usual care group, and the GMV group reported less interference with daily activities than the usual care group (Schillinger et al., 2009).
Patient preferences of diabetes self-management support are influenced by a number of characteristics, including social isolation (Banerjee, Perry, Tran, & Arafat, 2010), race, and ethnicity (Jack, Toston, Jack, & Sims, 2010) and led to some confusion regarding whether the PACIC documents objective differences in clinical care, or reflects different expectations, experiences, or needs related to self-management support on the part of patients. Incorporating patient preferences is important to efforts aiming to improve healthcare quality. Better understanding of the influence of sociodemographics on patient-reported outcomes is important because variability among demographic groups may have important implications for policymakers seeking to incorporate patient-reported data in evaluations of health plans, health systems, and individual clinicians serving diverse patient populations.
Although most studies incorporating the PACIC use it as a means of assessing practice improvements over time, studies of the influence of patient characteristics on PACIC ratings have been done on cross-sectional data (Wallace, Carlson, Malone, Joyner, & Dewalt, 2010). The objective of the current study was to examine the potential influence of patient characteristics on PACIC ratings over time using secondary data collected during a 3-arm trial to test methods of delivering diabetes self-management support in a diverse patient population. Specifically, using multivariate modeling, the influence of age, gender, race or ethnicity, language, birthplace (foreign vs. United States birth), marital status, education, health literacy, health insurance status, income, employment, duration of diabetes, social network, self-reported health status, A1C, and BMI were studied on both baseline and 1-year changes in PACIC ratings.
Multivariate modeling was used on a number of participant characteristics at baseline and 1-year changes in PACIC ratings using secondary analyses of data. Intervention group assignment and baseline PACIC Summary ratings were added to the model for 1-year changes in PACIC ratings as variables potentially influencing those changes during the clinical trial.
Eligible patients included those with a diagnosis of type 2 diabetes; who spoke English, Spanish, or Cantonese; who made one primary care visit in the prior year; and having a most recent A1C of >8.0%. Exclusion criteria included those who had moderate to severe dementia, or who were not expected to live through the year. In addition, patients were excluded if they anticipated travel of 3 months in the upcoming year, were too ill or unable to travel to a GMV, were without telephone access, reported hearing impairment, had visual acuity of 20/100, or were unable to follow instructions on a telephone keypad (Schillinger, Handley, Wang, & Hammer, 2009).
During 2004, participants attended a study enrollment visit at the San Francisco General Hospital Clinical Research Center. Informed consent was obtained after a language-concordant document written at the 6th-grade level was read to potential participants. Participants were then allocated to the intervention groups using stratified (on languages) blocked randomization.
Participants assigned randomly to ATSM received weekly, automated (prerecorded) telephone calls over 39 weeks (9 months). Participant responses triggered either immediate, automated health education messages; subsequent nurse phone follow-up; or both. The GMV arm involved 90-minute monthly sessions over 9 months, with 6 to 10 participants, cofacilitated by a primary care physician and health educator. Participants were encouraged to see their regular healthcare provider as usual (Schillinger, Handley, Wang, & Hammer, 2009).
The study was approved by the institutional review board of the University of California, San Francisco. The secondary analysis described here was deemed exempt by the institutional review board of the University of Iowa.
All self-reported study variables were derived from a survey that was developed in English, translated into Spanish and Cantonese, and back-translated in an iterative fashion to achieve concordance in meaning. The survey was tested for face validity among the non-English speakers; no difficulties in completing the survey among the different language groups were reported. Trained research assistants administered the survey verbally to participants at enrollment and 1 year after enrollment. In addition, A1C was measured (high performance liquid chromatography method; Bio-Rad, Hercules, CA) and BMI was calculated, again, at enrollment and 1 year after enrollment.
Based on a review of the literature, patient characteristics potentially associated with ratings of self-management support were age, gender, race or ethnicity, language, birthplace (foreign vs. US birth), marital status, education, health literacy, health insurance status, income, employment, duration of diabetes, social network, and self-reported health status. The A1c and BMI variables were included also as candidate covariates because of their potential effect on patient perceptions and because of the known variability in diabetes outcomes according to the sociodemographic variables being explored, whereas intervention group (GMV, ATSM, usual care) and baseline PACIC Summary ratings were added as variables potentially influencing 1-year changes in patient ratings during the clinical trial.
Although income was reported initially using the following categories: ≤ $5,000, $5,000–10,000, $10,001–20,000, $20,001–30,000, and >$30,000, because the majority of participants (60%) reported annual incomes at or below $10,000, this cut-off was selected for the dichotomous variable used to represent income in this study. Likewise, because of the high number of uninsured participants (50%), insurance status was represented by a dichotomized variable created by combining private insurance, Medicaid or MediCal, and Medicare into the insured category. Finally, education, which was reported originally on a scale from 1 = Never went to school to 8 = Graduate degree, was combined into three categories: No high school diploma, High school graduate/GED, and Some college/technical school and above.
Literacy was assessed using the shortened version of the Test of Functional Health Literacy in Adults (S-TOFHLA; Baker, Williams, Parker, Gazmararian, & Nurss, 1999). The S-TOFHLA is a reading comprehension test commonly used in healthcare settings, with scores ranging from 0 to 36. It is correlated highly with tests used in general education, such as the revised Wide Range Achievement Test (r = 0.74) and with scores on the Rapid Estimate of Adult Literacy in Medicine (r = 0.84; Davis et al., 1993). Clinically meaningful criteria have been established for using the S-TOFHLA to classify patients’ health literacy as inadequate (score of 0 to 16), marginal (score of 17 to 22), or adequate (23 to 36). The inadequate and marginal categories were merged because of existing evidence suggesting that the effects of low health literacy persist into the marginal category (Dewalt, Berkman, Sheridan, Lohr, & Pignone, 2004).
The participant questionnaire included three questions to assess social networking. The questions were: (a) How often do you visit friends and relatives? (b) On average, how many different homes of friends or relatives do you visit per month? and (c) How many people usually come to see you or call you per day? Participants were asked to answer these questions on a scale from 1 to 6, with 1 meaning never or none and 6 meaning several times a week or more than 10, as appropriate. An overall network support score was calculated for each participant by averaging replies to the three questions; higher scores indicate higher levels of social networking and vice versa. These 3 questions were combined to create a measure of social networking because they are concerned with the amount of interactions with friends, family and visitors. No validation studies have been conducted for this instrument; however, the Cronbach’s alpha for the 3 items was .54, indicating acceptable reliability.
Health status was assessed using a single question, asking participants to rate their health as poor, fair, good, very good, or excellent. The item was scored on a scale from 1 to 5, with 1 corresponding to poor and 5 to excellent health. The question is used widely in population surveys and, more recently, in clinical research in which it is associated highly with other health status measures, and is predictive of mortality and health behaviors (Rohrer, Herman, Merry, Naessens, & Houston, 2009). For ease of interpretation, a dichotomized health status variable was created by combining categories of the self-reported health status (Poor and Fair vs. Good, Very Good, and Excellent).
The primary outcome measure for these analyses was the PACIC. The PACIC is a questionnaire consisting of 26 items with responses ranging from almost never (0) to almost always (5), measuring the extent to which care received aligns with the CCM (Glasgow, Wagner et al., 2005; Glasgow, Whitesides, et al., 2005). It includes a summary score (mean of 20 items) and five subscales: Patient Activation (e.g., Asked for my ideas when we made a treatment plan), Delivery System Design/Decision Support (e.g., Given a written list of things I should do to improve my health), Goal Setting (e.g., Asked to talk about my goals in caring for my illness), Problem-Solving/Contextual Counseling (e.g., Helped make a treatment plan that I could do in my daily life), and Follow-Up/Coordination (e.g., Contacted after a visit to see how things were going). Subscale scores are calculated as means of the items within each subscale. The summary score is the mean of items 1 to 20. Items 21 to 26 are used to associate the PACIC with the organizational-level ACIC (Bonomi et al., 2002), which was not addressed in this study.
Studies have demonstrated that the PACIC is internally consistent (α = .93), demonstrates test-retest reliability (r = .58 over 3 months), and is correlated to patient activation and primary care measures (r = .32–.60; Glasgow, Wagner, et al., 2005; Glasgow, Whitesides, et al., 2005). For ease of interpretation, and to remain consistent with how results of the clinical trial serving as this study’s data source were reported, PACIC Summary scores were transformed to a 100-point scale, with higher scores representing higher ratings of self-management support and greater alignment with the tenets of the CCM.
One-year changes in PACIC ratings were calculated by subtracting the summary score at baseline from the summary score at the one-year follow-up. The change score was believed to capture variability best in patient-reported experiences during the 1-year trial comparing the three different self-management support strategies (GMV, ATSM, and usual care).
In the first stage of the analyses, bivariate relationships between potential predictors and outcomes were examined using Wilcoxon two-sample or Kruskal-Wallis test statistic for the categorical variables (gender, race or ethnicity, language, foreign vs. US birth, marital status, education, health literacy, health insurance status, income, employment, and health status) and the Pearson correlation coefficient for the continuous variables (age, duration of diabetes, social network, A1C, and BMI). Bivariate relationships among the predictors were checked for potential collinearity.
Variables related to the outcomes at α < .15 were considered in preliminary models, as well as interactions among these variables and with other potential predictors. Predictors for the final models were selected using (a) the least angle regression algorithm (LARS) with the conceptual predictive statistic (Cp) and (b) the F-test statistic for each candidate predictor, or interaction. The LARS is a variable-selection algorithm appropriate when the number of variables is large relative to the sample size (Efron, Hastie, Johnstone, & Tibshirani, 2004). Included in the final models were predictors and interactions that were statistically significant at α < .05, and also variables that were part of a statistically significant interaction.
The trial involved 339 patients. The mean age of participants was 56.1 years (SD = 12.0, range = 24–84). The sample was racially and ethnically diverse (23% Asian, 21% African American, 47% Latino, and 8% White) and came from severely depressed economic backgrounds (60% making ≤ $10,000 annually); the majority had limited health literacy (59%), were non-English speakers (55%), and rated their health as only poor to fair (76%). On average, participants had poor glycemic control (mean A1C = 9.5%, SD = 2.0, range = 5.2–16.3%), were obese (mean BMI = 31, SD = 7.5, range = 15.7–64.2), and had a great deal of experience with their diabetes (mean duration = 9.6 years, SD = 7.4, range = 1–35 years; Table 1) (Schillinger, Handley, Wang, & Hammer, 2009).
At baseline, PACIC ratings decreased with age (r = −.116, p = .033), and increased with the duration of diabetes (r = .131, p = .016), and with higher levels of social network (r = .215, p < .001; Table 2). Lower 1-year increases were associated with higher baseline ratings (r = −.545, p < .001). The p-values for Wilcoxon two-sample or Kruskal-Wallis tests for the categorical variables are shown in Table 1.
Bivariate associations revealed age, gender, race or ethnicity, language, birthplace, education, income, duration of diabetes, and social network as potential predictors of baseline PACIC ratings; while gender, race or ethnicity, language, education, health literacy, and intervention group assignment, as well as baseline PACIC rating, were identified as potential predictors of 1-year changes in PACIC ratings. Baseline A1C and BMI were not associated with either baseline PACIC ratings or changes in PACIC ratings, and were not included in the multiple regression models. No collinearity was detected among the variables; all potential predictors, as indicated by their statistically significant association with PACIC ratings, were considered for inclusion in the models.
Final multiple regression models are reported in Table 3 (PACIC ratings at baseline) and Table 4 (1-year changes in PACIC ratings). The models were statistically significant (p < .001). The model for baseline PACIC ratings explained 18% of the variance in the outcome, and the model for PACIC rating changes explained 36% of the variance in the outcome.
Age was related negatively to baseline PACIC ratings (b = −0.321, p = .014), while being a female and having lower income was associated with lower PACIC ratings (b = −6.310, p = .038, and b = −44.781, p = .002, respectively). Compared to White patients, African American and Asian patients tended to have higher PACIC ratings (b = 14.646, p = .076, and b = 16.024, p = .045, respectively), although the difference between White and African American patients was not statistically significant at the .05 level. Table 3 also reports regression coefficients for interactions between income and duration of diabetes, race or ethnicity, and social network. These coefficients indicate how the associations between baseline PACIC rating and the predictors varied for the two income groups (≤ $10,000 and > $10,000). For example, one unit increase in social networking was associated with almost a 12-point increase in baseline PACIC rating for patients with lower incomes, while no such increase in PACIC ratings was associated with social networking for patients with higher incomes, suggesting that social network meant more to patients with lower incomes.
Results for the model for changes in PACIC ratings indicate that, as expected, PACIC ratings increased more for patients in both intervention groups, compared to the control group (b = 12.551, p < .001, and b = 12.786, p < .001; Table 4). Although no race or ethnicity group differed from White patients, Latino and African American patients increased their ratings more than did Asian patients (b = 12.108, p = .008, and b = 12.178, p = .016, respectively) in the model with race or ethnicity recoded to have Asian patients as the comparison group.
Semipartial correlations reported in Tables 3 and and44 show approximate percentage of the explained variance for each predictor, with other predictors already in the model. For example, age explains approximately 2% of the variance in baseline PACIC ratings, and race or ethnicity explains 5% of the variance in baseline PACIC ratings and 2% of the variance in PACIC rating changes. Adding income and all interactions with income to the model that already includes other predictors explains additional 12% of the variance in baseline PACIC ratings.
Differences in perceptions of self-management support at baseline were related to a number of patient characteristics (age, gender, income, race or ethnicity). Further, participants’ social network somewhat mitigated the negative effects of income on PACIC ratings for lowest income patients (annual income ≤ $10,000). As expected, the changes in perceptions throughout the 1-year trial were influenced by assignment to one of the two intervention groups. However, patient characteristics continued to contribute to variability in 1-year changes both indirectly, through baseline ratings (higher baseline ratings associated with less change), and directly, through differences in mean changes between Latino and African American versus Asian patients. No interaction was found between the self-management support models and patient characteristics, indicating that the approaches to self-management support explored in these data were not rated differently according to the variables evaluated.
The racial or ethnic variability in PACIC ratings at baseline largely confirm reports by other researchers that patient satisfaction ratings are influenced by a number of sociodemographic characteristics (Fan et al., 2005; Nicolucci et al., 2009; Xiao & Barber, 2008). However, the interaction between income with race or ethnicity and social networking are unique to this study and may have implications for delivering self-management support to medically underserved and vulnerable patients. However, the negative effect of income was mitigated by the degree of social networking reported by patients with extremely low incomes. This finding confirms work that has identified the importance of social support in health outcomes, particularly for those with chronic illness (Rosland, Heisler, Choi, Silveira, & Piette, 2010), and may reflect the importance of social networking on how patients perceive and, possibly, engage in self-management efforts. Accessing and capitalizing on patients’ social networks may be an area in need of additional exploration, particularly for those aiming to improve the self-management support for this impoverished population. Additionally, there were differences in how Latino patients rated their care between the higher and lower incomes levels, suggestive of the interaction between financial resources and culture. While these findings need to be cautiously interpreted, they add to the growing evidence that race and ethnicity are complex social constructs rather than a variable to simply be addressed through the provision of culturally guided care.
In a recent paper, the PACIC was not related to the intensity of services delivered in one healthcare setting but, rather, was related to patient health literacy (Wallace et al., 2010). As a result, an important question is whether the PACIC – and probably other patient-reported quality measures – accurately reflect differences in healthcare quality, or if they are a measure of other unidentified patient preferences, states, or traits. While these data demonstrate that the PACIC is responsive to practice changes, characteristics inherent to patients remain an important factor in how they rate the quality of their care. As health policy efforts continue to move forward with integrating patient measures into ratings of healthcare quality, it may be imperative to consider how those caring for different patient populations (particularly those caring for medically underrepresented and underserved patients) may be impacted by incorporating quality measures that are heavily influenced by patient characteristics.
The interpretation of results is limited by disadvantages of secondary data, particularly related to the newly created social networking variable. Because the sample size did not allow for subgroup analyses within intervention groups (i.e., ATSM vs. GMV), and because both of these interventions were derived from Bandura’s (1997) theory of self-efficacy, understanding is limited regarding whether either method of delivering SMS may be more or less appropriate for different patient populations (e.g. different racial, language groups). However, these data did allow for a robust analysis of how patient characteristics may influence PACIC ratings and serves as an important reminder that, as healthcare systems continue to focus on improving efficiency and effectiveness through reorganizing clinical care, multiple factors may influence how patients interpret such efforts. These findings reinforce the need to continue exploring how patient ratings are associated with other measures of health care quality, both objective and subjective, and what is the contribution of each to disease-related outcomes.
Funding for this study was provided to Dr. Wallace by the National Institutes of Health grant nos. T32 NR08856 and P20 NR8369. Additional support for Dr. Wallace’s time was also provided by The Robert Wood Johnson Foundation, Nurse Faculty Scholars Program, grant no. 68031. Support for Dr. Schillinger is provided by National Institutes of Health grants P30 DK092924-01 and UL1 RR024131.
There are no known conflicts of interest related to the studies described in this manuscript.
Andrea Wallace, The University of Iowa College of Nursing, Iowa City, Iowa.
Yelena Perkhounkova, The University of Iowa College of Nursing, Iowa City, Iowa.
Huichen Tseng, The University of Utah College of Nursing, Salt Lake City, Utah.
Dean Schillinger, University of California at San Francisco School of Medicine Center for Populations, San Francisco General Hospital, San Francisco, California.