Rothman et al. developed and validated the SKILLD in 217 low income, poorly educated, low literacy patients with poorly controlled diabetes (average A1c was 10.8%).44
They found that the SKILLD had good reliability and validity in measuring diabetes knowledge in those with type 2 diabetes and low literacy. They also demonstrated that individuals with low literacy or lower economic status had poorer knowledge of diabetes, and that the results of the SKILLD was associated with metabolic control (that is, the higher the score of the SKILLD, the better the A1c).
Our construct validity analysis leads us to believe that health literacy and education level correlate well with SKILLD score independent of one another. Rothman et al. reported similar findings. This is indicative of a good knowledge test. Other portions of our analysis raise some noteworthy questions.
We analyzed the criterion validity using Pearson’s coefficient in which we evaluated how well the SKILLD predicted knowledge compared to the oral DKT. Previously two of the authors (WFM and KMJ) observed that the oral DKT evaluates knowledge as well as the written DKT (unpublished, 2009). The results of the current analysis demonstrate an only moderate correlation between the knowledge evaluated by the SKILLD and that evaluated by the oral DKT. This shows that, while they are similar, the SKILLD and the oral DKT are not the same test. It does not, however, indicate that one test is superior to the other, a point brought up in the original paper, which had no criterion validity measure. The SKILLD may be preferred because of its lower perceived difficulty, the practical nature of the questions, and its coverage of general concepts about diabetes management.
Our content validity analysis revealed a separate shortcoming that should be addressed. The domain of knowledge provided by the NDEP website was not sufficiently tested in the SKILLD, which likely led to test items that had little to do with one another resulting in a lower inter-item relatedness (Cronbach’s Alpha
0.54). Rothman et al. attempted to create a valid short version of the SKILLD due to time constraints of administering a knowledge test in a busy clinical practice.44
Our analysis suggests that the SKILLD may require more questions to fully evaluate the full set of knowledge available on the NDEP website. Attempting to shorten the test may miss useful information. Rothman et al. make the same point, noting that the SKILLD is more skill-related and lacks questions related to nutrition or medication.
Our conclusion about the deficits observed in criterion and content validity is that 1) the format and style of questions in the SKILLD are more practical than those in the DKT and may be an excellent way to test for DRK, but 2) the SKILLD does not have enough questions to adequately assess all domains of DRK. Consideration should be given to expanding the SKILLD into these other domains. Modifying question # 3 could be done as part of this process, since patients may omit “checking blood sugar” as they may not realize this to be a “treatment”.
Our regression analysis indicates that certain variables appear to predict DRK as measured by the SKILLD score. The results support previous findings that patients with a higher education level, health literacy, and income, and use insulin are more likely to have a higher SKILLD score.44
Previous findings found that duration of disease, younger age, and lower HbA1c were also related to higher SKILLD scores.44
We did not investigate a relationship between the SKILLD score and clinical outcomes as our aim was to determine if the SKILLD is a good knowledge test, independent of whether it is associated with metabolic control. Of note, our crude analysis identified a relationship between “non-black race” and DRK, but our regression analysis did not identify race as a predictor of DRK. We, therefore, conclude that race may have an association with another variable that independently correlates with DRK.
Most previous studies have shown a relationship between diabetes education and improved metabolic control.8–16
Our current investigation used schooling level as a measure of education, as did Rothman et al.44
A relationship between education in a disease management program and highest level of education attained may exist, but our literature search yielded no studies that drew direct comparisons. While further investigation is necessary to elucidate such a relationship, our analysis suggests that individuals with a higher education level have more DRK than those with a lower education level. Disease management programs may be used as a tool to increase DRK among those with lower health literacy. However, it is important to note that educational interventions increase knowledge of disease for all participants, and those with lower health literacy are unlikely to experience increases in disease knowledge sufficient to “catch up” to those with higher levels of health literacy enrolled in the same program.37
Current literature suggests that a higher level of health literacy27,37
are related to improved knowledge of disease as well as better disease outcomes.26,27,34,35
There is also an association between higher levels of DRK and many measurements of disease outcomes and self-care behaviors,4,6–8,16–20,36
however this association does not appear to be as strong,11,14,19,20,23,47
suggesting that the relationship between DRK and behaviors that lead to better self-care is not always predictable. “While adequate diabetes knowledge is a prerequisite to good self-care…proper knowledge does not always correlate with patient behaviors”.44
Our analysis demonstrated that higher levels of health literacy and education level independently correlate with higher levels of DRK. We did not attempt to demonstrate the link between DRK (i.e. SKILLD score) and HbA1c (a disease outcome) but we feel confident that the majority of the current literature (as well as our analysis) indicates that the model proposed in Figure is accurate (with the understanding that the relationship between DRK and behaviors may be affected by other variables as well).36
Construct of Diabetes Knowledge as it relates to Health Literacy, Education, and Behavior.
Our patient population was different than that of Rothman et al. who first introduced the SKILLD.40
Our population appears to have a higher level of income, education, and health literacy. While cross verifying a study in a different population is helpful in establishing test validity, neither of these populations was sampled to be nationally representative. Different conclusions may be drawn relating to the use of the SKILLD in private or rural practice, for example. Secondly, although our sample size was adequate to draw our conclusions, we recognize that a more powered study may have led to different conclusions, particularly regarding criterion validity of the SKILLD or other variables that may have predicted SKILLD score.