PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of bmjdrcBMJ Open Diabetes Research & CareVisit this articleSubmit a manuscriptReceive email alertsContact usBMJ
 
BMJ Open Diabetes Res Care. 2016; 4(1): e000182.
Published online 2016 June 29. doi:  10.1136/bmjdrc-2015-000182
PMCID: PMC4932248

The agreement of patient-reported versus observed medication adherence in type 2 diabetes mellitus (T2DM)

Abstract

Objective

Medication adherence in type 2 diabetes mellitus (T2DM) improves glycemic control and is associated with reduced adverse clinical events, and accurately assessing adherence assessment is important. We aimed to determine agreement between two commonly used adherence measures—the self-reported Morisky Medication Adherence Scale (MMAS) and direct observation of medication use by nurse practitioners (NPs) during home visits—and determine the relationship between each measure and glycated hemoglobin (HbA1c).

Research design and methods

We evaluated agreement between adherence measures in the Southeastern Diabetes Initiative (SEDI) prospective clinical intervention home visit cohort, which included high-risk patients (n=430) in 4 SEDI-participating counties. The mean age was 58.7 (SD 11.6) years. The majority were white (n=210, 48.8%), female (n=236, 54.9%), living with a partner (n=316, 74.5%), and insured by Medicare/Medicaid (n=361, 84.0%). Medication adherence was dichotomized to ‘adherent’ or ‘not adherent’ using established cut-points. Inter-rater agreement was evaluated using Cohen's κ coefficient. Relationships among adherence measures and HbA1c were evaluated using the Wilcoxon rank-sum test and c-statistics.

Results

Fewer patients (n=261, 61%) were considered adherent by self-reported MMAS score versus the NP-observed score (n=338; 79%). Inter-rater agreement between the two adherence measures was fair (κ=0.24; 95% CI 0.15 to 0.33; p<0.0001). Higher adherence was significantly associated with lower HbA1c levels for both measures, yet discrimination was weak (c-statistic=0.6).

Conclusions

Agreement between self-reported versus directly observed medication adherence was lower than expected. Though scores for both adherence measures were significantly associated with HbA1c, neither discriminated well for discrete levels of HbA1c.

Keywords: Adherence to Medications, Self-Reported Regimen Adherence

Key messages

  • In the clinical setting, medication adherence is difficult to assess due to the complexity of the medication regimen.
  • Agreement between self-reported medication adherence and nurse practitioner observed adherence ranked ‘fair’.
  • Higher adherence was significantly associated with lower glycated hemoglobin levels for both measures.

Introduction

Poor medication adherence is a public health threat that increases the risk of disability and death1 and disproportionately affects those with chronic illness,2 poor access to healthcare,3 and low health literacy.4 The cost of non-adherence in the USA falls between $100 billion and $289 billion annually.5 6 Efforts to improve adherence require valid and reliable measures that can be easily integrated into the existing workflow of real-world clinical settings. Using a self-reported medication assessment tool such as the Morisky Medication Adherence Scale (MMAS)7 has shown conflicting results in many chronic illness populations, including heart failure,8–10 hypertension,11 12 mental illness,13 and diabetes.14–16 In the USA, as the prevalence of type 2 diabetes mellitus (T2DM) rises to nearly 10% of the population,17 improving adherence to medications that affect glycemic control will be increasingly important to preventing debilitating complications and death. More evidence is needed to establish the validity and clinical utility of self-reported measures of medication adherence for patients with diabetes.

Research design and methods

In this study, we examined the level of agreement between patient-reported medication adherence using the MMAS and directly observed assessment of pills and injectables by nurse practitioners (NPs) during home visits. In addition, we sought to determine the ability of each of these two measures to discriminate for higher versus lower levels of blood glucose (glycated hemoglobin, HbA1c >7.5).

The cohort comprised high-risk patients (n=430) from four geographically diverse counties participating in the Southeastern Diabetes Initiative (SEDI): Durham County, North Carolina, USA; Cabarrus County, North Carolina, USA; Quitman County, Mississippi, USA; and Mingo County, West Virginia, USA. The study design and intervention of SEDI have been previously described.18 Participants were adults with a diagnosis of T2DM and defined as high risk for serious adverse events or death. The criteria used to define ‘high-risk’ for this study included baseline indicators of diabetes complications such as microvascular and macrovascular damage, renal and cardiovascular comorbidities, and social factors available in the electronic health record, including insurance and marital status. These factors were integrated with geographic information and used to create a geographic health information system (GHIS), which combined patient-level and community-level risks and was used to generate a ‘risk algorithm’.18 Using this algorithm-generated score, we identified the top 10% of the population as ‘highest-risk’ and invited these patients to enrol in the intensive intervention as represented here (n=430). Exclusion criteria included an inability to make health-related decisions, a terminal illness with a life expectancy of 6 months or less, a diagnosis of T1DM, or gestational diabetes or pregnancy.

In each county, multidisciplinary healthcare teams—including a physician, NP, dietitian, pharmacist, licensed clinical social worker, and community health assistant—provided care. Participants provided written informed consent to participate in the clinical intervention, consisting of a home visit that included a physical assessment, review of medications, and completion of patient-reported outcome surveys for medication adherence, nutrition, physical activity, diabetes care self-efficacy, and health behaviors. At each home visit, NPs administered the MMAS—a validated, 8-question survey that assesses self-reported medication adherence.7 NPs determined medication adherence using pill counts, medication bottle dates, and direct observation of insulin administration. Through these measures, NPs gave a categorical score to reflect adherence. Patients were assigned medication adherence scores of 0–20%; 21–80%, or >80%, reflecting the proportion of actual medications taken. Adherence >80% of the time was considered adherent, a method previously described.10 19

We used Cohen's κ coefficient to determine the level of inter-rater agreement between the two measures. We evaluated the association between adherence and blood glucose using a Wilcoxon rank-sum test. Finally, we calculated the c-statistic to assess the ability of each measure to discriminate between high and low blood glucose.

Results

Of 536 patients enrolled in the clinical intervention, 430 had complete data for baseline MMAS scores, observed adherence, and HbA1c values and were therefore eligible for analysis. There were no systematic differences between participants and those excluded for missing data. Descriptive results are shown in table 1. About half of the patients were categorized as adherent, though self-reported adherence (n=261, 61%) was lower than directly observed adherence (n=338, 79%) (table 2). The level of agreement between the two adherence measures was fair (κ=0.24; 95% CI 0.15 to 0.33; p<0.0001). The median number of medications was 6.0 (IQR 7.0). In addition, oral medications were associated with higher rates of adherence as compared with subcutaneous injections (χ2=6.88; p<0.009).

Table 1
Demographics of the baseline cohort (n=430)
Table 2
Correlation of baseline self-reported adherence (MMAS) versus directly observed assessment

HbA1c indicated uncontrolled blood glucose (HbA1c >7.5) in a majority of the cohort (n=350, 86%) (table 3). For both methods of assessment, higher adherence was significantly associated with lower HbA1c levels (p<0.001); yet, the ability of each measure to discriminate between lower and higher blood glucose was weak (c-statistics=0.63 and=0.61, respectively).

Table 3
Blood glucose control and self-report versus directly observed adherence

Discussion

In this large cohort of patients with high-risk diabetes across four geographically diverse counties of the Southeastern USA, the agreement between self-reported medication adherence and directly observed counts of pills, insulin pens, and medication bottles was only fair. Several possible conclusions may be drawn.

First, patients' perceptions of their medication-taking behavior may be inaccurate, confounded by the myriad of medications for which dosing instructions vary with life events, such as food intake, activity, or time of day. In this study, for example, 61% (n=261) of patients reported consistently taking medications as prescribed. Yet, of those, 31 people were actually considered non-adherent based on direct observation. Conversely, 40% (n=169) of patients reported poor adherence. Yet, of those, 110 people were doing better than they thought and were assessed as adherent by an NP. For many, this misperception of good versus poor adherence may simply be due to the high proportion of medications that are prescribed to be taken ‘as needed’ or in sliding scale doses. In previously reported studies, discrepancies in self-reported adherence have been attributed to the complexity of the medication regimen20 21 and the inherent difficulty in recognizing what adherence is.

Another interpretation of the results may be that in high-risk patients with T2DM, the MMAS may lack sensitivity and may be a poor indicator of actual medication use. Previous work with the MMAS in other populations has shown a strong correlation between self-reported adherence and actual adherence. In hypertension, for example, the MMAS has demonstrated a strong correlation between self-reported adherence and subsequent blood pressure control.7 12 Though early studies support the MMAS as being sensitive to actual changes in medication adherence, more recent studies have disputed these findings.11 22 23

In diabetes, numerous studies have evaluated self-reported medication adherence using the MMAS, some showing a positive relationship between adherence and HbA1c.24 25 In many of these studies, however, participant demographics differed widely from those in SEDI, with fewer comorbid illnesses, higher rates of insurance coverage, more frequent single-dose regimens, and higher levels of education and health literacy.16 In SEDI, factors that classified patients as high risk included recent hospitalizations, substance use, tobacco use, and multiple comorbidities—including coronary artery disease, hypertension, heart failure, or chronic kidney disease—all of which require complex medication regimens. As a result, patients may report that they are ‘getting enough medications’ daily, skewing self-reported results26 and suggesting that improvement in diagnostic measures is needed, particularly in illnesses with multiple comorbidities.

A second finding of this study was low discrimination. The c-statistics for both measures were similar, 0.63 for self-reported MMAS and 0.61 for direct observation. Though higher adherence scores were significantly associated with lower HbA1c values for both measures, neither was able to discriminate well between lower and higher levels of serum glucose control as indicated by HbA1c. Similar findings have been previously reported.11 16 One possibility is that other factors, beyond medication usage, are driving A1c in this high-risk population.

Regardless of the underlying reasons for lack of agreement between self-report and observed counts, every effort must be made to discover where, in this high-risk population, the breakdown occurs. Despite only fair agreement between self-report and observed medication counts, using the MMAS as a screening tool could improve identification of those highest risk patients in need of follow-up. The MMAS offers a quick survey that can be used in inpatient and outpatient as well as individual and group settings, and is useful for identifying those who need further evaluation and support for medication management. The individual reasons for non-adherence were not included as part of the analysis, as our focus was to evaluate and objectively report the agreement between patient-reported adherence and the observed assessment of a healthcare provider who was visualizing and counting bottles, vials, and pills. Though reasons for medication-taking behavior are critical in an intervention, this analysis was designed to provide evidence for the use of preintervention baseline data for determining the value of patients' self-reported medication adherence. Using these results substantiates the value of a baseline adherence assessment, and opens a forum for patients and providers to then earnestly consider reasons for various medication-taking behaviors and take these into account as part of the subsequent plan of care.

Limitations to this study include the following: a relatively long intake visit and placement of patient-reported outcome surveys, including the MMAS, at the end of the visit when patients may have been fatigued could affect results; patients took the self-reported survey while the healthcare member was present, which may have imposed an inadvertent Hawthorne effect; and a relatively high proportion of the cohort was unable to self-administer the survey due to blindness or very low health literacy. Though NPs read the survey for patients in the latter category, the results may have been skewed.

Conclusion

With 29 million diagnosed and undiagnosed people with diabetes,17 providing a brief but accurate assessment of medication adherence is a requisite tool for clinical practice. This study suggests that in chronic illnesses with multiple comorbidities in which a complex medication regimen is required, identifying more sensitive and discriminatory measures of medication adherence is necessary.

Acknowledgments

The authors wish to thank Charlotte Nelson (Duke Center for Predictive Medicine) for her work on the initial analysis, Shelley Rusincovitch (Duke Translational Research Institute) for her insight into the data elements selected, Melodie Staton (Community Care of North Carolina) for her work in cleaning the data and Pamela Barth (Duke Clinical Research Institute) for her facilitation of the final paper.

Footnotes

Contributors: The statistical authors (BAG, AW, and MVG-S) also contributed to the analysis. The clinical team (SES, BBG and KK) contributed to data collection and validation. All authors contributed to the writing and review of the final manuscript. AW and MVG-S had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. BBG is the guarantor of this manuscript.

Funding: The projects and the work described are supported in part by grant number 1C1CMS331018-01-00 from the Department of Health and Human Services, Centers for Medicare & Medicaid Services, and in part by the Bristol-Myers Squibb: Together on Diabetes programme. These contents are solely the responsibility of the authors and do not necessarily represent the official views of the US Department of Health and Human Services or any of its agencies.

Competing interests: None declared.

Ethics approval: Ethics committee(s) and institutional review board(s).

Provenance and peer review: Not commissioned; externally peer reviewed.

Data sharing statement: No additional data are available.

References

1. Zullig LL, Peterson ED, Bosworth HB Ingredients of successful interventions to improve medication adherence. JAMA 2013;310:2611–12. doi:10.1001/jama.2013.282818 [PubMed]
2. Mackey K, Parchman ML, Leykum LK et al. Impact of the Chronic Care Model on medication adherence when patients perceive cost as a barrier. Prim Care Diabetes 2012;6:137–42. doi:10.1016/j.pcd.2011.12.004 [PMC free article] [PubMed]
3. Polinski JM, Shrank WH, Huskamp HA et al. Changes in drug utilization during a gap in insurance coverage: an examination of the medicare Part D coverage gap. PLoS Med 2011;8:e1001075 doi:10.1371/journal.pmed.1001075 [PMC free article] [PubMed]
4. Bauer AM, Schillinger D, Parker MM et al. Health literacy and antidepressant medication adherence among adults with diabetes: the diabetes study of Northern California (DISTANCE). J Gen Intern Med 2013;28:1181–7. [PMC free article] [PubMed]
5. Oberje EJ, de Kinderen RJ, Evers SM et al. Cost-effectiveness of medication adherence-enhancing interventions: a systematic review of trial-based economic evaluations. Pharmacoeconomics 2013;31:1155–68. doi:10.1007/s40273-013-0108-8 [PubMed]
6. Chisholm-Burns MA, Spivey CA The ‘cost’ of medication nonadherence: consequences we cannot afford to accept. J Am Pharm Assoc (2003) 2012;52:823–6. doi:10.1331/JAPhA.2012.11088 [PubMed]
7. Morisky DE, Ang A, Krousel-Wood M et al. Predictive validity of a medication adherence measure in an outpatient setting. J Clin Hypertens (Greenwich) 2008;10:348–54. doi:10.1111/j.1751-7176.2008.07572.x [PMC free article] [PubMed]
8. Wu JR, DeWalt DA, Baker DW et al. A single-item self-report medication adherence question predicts hospitalisation and death in patients with heart failure. J Clin Nurs 2014;23:2554–64. doi:10.1111/jocn.12471 [PMC free article] [PubMed]
9. Wu JR, Moser DK, Chung ML et al. Objectively measured, but not self-reported, medication adherence independently predicts event-free survival in patients with heart failure. J Card Fail 2008;14:203–10. doi:10.1016/j.cardfail.2007.11.005 [PMC free article] [PubMed]
10. Granger BB, Ekman I, Hernandez AF et al. Results of the Chronic Heart Failure Intervention to Improve MEdication Adherence study: a randomized intervention in high-risk patients. Am Heart J 2015;169:539–48. doi:10.1016/j.ahj.2015.01.006 [PMC free article] [PubMed]
11. Krousel-Wood M, Holt E, Joyce C et al. Differences in cardiovascular disease risk when antihypertensive medication adherence is assessed by pharmacy fill versus self-report: the Cohort Study of Medication Adherence among Older Adults (CoSMO). J Hypertens 2015;33:412–20. doi:10.1097/HJH.0000000000000382 [PMC free article] [PubMed]
12. Bosworth HB, Olsen MK, McCant F et al. Hypertension Intervention Nurse Telemedicine Study (HINTS): testing a multifactorial tailored behavioral/educational and a medication management intervention for blood pressure control. Am Heart J 2007;153:918–24. doi:10.1016/j.ahj.2007.03.004 [PubMed]
13. Jonsdottir H, Opjordsmoen S, Birkenaes AB et al. Medication adherence in outpatients with severe mental disorders: relation between self-reports and serum level. J Clin Psychopharmacol 2010;30:169–75. doi:10.1097/JCP.0b013e3181d2191e [PubMed]
14. Aikens JE, Piette JD Longitudinal association between medication adherence and glycaemic control in type 2 diabetes. Diabet Med 2013;30:338–44. doi:10.1111/dme.12046 [PMC free article] [PubMed]
15. Gonzalez JS, Schneider HE, Wexler DJ et al. Validity of medication adherence self-reports in adults with type 2 diabetes. Diabetes Care 2013;36:831–7. doi:10.2337/dc12-0410 [PMC free article] [PubMed]
16. Clifford S, Perez-Nieves M, Skalicky AM et al. A systematic literature review of methodologies used to assess medication adherence in patients with diabetes. Curr Med Res Opin 2014;30:1071–85. doi:10.1185/03007995.2014.884491 [PubMed]
17. Centers for Disease Control and Prevention. 2014 National Diabetes Statistics Report. http://www.cdc.gov/diabetes/data/statistics/2014statisticsreport.html
18. Spratt SE, Batch BC, Davis LP et al. Methods and initial findings from the Durham Diabetes Coalition: integrating geospatial health technology and community interventions to reduce death and disability. J Clin Transl Endocrinol 2015;2:26–36. doi:10.1016/j.jcte.2014.10.006
19. Granger BB, Swedberg K, Ekman I et al. Adherence to candesartan and placebo and outcomes in chronic heart failure in the CHARM programme: double-blind, randomised, controlled clinical trial. Lancet 2005;366:2005–11. doi:10.1016/S0140-6736(05)67760-4 [PubMed]
20. Ratanawongsa N, Karter AJ, Parker MM et al. Communication and medication refill adherence: the Diabetes Study of Northern California. JAMA Intern Med 2013;173:210–18. doi:10.1001/jamainternmed.2013.1216 [PMC free article] [PubMed]
21. Jones CD, Holmes GM, DeWalt DA et al. Self-reported recall and daily diary-recorded measures of weight monitoring adherence: associations with heart failure-related hospitalization. BMC Cardiovasc Disord 2014;14:12 doi:10.1186/1471-2261-14-12 [PMC free article] [PubMed]
22. Voils CI, Maciejewski ML, Hoyle RH et al. Initial validation of a self-report measure of the extent of and reasons for medication nonadherence. Med Care 2012;50:1013–19. doi:10.1097/MLR.0b013e318269e121 [PMC free article] [PubMed]
23. Koschack J, Marx G, Schnakenberg J et al. Comparison of two self-rating instruments for medication adherence assessment in hypertension revealed insufficient psychometric properties. J Clin Epidemiol 2010;63:299–306. doi:10.1016/j.jclinepi.2009.06.011 [PubMed]
24. Matza LS, Park J, Coyne KS et al. Derivation and validation of the ASK-12 adherence barrier survey. Ann Pharmacother 2009;43:1621–30. doi:10.1345/aph.1M174 [PubMed]
25. Decoste M, Vanobberghen R, Borgermans L et al. Uncontrolled hypertension among black Africans in the city of Brussels: a case–control study. Eur Rev Med Pharmacol Sci 2013; 17:886–94. [PubMed]
26. Hong K, Muntner P, Kronish I et al. Medication adherence and visit-to-visit variability of systolic blood pressure in African-Americans with chronic kidney disease in the AASK trial. J Hum Hypertens 2016;30:73–8. doi:10.1038/jhh.2015.26 [PMC free article] [PubMed]

Articles from BMJ Open Diabetes Research & Care are provided here courtesy of BMJ Publishing Group