PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of jdstLink to Publisher's site
 
J Diabetes Sci Technol. 2011 July; 5(4): 999–1004.
Published online 2011 July 1.
PMCID: PMC3192607

Outcome Measures for Outpatient Hypoglycemia Prevention Studies

Roy W Beck, M.D., Ph.D.,1 Craig Kollman, Ph.D.,1 Dongyuan Xing, M.P.H.,1 Bruce A Buckingham, M.D.,2 and H. Peter Chase, M.D.3

Abstract

Systems are being developed that utilize algorithms to predict impending hypoglycemia using commercially available continuous glucose monitoring (CGM) devices and to discontinue insulin delivery if hypoglycemia is predicted. In outpatient studies designed to test such systems, CGM-measured glycemic indices will not only be important outcome measures of efficacy but, in certain cases, will be the only good outcome. This is especially true in short-term studies designed to reduce hypoglycemia since the event rate for severe hypoglycemic events is too low for it to be a good outcome, and milder hypoglycemia often will be variably detected. Continuous glucose monitoring inaccuracy can be accounted for in the study design by increasing sample size and/or study duration.

Keywords: continuous glucose monitoring, nocturnal hypoglycemia, severe hypoglycemia, type 1 diabetes mellitus

Introduction

The prevention of hypoglycemia is a goal of all management strategies for type 1 diabetes mellitus (T1DM). Although the hope for the future is that a well-functioning artificial pancreas can essentially eliminate hypoglycemia, the development of a fully automated closed-loop artificial pancreas will still take many years to develop and validate. An important first step would be to discontinue insulin delivery if hypoglycemia is predicted and the patient does not respond by decreasing pump insulin delivery or eating carbohydrate to raise the glucose level. With this goal in mind, algorithms are being developed to predict impending hypoglycemia using commercially available continuous glucose monitoring (CGM) devices. The initial step in clinical testing of a system that predicts hypoglycemia based on CGM and discontinues pump insulin delivery when impending hypoglycemia is predicted must take place in a well-monitored inpatient setting. After the algorithm has been optimized, the next step is to evaluate the efficacy and safety in an outpatient setting. In designing such a hypoglycemia prevention trial, it is necessary to decide what is the most appropriate outcome measure to assess the efficacy of the algorithm and how the sample size and study duration should be determined.

A system that discontinues insulin delivery when hypoglycemia is predicted would be particularly valuable during the overnight period when hypoglycemia occurs frequently and is not detected. In a study conducted by the Diabetes Research in Children Network (DirecNet), the incidence of nocturnal hypoglycemia (glucose ≥60 mg/dl) in children with T1DM was 48% on nights following afternoon exercise and 28% on nights after no exercise.1 In another study, 57% of adult subjects who did not have a bedtime snack became hypoglycemic (<70 mg/dl) with an average duration of hypoglycemia of 4.4 h.2

We will be conducting an outpatient clinical trial to assess efficacy and safety of a nocturnal hypoglycemia prediction algorithm. Participants will receive insulin through an insulin pump and utilize a CGM device. These two devices will communicate with a bedside computer that will receive glucose data, run a computer program with the algorithm, and turn off the insulin pump when hypoglycemia is predicted. In the clinical trial, a randomization schedule on the computer defines whether the algorithm is being implemented each night (intervention) or not (control).

In this commentary, we provide background information with respect to selection of outcome measures for such a trial and the impact of outcome selection on sample size and trial duration. Hypoglycemia outcomes could be (1) blood glucose documentation of a glucose value below a specified level with or without symptoms of hypoglycemia, (2) a severe hypoglycemic event (SH), or (3) CGM documentation of a glucose value below a specified level with or without symptoms of hypoglycemia. Herein, we will demonstrate that CGM-measured hypoglycemia is the only practical outcome measure for a study of this type. The principles that are presented are pertinent to any clinical trial in which the objective is prevention of nocturnal hypoglycemia or hypoglycemia in individuals with hypoglycemia unawareness.

Documented Hypoglycemia with Blood Glucose Measurement as an Outcome

Although, as noted earlier, biochemical hypoglycemia over- night is common, the vast majority of the time, nocturnal biochemical hypoglycemia does not awaken the individual.3 It is a reason people with diabetes can have hypoglycemic seizures during the night rather than awakening (55% of severe episodes occurred during the night in the Diabetes Control and Complications Trial4 and 75% of seizures in children have been reported to occur during the night5). The use of a real-time sensor with an alarm also is no guarantee that a subject will be awakened at the time of hypoglycemia. When subjects were videotaped wearing a real-time glucose sensor while sleeping, they awoke only to 29% of the alarms,3 and during the Juvenile Diabetes Research Foundation (JDRF) randomized clinical trial with subjects wearing sensors with real-time alarms, the mean duration of hypoglycemia when it occurred was 81 ± 75 min, with 47% of such nights having at least 1 h, 23% at least 2 h, and 11% at least 3 h of hypoglycemia.6 The prolonged duration of hypoglycemia with real-time alarms indicates that we cannot rely on a study participant's awakening to document hypoglycemia even if a sensor alarms. In addition, awakening the individual in the middle of the night to measure blood glucose would not be feasible, because participants would not do this for a large number of consecutive nights and a single time point during the night could miss the occurrence of hypoglycemia at other times during the night. Thus, an outcome relying on the subject awakening and documenting an overnight hypoglycemic blood glucose value would result in a very high false negative rate, limiting the feasibility of such an outcome.

Severe Hypoglycemia Events

As indicated earlier, many episodes of nocturnal hypoglycemia do not awaken the individual and thus would not be detected. In contrast, a SH (e.g., seizure, coma, or incapacitation sufficient that assistance was needed to treat the event) will be apparent. Thus, the only type of clinical hypoglycemic event that might be considered for an outcome in a nocturnal hypoglycemia study would be a SH. However, use of a SH as an outcome lacks feasibility in a clinical trial of this type, because the occurrence rate is very low. As a result, the required sample size would be extraordinarily and unfeasibly large and/or the trial would need to be extremely long.

Table 1 shows published rates of SH without respect to day or night. As can be seen, studies have been fairly consistent with SH rates in the range of 13 to 18 events per 100 person-years among adults and children with intensively treated T1DM. Even if it is assumed that nocturnal events are more common than daytime events, it is likely that the rate of nocturnal events is no greater than 10 events per 100 person-years.

Table 1
Published Rates of Severe Hypoglycemia Events in Type 1 Diabetes Mellitusa

In a trial, the SH rate in the control group might be greater if eligibility was restricted to individuals more likely to experience a SH [e.g., low hemoglobin A1c (HbA1c) and/or self-report of prior SHs], but this may limit recruitment, and the necessary sample size will still be very large. In the JDRF CGM randomized control trial (RCT),there were 36 subjects who had self-reported a SH in the 6 months prior to study entry, and the subsequent rate observed for these subjects was 13.1 nocturnal (midnight–6:00 am) SHs per 100 person-years (unpublished data).

Table 2 provides approximate sample sizes under a wide range of possible rates of nocturnal SH and study durations to achieve 90% power with a two-tailed test and type 1 error rate = 5% based on a Poisson model. Even if the nocturnal SH rate were projected to be 15 events per 100 person-years, the sample size for a 3-month study exceeds 3400 and the sample size for a 6-month study exceeds 1700.

Table 2
Sample Size and Study Duration Estimates for a Clinical Trial Using Severe Hypoglycemia as the Primary Outcomea

Continuous-Glucose-Monitoring-Measured Hypoglycemia

The problems inherent in using a blood glucose measurement to document overnight hypoglycemia are avoided by using CGM to assess hypoglycemia. Current CGM devices record a glucose measurement every 1 to 5 minutes, depending on the device. For certain outpatient studies designed to reduce hypoglycemia, particularly overnight studies, CGM may be the only good outcome. A drawback to using current generation CGM as an outcome measure is that its accuracy is less than blood glucose measurements. However, with available data to estimate the variability of CGM, this inaccuracy can be accounted for in the study design by increasing sample size as will be discussed later.

In a nocturnal hypoglycemia prevention trial, CGM-measured outcomes could include the occurrence of a single glucose level below a threshold such as 60 or 70 mg/dl, consecutive glucose values below a threshold, or such measures as area under the curve or low blood glucose index. From prior analyses of the JDRF CGM RCT data set (unpublished), all four of these outcome measures are highly correlated. When a single glucose level is the outcome, there are several considerations in selecting the level to use for this purpose. The outcome level (1) should be at or below the level at which a counter-regulatory response to hypoglycemia occurs, (2) should be rare in people without diabetes, (3) should occur with sufficient frequency overnight in people with diabetes with respect to determination of the sample size for the study, and (4) should be at or above the glucose level at which the CGM hypoglycemia alarm is set. Setting the threshold to be 60 mg/dl, 70 mg/dl, or a level in between could satisfy all four criteria.

As summarized by the American Diabetes Association Workgroup on Hypoglycemia,11 the counter-regulatory response to decreasing blood glucose is triggered at a blood glucose level of 65–70 mg/dl, and frequent episodes of lower glucose levels, including asymptomatic nocturnal hypoglycemia, lead to defective counter regulation and hypoglycemia unawareness. We (and others) have found that 1.5% of glucose readings in people who do not have diabetes are between 60 and 70 mg/dl (≈4 readings or 20 min each day) while values ≥60 mg/dl make up only 0.2% of CGM glucose readings in people who do not have diabetes.12 To achieve sufficient statistical power, sample size and study duration will be highly dependent on the frequency of hypoglycemia in the control arm (i.e., hypoglycemia prediction algorithm not active). Data from the JDRF CGM RCT were used to estimate the frequency of CGM-determined nocturnal hypoglycemia that might be expected in the control arm of a nocturnal hypoglycemia prevention trial (Table 3).13 Continuous glucose monitoring data were evaluated from 10:00 pm to 6:00 am during the first 6 months of the study in 231 subjects in the CGM treatment group (age 8–72 years, baseline HbA1c 4.7–10.6%). Analysis was restricted to 25,473 nights with at least 6 h of CGM data, ranging from 3 to 182 per subject. Results for frequency of hypoglycemia at different glucose thresholds are summarized in Table 3. It can be seen that, as expected, the hypoglycemia rate is greater at a threshold of 70 mg/dl than at 60 mg/dl, which is greater than the rate at 50 mg/dl. It also can be seen in the table that rates are similar for an outcome of a single value below the threshold and for an outcome requiring two consecutive values below the threshold. From a sample size perspective, the hypoglycemia rate for a threshold of 50 mg/dl is too low for a trial to be feasible. A level of either 60 or 70 mg/dl could be feasible, although the sample size requirement will be lower for a threshold of 70 mg/dl because of the higher hypoglycemia rate.

Table 3
Frequency of Nocturnal Hypoglycemia in the Juvenile Diabetes Research Foundation Continuous Glucose Monitoring Randomized Control Triala

Table 4 provides sample sizes and number of study nights for a range of treatment effects in order to have statistical power of 90% with a type 1 error rate of 5%. As can be seen, the sample size will be lower with a threshold of 70 mg/dl than of 60 mg/dl for a relative 50% reduction in the frequency of nocturnal hypoglycemia. It can also be seen in the table that another factor favoring 70 mg/dl as the threshold is that CGM accuracy is better at 70 mg/dl than at 60 mg/dl as will be discussed in the next section.

Table 4
Sample Sizea and Study Duration Estimates for Binary Hypoglycemia Outcomes in a Nocturnal Hypoglycemia Prevention Trialb

Effect of Continuous Glucose Monitoring Measurement Error on Sample Size

Measurement error can impact study results when CGM data are used as the outcome variable. When the outcome is a binary event such as whether or not hypoglycemia occurred, using CGM to measure outcome will give some false positives (indicating hypoglycemia when it did not actually occur) and some false negatives (missing true occurrences of hypoglycemia). Statistical analysis of CGM data as an outcome, therefore, must consider the potential for misclassification of the outcome when the outcome is binary and a bias and/or increased variance when the outcome is continuous compared with using blood glucose measurements as the outcome. Nondifferential bias (i.e., noise) such as this can be accounted for in the study design by increasing the sample size.

Bias can occur when the sensor systematically overestimates or underestimates the percentage of nights and the false positive rate fails to cancel out the false negative rate. Data from two DirecNet studies comparing overnight Navigator CGM glucose values with central laboratory blood glucose determinations indicate that the false negative rate is approximately 20% and the false positive rate approximately 20% for hypoglycemia occurring some time during the night for a threshold of 70 mg/dl and 35% and 20%, respectively, for a threshold of 60 mg/dl (unpublished). When the false positive and false negative rates are similar, a CGM-measured rate of biochemical hypoglycemia will be relatively unbiased. However, even if the false positive and false negative rates differ, use of a control group eliminates differential bias due to misclassification in the analysis of randomized trial data. If the null hypothesis is correct and the intervention has no beneficial effect, then the same bias affecting the observed rate of biochemical hypoglycemia would occur in both the intervention and the control groups, and thus any such bias could not produce a treatment group difference.

Although these data indicate that there will not be differential bias when using a sensor to determine a hypoglycemia outcome, sensor inaccuracy nevertheless potentially dilutes the observed treatment effect. This can be estimated by subtracting the estimated false positive and false negative rates as follows to determine the proportion of the true treatment effect that will be observed assuming the null hypothesis of no treatment group difference is false. For instance, if both false positive and false negative rates are 20% (the rates that we have estimated for a threshold of 70 mg/dl), this would equate with dilution of the treatment effect to be 60% of the true treatment effect (100–20–20%). However, if the rates are 20% and 35% (the rates that we have estimated for a threshold of 60 mg/dl), then the dilution of the treatment effect is approximately 45%. Knowledge of the degree of potential misclassification of a hypoglycemia outcome can then be used to adjust the sample size so that statistical power will not be reduced by misclassification of outcome. In this example, this would increase the sample size by a factor of 60%−2 ≈ 3 for a threshold of 70 mg/dl and by a factor of 45%−2 ≈ 5 for a threshold of 60 mg/dl.

Conclusion

In outpatient studies conducted to test strategies to reduce nocturnal hypoglycemia, CGM-measured glycemic indices will not only be the most important outcome measures of efficacy, but in many cases be the only good outcomes. This is especially true in short-term studies designed to reduce hypoglycemia since the event rate for SHs is too low for it to be a good outcome, and milder hypoglycemia often will be undetected. Although current generation CGM has greater inaccuracy than blood glucose measurements, with available data to estimate variability of CGM, this inaccuracy can be accounted for in the study design by increasing sample size and/or study duration.

Glossary

Abbreviations

(CGM)
continuous glucose monitoring
(DirecNet)
Diabetes Research in Children Network
(HbA1c)
hemoglobin A1c
(JDRF)
Juvenile Diabetes Research Foundation
(RCT)
randomized control trial
(SH)
severe hypoglycemic event
(T1DM)
type 1 diabetes mellitus

Funding

Study funding was provided by the Juvenile Diabetes Research Foundation, Inc. Grant #22-2009-802.

References

1. Tsalikian E, Mauras N, Beck RW, Tamborlane WV, Janz KF, Chase HP, Wysocki T, Weinzimer SA, Buckingham BA, Kollman C, Xing D, Ruedy KJ. Diabetes Research In Children Network DirecNet Study Group. Impact of exercise on overnight glycemic control in children with type 1 diabetes mellitus. J Pediatr. 2005;147(4):528–534. [PMC free article] [PubMed]
2. Raju B, Arbelaez AM, Breckenridge SM, Cryer PE. Nocturnal hypoglycemia in type 1 diabetes: an assessment of preventive bedtime treatments. J Clin Endocrinol Metab. 2006;91(6):2087–2092. [PubMed]
3. Buckingham B, Block J, Burdick J, Kalajian A, Kollman C, Choy M, Wilson DM, Chase P. Diabetes Research in Children Network. Response to nocturnal alarms using a real-time glucose sensor. Diabetes Technol Ther. 2005;7(3):440–447. [PMC free article] [PubMed]
4. The DCCT Research Group. Epidemiology of severe hypoglycemia in the Diabetes Control and Complications Trial. Am J Med. 1991;90:450–459. [PubMed]
5. Davis EA, Keating B, Byrne GC, Russell M, Jones TW. Hypoglycemia: incidence and clinical predictors in a large population-based sample of children and adolescents with IDDM. Diabetes Care. 1997;20(1):22–25. [PubMed]
6. Juvenile Diabetes Research Foundation Continuous Glucose Monitoring Study Group. Prolonged nocturnal hypoglycemia is common during 12 months of continuous glucose monitoring in children and adults with type 1 diabetes. Diabetes Care. 2010;33(5):1004–1008. [PMC free article] [PubMed]
7. The Diabetes Control and Complications Trial Research Group. Hypoglycemia in the Diabetes Control and Complications Trial. Diabetes. 1997;46(2):271–286. [PubMed]
8. Chase HP, Beck R, Tamborlane W, Buckingham B, Mauras N, Tsalikian E, Wysocki T, Weinzimer S, Kollman C, Ruedy K, Xing D. A randomized multicenter trial comparing the GlucoWatch Biographer with standard glucose monitoring in children with type 1 diabetes. Diabetes Care. 2005;28(5):1101–1106. [PubMed]
9. Fiallo-Scharer R, Cheng J, Beck RW, Buckingham BA, Chase HP, Kollman C, Laffel L, Lawrence JM, Mauras N, Tamborlane WV, Wilson DM, Wolpert H. Juvenile Diabetes Research Foundation Continuous Glucose Monitoring Study Group. Factors predictive of severe hypoglycemia in type 1 diabetes: analysis from the Juvenile Diabetes Research Foundation continuous glucose monitoring randomized control trial dataset. Diabetes Care. 2010;34(3):586–590. [PMC free article] [PubMed]
10. Bergenstal RM, Tamborlane WV, Ahmann A, Buse JB, Dailey G, Davis SN, Joyce C, Peoples T, Perkins BA, Welsh JB, Willi SM, Wood MA. STAR 3 Study Group. Effectiveness of sensor-augmented insulin-pump therapy in type 1 diabetes. N Engl J Med. 2010;363(4):311–320. [PubMed]
11. Workgroup on Hypoglycemia, American Diabetes Association. Defining and reporting hypoglycemia in diabetes: a report from the American Diabetes Association Workgroup on Hypoglycemia. Diabetes Care. 2005;28(5):1245–1249. [PubMed]
12. Fox LA, Beck RW, Xing D. Juvenile Diabetes Research Foundation Continuous Glucose Monitoring Study Group. Variation of interstitial glucose measurements assessed by continuous glucose monitors in healthy, nondiabetic individuals. Diabetes Care. 2010;33(6):1297–1299. [PMC free article] [PubMed]
13. Tamborlane WV, Beck RW, Bode BW, Buckingham B, Chase HP, Clemons R, Fiallo-Scharer R, Fox LA, Gilliam LK, Hirsch IB, Huang ES, Kollman C, Kowalski AJ, Laffel L, Lawrence JM, Lee J, Mauras N, O'Grady M, Ruedy KJ, Tansey M, Tsalikian E, Weinzimer S, Wilson DM, Wolpert H, Wysocki T, Xing D. Juvenile Diabetes Research Foundation Continuous Glucose Monitoring Study Group. Continuous glucose monitoring and intensive treatment of type 1 diabetes. N Engl J Med. 2008;359(14):1464–1476. [PubMed]

Articles from Journal of Diabetes Science and Technology are provided here courtesy of Diabetes Technology Society