The overall study incorporated three components: (1) a study to assess patient accuracy in estimating the carbohydrate content of meals, (2) a simulation study comparing insulin bolus recommendations for the bolus guide and bolus calculator and consequent BG levels, and (3) a usability study to assess patient perceptions of the bolus guide.
Carbohydrate Content Accuracy Study
Type 1 diabetes pump users (n = 60, pump usage time 6.4 ± 3.9 years) familiar with carbohydrate quantification participated in this study. The study was approved by an institutional research review board provided by Chesapeake Research Review. Participants signed an informed consent form. shows patient demographics.
Eight packaged meals () of known carbohydrate values, containing typical amounts of carbohydrate in meals consumed by the intended use population, were shown to participants, and they were asked to record their estimates of the carbohydrate content of each meal on a form provided. A total of 48 participants estimated the carbohydrate load of meals A–D, and 42 participants estimated the carbohydrate load of meals E–H (30 participants estimated all meals, 18 participants estimated only meals A–D, and 12 participants estimated only meals E–H).
The results of this study (i.e., human error in carbohydrate estimation and error distribution) served as the basis for subsequent computer simulations.
Bolus calculators typically ask the user to identify a specific amount of carbohydrate to be consumed in order to make a recommendation for the amount of insulin to be administered in a bolus. While mathematically appropriate, this does not take into account the difficulty of precisely estimating carbohydrate load in a meal. The bolus guide tool was designed to recommend a bolus insulin dose based on user input of the estimated carbohydrate load within a range rather than a single point.
Tables of carbohydrate and glucose ranges are stored in the bolus guide software memory, with each table related to a specific combination of target blood glucose (TBG), insulin sensitivity factor (ISF), and insulin-to-carbohydrate ratio (ITC). The number of tables stored in the database is limited to combinations of the TBG, ISF, and ITC values [minimum, maximum, and increments (in brackets)] as shown in . Should data entered exceed these limits, the bolus guide will present an error message. Future work will be done to increase these ranges to cover patients with even greater insulin sensitivity.
Combinations of the Target Blood Glucose, Insulin Sensitivity Factor, and Insulin-to-Carbohydrate Ratio Values Supported by the Bolus Guide
The columns and rows of the tables stored in the bolus guide include ranges of carbohydrate and current blood glucose (CBG), respectively [range boundaries (low carbohydrate, high carbohydrate; low BG, high BG) are shown in ]. Each cell in represents a pair of discrete reference values, one for carbohydrate load range and one for CBG range. The bolus (in units) recommended in each cell (in , for example, 3.2 U for the cell represented by carbohydrate range 61–75 g and BG range 71–100 mg/dl) is precalculated according to the formula using the reference values (in the given example, 64 g and 98 mg/dl, respectively). The reference values were selected to minimize hypoglycemia related to the bolus dose recommendation of each cell. The reference values are not the mid values of carbohydrate and CBG range boundaries and are different for each bolus guide grid (defined by various combinations of TBG, ISF, and ITC). The reference values were chosen in each cell to produce the smallest error for the extreme cases of carbohydrates and BG in the cell (lowest and highest corner values in the carbohydrates × BG cell). However, the error was made artificially large if it resulted in an outcome blood glucose (OBG; as explained further later) under 60 or over 200. These values were originally given as constraints on the accuracy requirements of the bolus guide tool.
Bolus Guide Grid and Range Boundaries
The bolus guide process for recommending an insulin bolus dose is based on the user’s input of TBG, ISF, and ITC, entered once at the initial setting. The CBG and a particular carbohydrate range selected from a set of ranges that the system prespecifies are entered every time the bolus guide is used. Carbohydrate range is the range comprising the total amount of carbohydrates to be consumed. The software also enables entry of a discrete carbohydrate value that is attributed to the relevant carbohydrate range. For example, if a user enters a discrete carbohydrate value of 68 g, the reference value (i.e., 64 g) of the relevant carbohydrate range (i.e., 61–75 g) is used to calculate the bolus dose ().
The remaining insulin or “bolus on board” (BOB) is based on delivery times and doses of previous boluses and is calculated in the same manner as in available bolus calculators.
A computer simulation was performed wherein hundreds of carbohydrate estimation error values were generated within the coefficient of variation (CV) range (30–50%) resulting from the carbohydrate estimation study outcome for each of the meals displayed. In addition to CVs yielded by the study, simulation results for lower CV values of 0–30% were also evaluated. The carbohydrate estimation error values were also simulated according to the error distribution patterns resulting from the carbohydrate estimation study outcome. Normal (Gaussian) and gamma (an asymmetrical distribution with certain skewness) distributions around the estimated and true carbohydrate (TC) values (four different error distribution patterns) were thus simulated. Shape and scale parameters for gamma distribution were extracted by mean and standard deviation (scale = CV2
× mean and shape = 1/CV2
; as mean, the TC was taken for each meal, and CV was fixed at 0% to 50% by step of 10%).
An additional error that was considered and simulated is the error of the BG measurement (CBG). The simulated CVs of the CBG error were 0% (i.e., accurate BG measurements) and 10%. The BG measurement errors were assumed to be normally distributed around the true BG level with a standard deviation equal to the true BG times the given error CV for BG.
Recommended insulin boluses were simulated to yield three bolus recommendations as follows:
- Bolus guide recommendation based on
- carbohydrate—reference value representing the user-estimated bolus guide range,
- CBG—reference value representing bolus guide range that includes the user’s CBG value, and
- ITC and ISF—rounded to the nearest available bolus guide value ().
- Calculator recommendation based on CBG, ITC, ISF, and user-estimated carbohydrate (discrete value) according to the following formula:
- Correct recommendation based on CBG, ITC, ISF, and TC value (). The correct recommendation is based on no error of the intake of carbohydrate. The carbohydrate intake is expressed as a discrete value rather than as a range of values.
The ISF and ITC are assumed to be accurate and therefore do not contribute to the error of the calculator recommendation or the correct recommendation.
The process of generating the three bolus recommendation comparisons (bolus guide, calculator, and correct) was repeated for 140 random combinations of ISF, ITC, and TBG for each of eight meal types (A–H). These 1120 combinations of parameters were simulated with 12 different values of CBG (total 13,140). Each combination was simulated with 10 different estimated carbohydrate values based on a given error CV for carbohydrate (yielding 131,400 observations). These comparisons were repeated for six carbohydrate estimation CVs of 0%, 10%, 20%, 30%, 40%, and 50% (yielding 806,400 observations). These comparisons were repeated for two BG measure-ment errors of 0% and 10% for each of the 12 simulated CBG values (yielding 1,612,800 observations).
An additional simulation (875,712 observations) with uniform distribution running over a narrower range of the most commonly used user parameters (ISF 40–110, TBG 80–120, ITC 8–31) was also completed. Such a simulation resembles a more realistic distribution of the combinations of ISF, ITC, and TBG in the population.
An analysis was conducted to evaluate the clinical significance of the bolus guide and the calculator recommendation errors in comparison to the correct recommendation. The BG levels that result from administration of a certain amount of insulin (either that recommended by the bolus guide or the calculator) can be predicted using the following equation:
wherein U is insulin units (provided by either calculator or bolus guide). Note that BOB is considered 0 in the simulations since it would affect both tools equally. When generating a bolus dose using the discrete values of perfectly accurate BG and carbohydrate estimations (i.e., the “correct” recommendation), the OBG always equals exactly the TBG value.
Outcome BG levels were assigned to one of the following OBG categories ≤60, 61–90, 91–120, 121–150, 151–200, 201–250, and >250 mg/dl, and the percentage of OBG for the bolus guide and the bolus calculator in each category was compared. In addition, we compared the proportion of inaccurate OBG with each tool. Outcome BG was considered inaccurate if it fell outside the TBG range (70–180 mg/dl) or if the OBG was not in the same category as the TBG.
Type 1 diabetes pump users (n = 30) participated in a usability study (n = 30) to estimate patient satisfaction using the bolus guide. The study was approved by Chesapeake Research Review. Participants signed an informed consent form. The bolus guide application was implemented as a software program on a personal computer [(PC) yet it can be implemented using any device that includes memory and computing capabilities, e.g., an insulin pump, a PC, a cell phone, or a personal digital assistant]. Its use and application were demonstrated with the PC software. After training by a facilitator, the participants practiced until they felt confident and then responded to four written scenarios (demonstrated on cue cards) to submit bolus recommendation requests (). In order to receive a bolus recommendation, the participants had to enter BG value, carbohydrate range, ISF, ITC, TBG, and the previous boluses delivered (for BOB calculation). Distractions and errors () were intentionally introduced, and the participants needed to arrive at the correct recommendation by responding to error messages and making the needed corrections. Participants completed a 15-item questionnaire () about their experience using the bolus guide, with responses on a five-point scale, with 1 being “strongly disagree” and 5 being “strongly agree.” Questions included an item “choosing carbohydrate values from a ranges list is easier than estimating exact amounts.”
Four Written User Scenarios with Distractions and Errors
Descriptive Statistics for the User’s Agreement by Item