The present study showed that a low-order IVP simulator can reproduce clinical study results obtained in a population of subjects that are independent from those used to develop the model. Key observations made in the clinical study12
used for comparison were that (1) 85% of all sensor glucose levels were between 70 and 180 mg/dl, (2) there was large intersubject variability in peak post-prandial glucose levels, (3) the closed-loop algorithm did not respond until 15–20 min into the meal, (4) peak insulin concentrations did not occur until 120 min after the meal started, and (5) plasma insulin concentration did not return to basal level until 8–10 h after dinner. These observations were all reproduced in the simulation study with only minor differences noted.
Differences that were noted included a slightly lower percent of sensor glucose readings in target range (71 vs 85%, simulation vs clinical) and an increase in the use of supplemental carbohydrates (6 occurrences in 20 patient days vs 3 in 17). The difference in sensor values in the target range could potentially be attributed to the sensor underestimating glucose at high values in the clinical study.12
Underestimation of sensor glucose can be due to nonlinearities in the sensor response, intrinsic offset in sensor current, or delays in glucose equilibration with ISF.24
In our simulations, we included the delay in glucose equilibration with the SC ISF but did not simulate an offset in sensor current or nonlinearity in the sensor response. This resulted in a better MARD for the simulation than was reported in the pediatric study (5.1 vs 13.2%). Despite the better sensor, the virtual subjects still proved to be more difficult to control, with supplemental carbohydrate being used to correct hypoglycemia 7 times during 20 patient days of control in the simulation study and only 3 times in 17 patient days of real control. Although the difference was not statistically significant (p
= .29), having a simulator that predicts the need for supplemental carbohydrate may be beneficial when testing algorithms for safety in advance of performing a clinical study.
Minor differences were also observed in the relative magnitude of the breakfast versus lunch glucose excursions within each study. The two excursions were of similar magnitude in the simulation study, whereas the lunch excursion was smaller than the breakfast excursion in the clinical study. This may have been due to differences in the meal sizes consumed in each study. The meal sizes in the clinical study used to identify virtual patient subjects11
and the pediatric study used for comparison12
were determined by the subjects, with the total amount of carbohydrate not different. However, while the total amount of carbohydrate consumed was not different, the distribution by meal was. Thus, to have the same ratio of breakfast to lunch carbohydrate in the two studies, the pediatric subjects would have had to consume 41% morecarbohydrate at lunch in the FCL control subjects and 31% more in the HCL subjects. Still, despite the differences in patient population and relative carbohydrate consumed at each meal, the average closed-loop glucose profiles obtained in the simulation closely matched that of the clinical study despite insulin delivery rates that were approximately one-third less in the virtual subjects. Similar glucose profiles with different insulin delivery rates can be expected for a closed-loop system that adjusts insulin delivery rates to achieve the same target at the same rate.
The inability to match all conditions in the simulation to those of the clinical study is nonetheless a limitation in the present study. The differences were, however, unavoidable, as IVP model parameters have not been published for pediatric subjects. Differences in the clinical versus simulation study design, population, and meals, may have masked our ability to detect differences between the simulation and clinical studies, or otherwise have affected the power of our statistical analysis. For example, in the simulation study, the difference in the peak postprandial glucose for breakfast on day 2 was significant [77 mg/dl with 95% confidence interval (CI) (55.12 to 99.16); FCL vs HCL], whereas the same comparison in the clinical study was not [32.43 ± 24.95 with 95% CI (-20.75 to 85.61)]. Effectively, the simulation study had more power to detect differences in peak postprandial glucose than the clinical study. However, rather than being an advantage, this suggests that there was an important component on variability that was not captured in the simulation. Conversely, the two-way ANOVA used to evaluate differences between the simulated and clinical results failed to detect differences in the peak postprandial glucose in the simulated and clinical data, raising the possibility that the comparison was underpowered. For the analysis performed, the peak postprandial values for breakfast, lunch, and dinner were combined leaving only study type (simulation vs clinical) and controller type (FCL vs HCL) as factors in the two-way ANOVA. However, combining breakfast, lunch, and dinner resulted in similar values [231 ± 35 (SD) vs 226 ± 51 for FCL, and 205 ± 38 vs 194 ± 47 for HCL]. As the difference was small, just 5 mg/dl for FCL, the power to detect significance was also low (13% estimated from Monte Carlo simulation). We estimate that to have detected a difference between simulated and clinical results, this difference would have needed to be at least 15% (~30 mg/dl) for both the FCL and HCL peak postprandial values (80% power). Given the limitations introduced by the nonmatched patient populations consuming different meals, further validation will be required to validate clinical closed-loop study results that can be reproduced with this simulator.
The IVP model represents one of several models that could have been used for the simulations performed here. Other models include the KADIS simulator,1
the Cambridge simulator,2
and the simulator developed at UVA.3
It is not clear if the simulators yield significantly different results. Generally, equations for all the simulators are known but model parameters for individual subjects are often proprietary. The exceptions to this are the parameters for the 10 adult IVP subjects used here (reported in Kanderian and colleagues)6
and the 10 patients in each of the adult, pediatric, and child groups described in the UVA patent.25
Equations for the different models differ primarily in number (model order) and the number of parameters. The trend has been to use high-order models for simulation,26
although the parameters of such models are unidentifiable from data that is routinely collected in clinical studies (meal information, insulin delivery, and insulin and glucose concentrations). Lower order models are still well accepted for measuring metabolic parameters or as components of model predictive control algorithms.26
However, low-order models do continue to be used as simulators. The KADIS model, for example, is identifiable from logbook or continuous glucose monitoring data. Once identified, the model is used to perform simulations to optimize open-loop therapy.27
Although IVP model parameters are identifiable from the same routine data collection, the model is not suited for predicting a single future meal excursion as the model parameters are time-varying. Time-varying parameters were chosen as a basis for the model as it is well known that individuals with type 1 diabetes can have high inter- and intraday variability in insulin requirements. As the IVP model is set up to capture this variability, it can only be used to predict average glucose responses that would be obtained by an individual on repeated days, or the average results obtained in a study involving multiple subjects. Results from the present study support its ability to predict results obtained across multiple subjects. The model has also been used as a guide in developing modifications to the PID controller,14
which have been tested and shown to work well in subsequent clinical studies.13
Nonetheless, given the limitations described earlier (nonmatched patient populations consuming different meals), further validation may still be required to assess its ability to reproduce other closed-loop clinical study results.
Components of the IVP model—specifically the two-compartment PK model described by Equations (1)
—have been used as components of different control algorithms. The PID closed-loop insulin delivery algorithm developed by Medtronic13
and the MPC controller developed by Damiano and colleagues28
both incorporated this characterization of the PK insulin response to predict plasma insulin concentrations during closed-loop insulin delivery. Noteworthy is that in both of these cases,13,28
model parameters were preset to values obtained from the literature and no attempt was made to adjust or identify them in real time. In the study by Damiano and colleagues,28
the insulin profiles were not well predicted in some individual cases, leading to poor control. However, when the parameters were retrospectively identified and the subjects studied on a subsequent occasion, both the real-time prediction and resulting control were substantially improved. Generally, closed-loop controllers utilizing models as part of the control algorithm can be configured to identify the parameters of the model in real time; however, the ability of the real-time identification algorithms to track time-varying parameters needs to be carefully assessed. Subjects with parameters that differ substantially may need their parameters to be set off-line.
In summary, results presented in the original IVP model development and identification study6
showed that the IVP model can be identified from routine insulin, glucose, and meal data. The present study showed that once a population of virtual patients is identified, that population can be used to perform closed-loop studies with the expected result not statistically different from that which would be obtained in a clinical study. Additional validation of the model, however, is required to confirm that the model can be used to optimize control in an individual subject or identify regions of control stability for the individual.