|Home | About | Journals | Submit | Contact Us | Français|
To describe the development process for defining an appropriate model structure for the economic evaluation of test–treatment strategies for patients with monogenic diabetes (caused by mutations in the GCK, HNF1A or HNF4A genes).
Experts were consulted to identify and define realistic test–treatment strategies and care pathways. A systematic assessment of published diabetes models was undertaken to inform the model structure.
National Health Service in England and Wales.
Experts in monogenic diabetes whose collective expertise spans the length of the patient care pathway.
A defined model structure, including the test–treatment strategies, and the selection of a published diabetes model appropriate for the economic evaluation of strategies to identify patients with monogenic diabetes.
Five monogenic diabetes test–treatment strategies were defined: no testing of any kind, referral for genetic testing based on clinical features as noted by clinicians, referral for genetic testing based on the results of a clinical prediction model, referral for genetic testing based on the results of biochemical and immunological tests, referral for genetic testing for all patients with a diagnosis of diabetes under the age of 30 years. The systematic assessment of diabetes models identified the IMS CORE Diabetes Model (IMS CDM) as a good candidate for modelling the long-term outcomes and costs of the test–treatment strategies for monogenic diabetes. The short-term test–treatment events will be modelled using a decision tree which will feed into the IMS CDM.
Defining a model structure for any economic evaluation requires decisions to be made. Expert consultation and the explicit use of critical appraisal can inform these decisions. Although arbitrary choices have still been made, decision modelling allows investigation into such choices and the impact of assumptions that have to be made due to a lack of data.
Monogenic diabetes, caused by mutations in the GCK, HNF1A or HNF4A genes, differs to known type 1 and type 2 diabetes in terms of disease progression and the treatments patient receive. Rather than the more invasive and expensive treatment of subcutaneous insulin, patients with HNF1A and HNF4A mutations can be successfully treated with sulfonylureas, while patients with GCK mutation rarely require pharmacological treatment.1–5 The challenge is to diagnose monogenic diabetes, as many patients with monogenic diabetes will be mislabelled as having type 1 or type 2 diabetes and will continue to inject insulin.
Diabetes has a substantial clinical impact on the patient's health and well-being, in addition to being a large economic burden for healthcare systems. It is estimated that most people with monogenic diabetes are initially inappropriately managed due to misdiagnosis.6 7 Identifying patients with monogenic diabetes and selecting the appropriate treatment option should lead to reduced treatment costs and improved health status for patients. Genetic testing for monogenic diabetes is available via the National Health Service in England and Wales (NHS) and costs £350 per test.8 However, evidence indicates varied patterns of referral rates for genetic testing for monogenic diabetes across geographical regions in the NHS.9 Approaches to improve the identification of patients eligible for genetic testing of monogenic diabetes exist: improving the awareness and understanding of monogenic diabetes in healthcare professionals,10 the development of statistical clinical prediction models,11 and the use of two phenotype-based tests to (1) measure biochemical levels using the urinary C-peptide creatine ratio (UCPCR)12 and (2) identifying the presence of pancreatic autoantibodies.13 Using such approaches alongside genetic testing and followed by appropriate treatment selection (oral sulfonylurea or a controlled diet) has the potential to inform the effective, safe and cost-effective management of people with monogenic diabetes. To date, there has been no robust evaluation to assess the most cost-effective approaches to identify patients with monogenic diabetes.
Modelling test–treatment strategies has many challenges. These include a greater number of strategies to be considered due to different sequences or combinations of tests,14 and the greater complexity of such models to reflect not only the initial testing aspects, but also any follow-up testing, treatment and monitoring of patients.14 Thus, it is important to accurately reflect the test–treatment strategies and associated care pathways when building a model.15 Each structural choice involves a consideration of the inclusion of specific parameters, the definition of those parameters, how they relate to other parameters in the model and the availability of research or other evidence to inform them.16 As the model structure impacts on all parts of the study,16 17 even on the results of the cost-effectiveness modelling,18 it is essential that the structure is informed, defensible and explicit.16 19 20 The aim of this study is to describe the development process for defining an appropriate model structure for test–treatment strategies for monogenic diabetes. There were two specific objectives: (1) to identify the number of possible test and treatment strategies that are potentially feasible to target the treatment of people with monogenic diabetes and (2) to select an appropriate model type and structure to represent the identified test–treatment strategies and subsequent impact on care pathways.
This economic evaluation forms a part of a larger project: Using genetics to Improve Treatment of Early-onset Diabetes—the UNITED study (see http://www.hicfund.org.uk/HICFundPortfolio/Theme1.aspx). The main aims of the UNITED study are (1) to estimate the prevalence of monogenic diabetes in the UK and (2) to provide evidence for a systematic care pathway for the diagnosis and treatment of patients with monogenic diabetes.
In the Methods section, the methods used to identify and define realistic test–treatment strategies and the selection of a long-term diabetes model are described. In the Results section, the defined test–treatment strategies and model structure (including assumptions and likely data sources) are given in detail. Challenges and limitations in the development of an economic evaluation of test–treatment strategies in monogenic diabetes are discussed, followed by brief conclusions.
The identification and definition of care pathways was undertaken iteratively through discussion with a number of experts. A meeting was held with the consultant physician, the diabetes nurse and the project coordinator for the UNITED study. During this initial meeting, discussion centred on the test–treatment strategies that were feasible for patients with monogenic diabetes, and the care pathways associated with these strategies. Once possible test–treatment strategies were agreed upon, further development on the specifics of the strategies and the care pathways was undertaken through email, telephone and one-to-one meetings with individual members of the UNITED clinical team (in particular, the diabetes nurse, project coordinator and the clinical scientists) where necessary. When these were further defined, they were presented at a meeting with members of the UNITED clinical team (including diabetes nurses, clinical molecular geneticists and clinical scientists) and the UNITED study Steering Group for discussion. The UNITED study Steering Group consists of 12 members, including representatives from the UNITED study funding bodies, experts in diabetes, statistics and economic evaluation. The collective expertise of the Steering Group and the UNITED study clinical team spans the length of the care pathway. Further detailing and amendments to the proposed test–treatment strategies were made as a result of these discussions.
A systematic assessment of diabetes models available in the published literature was undertaken to help inform the type and structure of the decision model to be used. The diabetes models that were assessed were those evaluated in two recent reviews of diabetes models by Yi et al21 and Tarride et al.22 Criteria for diabetes models from the American Diabetes Association (ADA)23 and criteria specific to the needs of this economic evaluation were used to assess the established models. These criteria included demonstrated internal and external validity, a long-term time horizon, interdependence between modelled complications, capture of glycated haemoglobin, analysis of quality-adjusted life-years (QALYs) and previous use in the published literature and in health technology assessment (HTA) submissions to reimbursement organisations. Taking information from the descriptive articles for these models and information reported in Yi et al and Tarride et al, the assessment criteria were applied to each diabetes model.
Based on discussions with experts, five strategies were defined to represent realistic options within the NHS, and are summarised in table 1.
Strategy 1 is the most basic strategy where no patients are referred for genetic testing, and therefore no treatment change is modelled for any patient. This strategy represents the current situation as seen by the National Institute for Health and Clinical Excellence (NICE), since there is no NICE guidance specifically for monogenic diabetes as distinct from type 1 or type 2 diabetes.
Strategy 2 is intended to represent the observed current practice in the UK where some patients are referred for genetic testing based on their clinical features as noted by clinicians. Currently, referral for genetic testing is dependent on the knowledge and awareness of clinicians or diabetes nurses. Data from Shields et al9 demonstrate great variation in referral and diagnosis rates between different regions of the UK with the South West of England having the highest referral rate (105.9 per million regional population leading to a total of 28.4 cases per million) and Wales having the lowest referral rate (13.1 per million of the regional population leading to 12.4 cases per million).
Strategy 3 is modelled as a potential improvement on current practice (strategy 2), where referral for genetic testing is more systematic; yet it is still based on the clinical features of the patient. A clinical prediction model has been developed for identifying patients likely to have monogenic diabetes11 and is available online (http://www.diabetesgenes.org/content/mody-probability-calculator).
Strategy 4 represents a systematic pathway consisting of biochemical (UCPCR) and immunological (pancreatic autoantibody) testing. While patients with monogenic diabetes continue to produce their own insulin, type 1 diabetes is characterised by insulin deficiency. C-peptide testing in blood has been identified as a marker of those patients producing their own insulin,24 and recent work has demonstrated that UCPCR is a good discriminator between type 1 patients and those with HNF1A, HNF4A or GCK mutations.12 25 However, patients newly diagnosed with type 1 diabetes are considered to be in a ‘honeymoon’ phase, where they continue to produce insulin of their own, up to 5 years post-diagnosis, and therefore will be UCPCR test positive.26 For strategy 4 to be effective, a test to help distinguish between honeymooning type 1 patients and those with HNF1A, HNF4A or GCK mutations is needed. McDonald et al13 have identified pancreatic autoantibody testing for this purpose. The presence of these antibodies characterises type 1 diabetes, but their prevalence in cohorts of patients with monogenic diabetes is similar to the general population.13 Tests for two particular antibodies have been proposed and evaluated by McDonald et al: glutamate decarboxylase (GAD) and islet antigen-2 (IA-2). Strategy 4 is therefore defined by patients currently on insulin treatment being offered the UCPCR test. Those testing positive or not currently on insulin treatment (therefore producing their own insulin) will be offered the GAD and IA-2 tests. Patients testing negative for these antibodies will then be offered genetic testing for the HNF1A, HNF4A and/or GCK mutations.
Strategy 5 is modelled as the extreme to no testing (strategy 1) where all patients diagnosed with diabetes under 30 years of age are referred for genetic testing.
An important distinction between patients diagnosed with diabetes is the treatment they are currently receiving: patients not currently receiving insulin will have demonstrated their ability to produce insulin and therefore do not have type 1 diabetes. In the economic evaluation, the possibility of different strategies being the most cost-effective screening strategy, depending on current treatment, will be assessed (see table 2 for the 25 different strategies defined by current treatment) in addition to the five strategies not distinguishing individuals by treatment (the 5 shaded strategies in table 2).
To allow modelling of the short-term test–treatment events as well as their long-term consequences, a hybrid decision model will be used consisting of a decision tree (in MicroSoft Excel 2007) to model the short-term costs and benefits which will inform parameters in an established, long-term diabetes model.
Genetic testing is undertaken in strategies (2–5). A modelling assumption is that if a mutation is suspected in the HNF1A or HNF4A genes, both will be tested simultaneously. This is based on the current system at the UK referral centre for monogenic diabetes, the Royal Devon and Exeter NHS Foundation Trust (RDEFT), which allows simultaneous testing of HNF1A and HNF4A genes at a reduced cost. Assessment of clinical criteria for deciding whether the HNF1A and HNF4A gene mutations or the GCK gene mutation is initially tested is based on the judgement of experts in monogenic diabetes. These clinical criteria include patient age at diagnosis, BMI, any evidence of sensitivity to sulfonylureas, family history, HbA1c and fasting blood glucose among others.27 It is assumed that all positive genetic tests are correct, that is, that there are no false-positive genetic test results, and any negative tests are correct, that is, there are no false-negative test results.
Data on the decision as to whether a sample is initially tested for the HNF1A and HNF4A gene mutations or the GCK mutation are available from the current database of referrals to the UK referral centre.
The UCPCR test is part of the UNITED strategy (strategy 4). This test is only offered to those in the modelled cohort currently receiving insulin treatment. Patients testing positive for the UCPCR test are assumed to be producing their own insulin.
Thresholds for the UCPCR test that discriminates patients who are producing their own insulin from patients who are not have been identified for adults12 and children (unpublished data). In adults with a diagnosis of diabetes of ≥5 years, a threshold of 0.2 nmol/mmol has 97% sensitivity and 96% specificity for discriminating patients with mutations in HNF1A or HNF4A genes from those with type 1 diabetes.12
The optimum UCPCR thresholds for clinical practice may not necessarily be the optimum thresholds for cost-effectiveness. Therefore, in sensitivity analyses, the impact of assuming different UCPCR thresholds for discriminating monogenic diabetes from type 1 diabetes will be assessed on the estimates of cost-effectiveness.
Pancreatic autoantibody tests are also part of the UNITED strategy (strategy 4) to distinguish patients with monogenic diabetes from those with type 1 diabetes in the honeymoon phase. All patients testing positive for UCPCR and those patients who currently receive non-insulin treatment are offered the tests. As in practice, the two antibodies (GAD and IA-2) are tested simultaneously. If patients test negative for either of these antibodies, they are offered the genetic test.
Recent data are available on the ability of pancreatic autoantibody tests to discriminate between patients with mutations in GCK, HNF1A or HNF4A genes and those with type 1 diabetes.13 A negative GAD test had a sensitivity of 99% and a specificity of 62%, while a negative IA-2 test has a sensitivity of 100% and a specificity of 57%. However, by combining these tests so that a negative result from the GAD or the IA-2 test is defined as a negative result, greater discriminatory power is achieved: 99% sensitivity and 82% specificity.
The uptake of testing may not be 100% in the screening strategies due to the fact that urine and blood samples are required and the decision tree will allow for this.
The model also allows for the possibility that some patients will be asked to provide more than one (urine or blood) sample for the C-peptide and autoantibody tests. Repeat tests (UCPCR, pancreatic autoantibody or genetic) on the same sample may also be required for some patients and these will be incorporated into the model.
For strategies 2, 3 and 5, it will be difficult to estimate the proportion of patients refusing referrals for genetic testing as these data do not exist. However, for strategy 4, data from the UNITED study will inform the uptake of genetic testing, as well as the uptake of the C-peptide and autoantibody tests. Estimates of uptake from the UNITED study will be used to inform the uptake of genetic testing in strategies 2, 3 and 5. Similarly, data on the proportion of additional urine and blood samples required from patients for repeat tests in strategy 4, and the number of repeat tests carried out, available from the UNITED study, will be used to inform estimates for strategies 2, 3 and 5.
For all strategies involving a treatment change, only patients having a genetically confirmed diagnosis of monogenic diabetes due to mutations in the GCK, HNF1A and HNF4A genes will be offered a change in treatment. Patients with the GCK mutation currently receiving treatment will be offered dietary advice so that they can stop treatment, whereas patients with the HNF1A or HNF4A mutation will be offered treatment transfer to sulfonylureas. Some patients may not cease insulin treatment, but with the addition of sulfonylureas, their insulin dose may be reduced.5 Data from the UNITED study will be used to inform this aspect of the model. The impact of any non-compliance of the current or new treatment will be modelled, as will any treatment-related adverse events, for example, mild hypoglycaemia when transferring from insulin to sulfonylureas.5 There may also be an increase in home blood glucose monitoring during treatment transfer. The model will allow for the possibility that some patients may refuse or fail treatment transfer. If patients do not have a test for monogenic diabetes or have a negative genetic test result, it is assumed that they will continue on their current treatment.
The decision tree time horizon will be long enough to capture all costs and health-related outcomes up until 12 months after patients with monogenic diabetes have attempted treatment transfer onto sulfonylureas or diet, as appropriate. The UNITED strategy has the longest time delay before patients attempt treatment change, and so, based on this strategy, the time horizon for the decision tree will be 16 months in the base case analyses.
A total of 11 diabetes models, described in Yi et al21 and/or Tarride et al,22 were reviewed. They were the IMS CDM,28 UKPDS,29 Archimedes,30 EAGLE,31 DCCT,32 DMM,33 Eastman,34 35 GDM,36 JADE,37 DiDACT38 and CDC39 models. The results from applying the assessment criteria defined in the ‘Defining model structure’ section to these 11 models are given in web appendix 1. For some of the diabetes models, not enough evidence could be found to evaluate all the criteria. However, a number of diabetes models met all or many of the criteria including the IMS CDM,28 the UKPDS model,29 the Archimedes model30 and the Eagle model.31 We identified the IMS CDM in particular as it corresponds to the ADA modelling criteria, was well received at the fourth Mount Hood challenge,40 has been used in NICE HTAs41 as well as in HTAs worldwide42 43 and has been developed over a period of 12 years.
Details on the IMS CDM have been described and summarised in a number of published articles (eg, Palmer et al,28 Yi et al21 and Tarride et al22). It uses 15 Markov submodels to simulate outcomes and costs for type 1 and type 2 diabetes patient cohorts, covering a range of macrovascular, microvascular and treatment-related complications with interaction between the submodels.28 Baseline patient characteristics contribute to the calculation of the clinical outcomes and their costs, as do the risk factors such as age, duration of diabetes, HbA1c, body mass index (BMI) and treatment.
Data on HbA1c, age, weight, sex, duration of diabetes and current treatment, informed by the UNITED study, as well as the total costs and health outcomes from the short-term decision tree model will feed into the long-term diabetes model, so that total costs and outcomes can be calculated for each strategy.
The economic evaluation will consider all patients under the age of 30 years who are diagnosed with diabetes, regardless of whether they are insulin-treated or not. The cut-off of 30 years is based on evidence indicating that patients with monogenic diabetes are generally diagnosed with diabetes by this age, but usually misdiagnosed with type 1 diabetes.44 45 The cost-effectiveness of different test–treatment strategies will be evaluated in a prevalent cohort of patients, as well as in a mixture of prevalent and future incident cohorts to evaluate the cost-effectiveness of strategies should they be rolled out on a permanent basis.45
Data relevant to the long-term outcomes of patients with the HNF1A mutation are available from two papers: Steele et al 201046 and Isomaa et al 1998.47 Steele et al46 report on the risk of mortality (cardiovascular and all cause) in HNF1A patients compared to non-diabetic family members in the UK from a retrospective analysis. Isomaa et al47 report the prevalence of diabetic complications in HNF1A, type 1 and type 2 patients in Finland in a cross-sectional study. Both these studies are limited by the study design (Isomaa et al is cross-sectional, Steele et al is retrospective). However, as the identification of monogenic diabetes is relatively new, there are less long-term data available, which will therefore be subject to greater uncertainty than for type 1 or type 2 diabetes.
For base case analyses, utility values will be defined using the EQ-5D where possible, in line with NICE's preferred method.48 EQ-5D data are being collected in the UNITED study and will inform the short-term decision model. Utility values, those from the EQ-5D for a UK population, if available, will be sought from the literature for the long-term model.
Patterns of resource use by patients will differ by screening strategy. The hybrid model will include:
For some of the strategies, there will be a one-off cost associated with implementation of the strategy itself, as well as likely ongoing costs for maintenance or training.
Treatment costs will include the cost of drug acquisition, taken from the most recent BNF (currently BNF6449), as well as any drug administration costs. There will also be a one-off cost for dietary advice for patients transferring off all treatment. In addition to the treatment costs, the model will include treatment transfer costs, for example, the costs for diabetes nurse time for monitoring of patients during transfer of treatment as estimated from the UNITED study.
A major cost component will be the costs of diabetes-related complications experienced during a patient's lifetime. These cost estimates will be based on the most recent NHS Reference costs (currently50) and data from the literature.
The primary outcome will be the cost per QALY gained associated with each test–treatment strategy in comparison with the other strategies. An incremental analysis will be undertaken as all strategies to be modelled are realistic alternatives to each other. Secondary outcomes will include the cost per life-year gained, the cost per case identified and the cost per successful treatment transfer.
As preferred by NICE, the perspective of the model will be that of the NHS and Personal Social Services (PSS), with all costs and outcomes after the first year discounted at an annual rate of 3.5%.48 A lifetime horizon is modelled.
The decision model will undergo thorough testing and debugging by the model developer, and will be checked by a second experienced decision modeller. Internal validation of the decision model will be undertaken using data from the UNITED study.
In this paper, we have described the design and development of a new model for evaluating the long-term clinical effectiveness and cost-effectiveness of test–treatment strategies for patients with monogenic diabetes. Using critical appraisal methods and explicit expert input, the test–treatment strategies and model structure have been defined. Defining a model structure for any economic evaluation requires decisions to be made. However, the results of decision models are highly dependent on the range of comparators used, the decisions and assumptions regarding model structure as well as the evidence to inform parameters.15 Thus, explicit definition of the model with expert input and critical appraisal of existing models provides a justifiable model structure, lending credibility to the decision model and the subsequent cost-effectiveness results.
Although much of the model structure has been informed either by literature or experts, arbitrary choices have still been made. For instance, the time horizon of the short-term decision model is 16 months, based on the UNITED pathway: 4 months from initial test to treatment change plus 12-month treatment change follow-up. This is based on the length of the UNITED study, with the 12-month follow-up data used to predict long-term events in the IMS CDM. It is not clear whether outcomes at 12-month follow-up are the best data to predict long-term events in the IMS CDM. However, decision modelling allows investigation of the impact of such choices through sensitivity analysis.
More important is the validity of the model—that is, its ability to accurately predict the outcome or other relevant data on which the model was not originally based. As there are less data on monogenic diabetes, all available data on the test–treatment strategies and care pathways have or will be used to inform and/or parameterise the short-term model. Thus, there are no data available to allow external validation of the short-term model. Similarly, although the IMS CDM has undergone internal and external validation for type 1 and type 2 cohorts of patients, there has been no validation of this model, or any diabetes model, for monogenic diabetes. This further highlights the importance of face validity of the model, including the strategies to be evaluated and the model structure.
The next steps in developing the decision model are: (1) to validate the short-term decision tree model when relevant data from the UNITED study are available and (2) to identify relevant and most up-to-date data to populate the decision model.
Economic evaluation of test–treatment strategies in monogenic diabetes is difficult because of the possible number of realistic test–treatment strategies and associated care pathways. However, it is important to define realistic alternative strategies and to have an informed and defensible model structure as this is likely to impact on the cost-effectiveness. This paper demonstrated the feasibility of the explicit use of critical appraisal methods and expert consultation approaches for defining the strategies and model structure, and similar methods can be used by modellers in other conditions for developing decision models.
We would like to thank the following members of the UNITED study team for their advice, input and discussion on the development of the decision model: Professor Andrew Hattersley, Dr Maggie Shepherd, Dr Beverley Shields, Michelle Hudson, Dr Tim McDonald, Dr Rachel Besser (University of Exeter Medical School), Professor Sian Ellard, Kevin Colclough (Royal Devon and Exeter NHS Foundation trust, Exeter). We also thank all members of the UNITED Steering Group for their advice and discussion of the model structure. We are grateful to Professor Katherine Payne for her time and effort in reviewing a previous version of this manuscript and Professor Norman Waugh for comments. We would also like to thank IMS Health for the use of the IMS CDM, in particular thanks to Dr David Grant. This work presents independent research commissioned by the Health Innovation Challenge Fund (grant number HICF-1009-041), a parallel funding partnership between the Wellcome Trust and the Department of Health. The views expressed in this publication are those of the authors and not necessarily those of the Wellcome Trust or the Department of Health.
Contributors: JP led the development of the economic evaluation, consulted with experts, carried out the systematic assessment of published diabetes models and drafted the manuscript. RA and CH contributed to the development of the economic model, discussion with experts and helped to draft the manuscript. All authors read and approved the final manuscript.
Funding: Health Innovation Challenge Fund. (grant number HICF-1009-041), a parallel funding partnership between the Wellcome Trust and the Department of Health.
Competing interests: None.
Provenance and peer review: Not commissioned; externally peer reviewed.
Data sharing statement: Data used in the development of the model-based economic evaluation are available in the manuscript text and the web appendix.