PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
J Health Care Finance. Author manuscript; available in PMC 2017 March 7.
Published in final edited form as:
J Health Care Finance. 2016 Summer; 2016(Spec FEATURES): http://www.healthfinancejournal.com/~junland/index.php/johcf/article/view/82/83.
PMCID: PMC5340422
NIHMSID: NIHMS793039

Assessing the cost burden of United States FDA-mandated post-approval studies for medical devices

Abstract

Approved medical devices frequently undergo FDA mandated post-approval studies (PAS). However, there is uncertainty as to the value of PAS in assessing the safety of medical devices and the cost of these studies to the healthcare system is unknown. Since PAS costs are funded through device manufacturers who do not share the costs with regulators, we sought to estimate the total PAS costs through interviews with a panel of experts in medical device clinical trial design in order to design a general cost model for PAS which was then applied to the FDA PAS. A total of 277 PAS were initiated between 3/1/05 through 6/30/13 and demonstrated a median cost of $2.16 million per study and an overall cost of $1.22 billion over the 8.25 years of study. While these costs are funded through manufacturers, the ultimate cost is borne by the healthcare system through the medical device costs. Given concerns regarding the informational value of PAS, the resources used to support mandated PAS may be better allocated to other approaches to assure safety.

Introduction

Failure of medical devices, particularly permanent implanted devices, carries substantial risk of serious harm. Post-marketing surveillance is critical to evaluating the safety and efficacy of these devices, which generally enter the U.S. market with less clinical data than pharmaceutical agents.1, 2 A recent evaluation demonstrated that Food and Drug Administration (FDA) mandated post-approval studies (PAS) often fail to provide useful clinical information for either regulators or treating physicians.3 While the high costs of premarket studies have been documented4, there has been limited exploration of the costs of mandated PAS which are financially supported by device manufacturers, but whose costs are ultimately borne by healthcare consumers. Given recent efforts to improve the postmarketing surveillance of medical devices5, 6, we sought to better understand the costs to manufacturers of FDA-mandated PAS. We hypothesized that the aggregate costs of PAS, as currently performed, may significantly exceed their value in helping to inform regulators, healthcare providers and the public as to the safety and efficacy of recently approved medical devices. Because the actual costs of PAS borne by the manufacturers are considered confidential business information, there is currently no publicly available inventory of such costs. Therefore, we sought to develop cost estimates for PAS for medical devices by generating a cost model that could be used to predict the cost of a proposed or completed PAS.

Methods

Overall Study Design

We sought to estimate the cost of recently FDA-mandated PAS studies by developing a cost estimation model through iterative structured interviews with domain area experts from clinical trial design arena, using a modified Delphi approach to develop cost estimation consensus7. Participants were interviewed in two phases, with the development of a cost estimation model after Phase I and calibration of the cost model after Phase II. The final cost model was then applied to the existing FDA PAS database to estimate the costs to manufacturers of PAS studies ordered by FDA during the study period of 2005 through 2013.

Interviews of Expert Participants and Development of Initial Cost Estimation Model

We interviewed experts from the medical device manufacturing industry, academic research organizations/clinical research organizations, and experts who act as site level investigators at clinical investigative centers. These experts were identified by the investigators based on reputation and expertise in the field of medical device development and investigation. A total of 14 subjects were invited to participate, 10 accepted and were interviewed (4 from the medical device industry, 4 from academic research organizations, and 2 from clinical investigative centers).

Experts were interviewed in two phases in accordance with a modified Delphi approach to developing consensus from an expert panel7. During the Phase I interviews, participating experts were asked structured open-response questions to determine how the participant estimates the costs of post-marketing surveillance studies and what prior experience or knowledge was used in the estimation. The participants were then asked structured questions asking them to provide numerical total cost estimates for fifteen hypothetical post-approval study scenarios in order to isolate how different study design factors would affect the participants’ estimates of the cost of the PAS scenario.

The results of the phase I interviews was used to construct a preliminary cost model, which was then re-calibrated based on the results of a second set of expert interviews. For the purposes of this exploration, we focused our estimates on the costs to the medical device industry related to execution of the post-approval studies. We explicitly excluded the cost of the approved medical device itself, as well as all costs associated with “usual care” that included the clinical evaluation of patients receiving the devices. During the Phase II interviews, participants were re-contacted and asked to estimate the costs of three additional scenarios in order to calibrate the model generated from Phase I.

PAS Study Scenarios and Cost Estimation Model Development

The 15 hypothetical PAS scenarios represented a broad spectrum of study types which varied the organ system, device type, study design (prospective randomized vs. registry), presence or absence of a concurrent control population, number of study sites, duration of subject follow-up and geography of study-site locations. The study design features that were altered in the hypothetical scenarios were selected based on the general availability of those features within the FDA PAS database (see below). Each scenario was designed such that only one study feature varied from the previous scenario in order to isolate the estimated impact of the change in the study design feature on the overall cost of the study. The 15 PAS scenarios are summarized in Table 1, and a detailed description of each scenario, as provided to the study participants are provided in Appendix 1

Table 1
Summary of design features of the Phase I interview Post-Approval study scenarios.

The initial cost estimation model was developed from the participating expert estimates for each scenario by calculating the change in the median of the cost estimate for the scenario as compared to an alternative scenario in which there was only one study design factor difference. We then assumed that the difference in median study costs was fully attributable to the single changed study feature. The interview process identified three principle categories responsible for overall PAS cost: the infrastructure costs associated with any research study (per study costs or study overhead), the size of the study (per subject costs) and the number of sites (per site costs). We therefore allocated the incremental cost for the change in one study feature to one of three major components: per study costs, per subject costs, and per study site costs based on the general budgeting approaches described by the domain expert participants. The initial cost estimation model was then fit to the median scenario cost predictions by iteratively adjusting weights for each cost component for each study feature. The weight adjustment process was continued until the cost model was able to generate estimates that accurately predicted the median overall cost estimates for the scenarios.

In Phase II the initial cost estimation model was applied to three new hypothetical PAS scenarios, which were then reviewed, using structured interview techniques, with the expert panel participants.

FDA PAS database and Estimated Total PAS Costs

The final cost estimation model was applied to each of the post-approval studies in a dataset provided by FDA reflecting all FDA-mandated post-approval studies for medical devices from 3/1/05 through 6/30/13, based on the publicly available PAS database, supplemented by a second data file provided by FDA to address missing data in the original database. Studies were excluded from the cost estimating exercise if they were categorized as “bench” or “laboratory” studies.

The costs estimates were intended to reflect the budgeted costs that would be anticipated if the study was conducted as originally planned. Therefore, no cost savings are assumed for incomplete or slowly enrolling studies, nor are cost overruns assumed for studies that are extended or expanded beyond the PAS filing as detailed to the FDA.

Results

Twelve experts, with a median professional experience of 14.5 years in clinical trial design (combined 149 years of experience), participated in the Phase I structured interview process. The participants reported leadership roles in the design or implementation of 139 post approval studies (median of 9.5 PAS per participant, interquartile range (IQR): 5.0-18.75). The participants rated the total number of subjects enrolled in a PAS, the frequency (and type of clinical follow-up), the use of randomization, and the inclusion of a concurrent control group as the most important trial design features influencing overall cost of the PAS. The participants rated their confidence in estimating the costs of PAS as 8.0 on a 10 point scale (IQR: 7.0 – 8.0).

PAS scenario 1 served as the base case on which all other scenarios were based, and was to include 1000 patients followed for three years following the implant of a coronary stent, the median estimate for the total cost of the study was $5.75 million (IQR: $3.35 - $9.25 million). An initial cost estimation model was developed and fit to the Phase I survey results, as described above, demonstrated excellent correlation (r2 = 0.996) to the median expert estimated costs for the 15 hypothetical PAS scenarios (Figure 1). During the Phase II interviews, the cost estimation model was applied to three additional hypothetical PAS scenarios (see Appendix 2), and all 10 participants were re-interviewed to review the assumptions of the model and ask the participants to refine the model as applied to the new scenarios. The Phase II interviews confirmed consensus agreement of the cost estimation model by all 10 participants and there were no additional changes to the cost estimation model.

Figure 1
Correlation of initial cost estimation model to median expert participant estimates

Table 2 provides details of the final cost estimation model which includes a simple “base-case” PAS that is then adjusted through assigning additional costs based on study features beyond those included in the base case. The base case study assumed two year extended follow-up of an existing cohort of 1,000 patients exposed to the device as part of premarket study of 100 enrolling centers, with phone surveys of the patients performed at 12 and 24 months. Additional study features included in the final model include: the need to recruit subjects, inclusion of control group, requiring randomization for treatment, type of clinical follow up (by phone, in person, with imaging or with an invasive study), extending the follow-up period beyond two years, adjustment of costs for very large studies as well as adjustments for the proportion of sites enrolling in the study that were located outside the U.S. and for those studies that were not of the cardiovascular system (which was universally identified by the participants as the highest cost medical device PAS to execute).

Table 2
Final PAS cost estimation model.

The FDA PAS dataset included 293 unique PAS approved between March 2005 and June 2013, of which 16 were excluded since they were categorized as “bench” or “laboratory” studies. Over the 8.25 years analyzed, a mean of 33 PAS were initiated in each year. As shown in Table 3, among the 277 PAS evaluated in this analysis, 203 (77%) were mandated in three organ systems: cardiovascular (138), orthopedics (39) and plastic surgery (36). Only 23 (8.3%) of studies included randomized control populations (the most expensive design, as estimated by expert participants), and 108 (39%) included no control group at all (the least expensive PAS design). To account for the very high predicted costs of three specific studies (1.1% of total PAS), a conservative assumption was used to cap the maximum budget for any individual PAS at the 99th percentile of estimated costs of the PAS dataset; $35 million.

Table 3
Results of final cost estimation analysis of the FDA PAS database

Application of the final cost estimation model to all 277 clinical PAS demonstrated an estimated median cost of $2.16 million per individual PAS with an overall budgeted cost of $1.22 billion during the 8.25 year time period. Therefore, the average “budgeted” cost of PAS was $146 million per year. The principal cost drivers of individual PAS were the number of subjects and study sites, as well as the inclusion of active control groups, the need for new subject recruitment, and randomization. Based on the distribution of these study features, there were substantial differences in the median cost per study as stratified by organ system (Figure 2).

Figure 2
Median study costs ($) by organ system.

Conclusions

We conservatively estimate that more than $1.22 billion was planned to be spent on FDA-mandated PAS for medical devices approved between March 2005 and June 2013, representing more than $145 million per year anticipated to be spent by medical device manufacturers on these studies. This annual budgeted expenditure is intended to support the assessment of the safety and effectiveness of these recently approved devices. However, as recent investigators have reported, traditional PAS offer limited value from either a regulatory or clinical standpoint3. The sizeable annual expenses of PAS, borne by the public through the costs of medical devices themselves, may provide relatively modest informational value to the medical community, the public and industry regulators.

However, the FDA has proposed new strategies intended to improve the post-market safety assessment of medical devices1,8. These include implementation of unique device identifiers, leveraging national clinical device registries, and further development of methodologies for “real time” monitoring of post-market information. Given the high costs and uncertain information value of traditional PAS, we advocate shifting resources from traditional PAS to support the building of a high-performance, continuous safety monitoring infrastructure for medical devices. Among the highest priority initiatives to consider, we would recommend focusing on cardiovascular devices (representing nearly 50% of PAS) and piloting the use of national registries to avoid the high costs of traditional patient recruitment. Collaboration between industry, FDA and other stakeholders to devise strategies to redirect current investment in PAS execution into higher value efforts would better serve the public interest and more efficiently invest the resources that are already being spent.

Acknowledgement

This work was supported by a research grant from the Pew Charitable Trusts, Washington, DC.

Funding Sources: This study was supported by a research grant from The Pew Charitable Trusts, Washington D.C. In addition, Dr. Matheny is supported by Veterans Health Administration HSR and D CDA-08-020 and IIR-11-292. Dr. Normand is supported by FDA MDEpiNet Medical Counter Measure Research Study: U01-FD004493. Dr. Resnic is is supported by FDA research grant U01-FD004493, FDA MDEpiNet Methodology Center Contract HHSF223201110172C, and FDA MDEpiNet Medical Counter Measure Research Contract: U01-FD004493.

Abbreviations

PAS
Post-approval study
FDA
Food and Drug Administration

Appendix

Appendix 1

Phase I Interviews hypothetical post-approval study scenario descriptions.

Scenario 1

The proposed is a post-marketing surveillance study evaluating the efficacy of a new implantable cardiac device called Widget A. The device is a next generation coronary stent that is implanted in a cardiac catheterization laboratory. The proposed study will continue follow-up of a pre-market cohort of 1,000 patients against historic controls. Both men and women, ages 50-74 are included. The patients will be followed for 3 years, yearly by telephone, with mandated re-look angiography after 1 year. The study will be conducted at 100 US-only sites.

Scenario 2

Implanted cardiac device such as a coronary stent study as above, but the study population is a new cohort, not a pre-market cohort

The proposed is a post-marketing surveillance study evaluating the efficacy of a new implantable cardiac device called Widget A. The device is a next generation coronary stent that is implanted in a cardiac catheterization laboratory. The proposed study will evaluate a new cohort of 1,000 patients against historic controls. Both men and women, ages 50-74 are included. The patients will be followed for 3 years, yearly by telephone, with mandated re-look angiography after 1 year. The study will be conducted at 100 US-only sites.

Scenario 3

Implanted cardiac device such as a coronary stent study as above, but the follow-up period is extended by 2 years to 5 total

The proposed is a post-marketing surveillance study evaluating the efficacy of a new implantable cardiac device called Widget A. The device is a next generation coronary stent that is implanted in a cardiac catheterization laboratory. The proposed study will evaluate a new cohort of 1,000 patients against historic controls. Both men and women, ages 50-74 are included. The patients will be followed for 5 years, yearly by telephone, with mandated re-look angiography after 1 year. The study will be conducted at 100 US-only sites.

Scenario 4

Implanted cardiac device such as a coronary stent study as above, but the number of sites is halved from 100 to 50

The proposed is a post-marketing surveillance study evaluating the efficacy of a new implantable cardiac device called Widget A. The device is a next generation coronary stent that is implanted in a cardiac catheterization laboratory. The proposed study will evaluate a new cohort of 1,000 patients against historic controls. Both men and women, ages 50-74 are included. The patients will be followed for 5 years, yearly by telephone, with mandated re-look angiography after 1 year. The study will be conducted at 50 US-only sites.

Scenario 5

Implanted cardiac device such as a coronary stent study as above, but the sites are international as opposed to just US-only

The proposed is a post-marketing surveillance study evaluating the efficacy of a new implantable cardiac device called Widget A. The device is a next generation coronary stent that is implanted in a cardiac catheterization laboratory. The proposed study will evaluate a new cohort of 1,000 patients against historic controls. Both men and women, ages 50-74 are included. The patients will be followed for 5 years, yearly by telephone, with mandated re-look angiography after 1 year. The study will be conducted at 50 US and international sites.

Scenario 6

Implanted cardiac device such as a coronary stent study as above, but without mandatory re-look angiography. Instead, clinical follow-up with phone contact is performed for 3 years

The proposed is a post-marketing surveillance study evaluating the efficacy of a new implantable cardiac device called Widget A. The device is a next generation coronary stent that is implanted in a cardiac catheterization laboratory. The proposed study will evaluate a new cohort of 1,000 patients against historic controls. Both men and women, ages 50-74 are included. The patients will be followed for 5 years, with yearly phone contact only. The study will be conducted at 50 US and international sites.

Scenario 7

Implanted cardiac device such as a coronary stent study as above, but with a 5,000 patient cohort compared to a 1,000 patient cohort

The proposed is a post-marketing surveillance study evaluating the efficacy of a new implantable cardiac device called Widget A. The device is a next generation coronary stent that is implanted in a cardiac catheterization laboratory. The proposed study will evaluate a new cohort of 5,000 patients against historic controls. Both men and women, ages 50-74 are included. The patients will be followed for 5 years, with yearly phone contact only. The study will be conducted at 50 US and international sites.

Scenario 8

Implanted biliary device such as a biliary stent study as above, compared with a coronary stent

The proposed is a post-marketing surveillance study evaluating the efficacy of a new implantable biliary device called Widget B. The device is a next generation biliary stent that is implanted in an endoscopy suite during ERCP (endoscopic retrograde cholangiopancreatography). The proposed study will evaluate a new cohort of 5,000 patients against historic controls. Both men and women, ages 50-74 are included. The patients will be followed for 5 years, with yearly phone contact only. The study will be conducted at 50 US and international sites.

Appendix

Appendix 2

An example of a Phase II Interview PAS cost estimation using initial cost model.

Scenario: The proposed study is a post-marketing surveillance study evaluating the efficacy of a new implantable cardiac pacemaker device called Pacer-A. The device is a next generation leadless pacemaker that is implanted in a cardiac electrophysiology laboratory. The proposed study will evaluate a new cohort of 250 patients who receive Pacer-A for bradycardia compared to 250 controls who will receive a traditional pacemaker for a similar indication. The subjects will not be randomized, but will be given the choice of either device. Both men and women, ages 50-74 are included. The patients will be followed for 3 years, with yearly telephone interviews as well as a required cardiac event monitor after Year 1. The study will be conducted at 30 US-only sites. Study endpoints will be independently adjudicated.
Per Subject CostsPer Site CostsStudy OverheadNotes:
Base Case:Assumes a 2yr observational follow up of existing study cohort, with one phone evaluation per year$600$2,800$400,000

Adjustments:Need to Recruit Subjects$2,600$3,250$325,000If de novo cohort required
Adding a control group (1:1)$325$406$40,625Due to complexity of recruiting patients to two therapies
Requiring Randomization$1,300$1,625$162,500
Additional Phone evaluations per year$28$129$-Per additional phone interview per year
In-Person clinical evaluations$138$644$-Per additional in-person clinical evaluation per year
Imaging study required$1,000$250$50,000Examples: Chest CT, renal ultrasound, echocardiogram
Required invasive study$2,000$500$100,000Examples: coronary angiography, EGD, ERCP
Extending additional year (w/ one eval per year)$198$924$264,000per year costs
Large Study Adjustments:
Overhead Costs for >1000 subjects$-$-$200,000per 1000 additional subjects
Overhead Costs for >100 sites$-$-$300,000per 100 additional sites
Reduced base-case cost if OUS sites$480$2,240$320,000Based on proportion of study performed OUS
Discount for non CV Organ system device study$(120)$(560)$(80,000)

$2,500,000 $256,500 $1,343,700

Total cost: $4,100,000

Footnotes

Authors’ Contributions: Drs. Wimmer and Resnic were responsible for the design and execution of this study, had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Drs. Wimmer and Resnic, as well as Ms. Robbins, Yang, and Mr. Ssemaganda, conducted and are responsible for the data analysis. Dr. Normand and Dr. Matheny provided statistical and methodological guidance in the development of the study, and critical revisions to the manuscript. Dr. Rising and Ms. Herz assisted with interpretation of the data and in the writing of the report.

Conflicts of Interest: No author reports any conflict relevant to this study.

References

1. Resnic FS, Normand SL. Postmarketing surveillance of medical devices--filling in the gaps. The New England journal of medicine. 2012;366(10):875–7. [PubMed]
2. Hines JZ, Lurie P, Yu E, Wolfe S. Left to their own devices: breakdowns in United States medical device premarket review. PLoS medicine. 2010;7(7):e1000280. [PMC free article] [PubMed]
3. Reynolds IS, Rising JP, Coukell AJ, Paulson KH, Redberg RF. Assessing the safety and effectiveness of devices after US Food and Drug Administration approval: FDA-mandated postapproval studies. JAMA internal medicine. 2014;174(11):1773–9. [PubMed]
4. Kaplan AV, Baim DS, Smith JJ, Feigal DA, Simons M, Jefferys D, et al. Medical device development: from prototype to regulatory approval. Circulation. 2004;109(25):3068–72. [PubMed]
5. Daniel GW, McClellan MB, Colvin H, Aurora P, Khaterzai S. [May 10, 2015];Strengthening Patient Care: Building an Effective National Medical Device Surveillance System. http://www.brookings.edu/research/papers/2015/02/23-medical-device-postmarketsurveillance-roadmap-daniel.
6. FDA [May 10, 2015];Strengthening Our National System for Medical Device Postmarket Surveillance. 2013 http://www.fda.gov/AboutFDA/CentersOffices/OfficeofMedicalProductsandTobacco/CDRH/CDRHReports/ucm301912.htm.
7. Spertus JA, Eagle KA, Krumholz HM, Mitchell KR, Normand SL. American College of Cardiology and American Heart Association methodology for the selection and creation of performance measures for quantifying the quality of cardiovascular care. Circulation. 2005;111:1703–1712. [PubMed]
8. Normand SL, Hatfield L, Drozda J, Resnic FS. Postmarket surveillance for medical devices: America's new strategy. Bmj. 2012;345:e6848. [PubMed]