|Home | About | Journals | Submit | Contact Us | Français|
Radiation oncology relies on rapidly evolving technology and highly complex processes. The US Food and Drug Administration collects reports of adverse events related to medical devices. We sought to characterize all events involving radiation oncology devices (RODs) from the US Food and Drug Administration’s postmarket surveillance Manufacturer and User Facility Device Experience (MAUDE) database, comparing these with non–radiation oncology devices.
MAUDE data on RODs from 1991 to 2015 were sorted into 4 product categories (external beam, brachytherapy, planning systems, and simulation systems) and 5 device problem categories (software, mechanical, electrical, user error, and dose delivery impact). Outcomes included whether the device was evaluated by the manufacturer, adverse event type, remedial action, problem code, device age, and time since 510(k) approval. Descriptive statistics were performed with linear regression of time-series data. Results for RODs were compared with those for other devices by the Pearson χ2 test for categorical data and 2-sample Kolmogorov-Smirnov test for distributions.
There were 4234 ROD and 4,985,698 other device adverse event reports. Adverse event reports increased over time, and events involving RODs peaked in 2011. Most ROD reports involved external beam therapy (50.8%), followed by brachytherapy (24.9%) and treatment planning systems (21.6%). The top problem types were software (30.4%), mechanical (20.9%), and user error (20.4%). RODs differed significantly from other devices in each outcome (P<.001). RODs were more likely to be evaluated by the manufacturer after an event (46.9% vs 33.0%) but less likely to be recalled (10.5% vs 37.9%) (P<.001). Device age and time since 510(k) approval were shorter among RODs (P<.001).
Compared with other devices, RODs may experience adverse events sooner after manufacture and market approval. Close postmarket surveillance, improved software design, and manufacturer–user training may help mitigate these events.
Modern radiation oncology relies on rapidly evolving technology of staggering complexity. The American Association of Physicists in Medicine has made recommendations for safe operation of medical accelerators (1–5), and quality and safety efforts from groups including the American Society for Radiation Oncology (ASTRO) have become more comprehensive (6). However, technological evolution in radiation oncology allows opportunities for errors. Of all devices, linear accelerators were the most recalled from 2003 to 2012 according to the US Food and Drug Administration (FDA), most often because of software-related problems (7). The FDA wrote to manufacturers of radiation oncology devices (RODs) in 2010, concerned about underdosing, overdosing, and misaligned exposures (8). Radiation oncology has also received national attention in recent years because of catastrophic and avoidable errors (9–11).
Thus, critical questions arise given the field’s reliance on such sophisticated devices. How often do these devices malfunction, and why? How many device-related injuries or deaths have occurred? Are there meaningful differences in the characteristics of device-related adverse events within radiation oncology and compared with other fields?
The FDA divides medical devices into classes I through III. The vast majority of radiation therapy (RT) devices are class II, cleared through a process of premarket notification, or “510(k)” after the relevant section in the Food, Drug, and Cosmetic Act (21 USC §360) (12). This requires demonstration that the device is substantially equivalent in safety and efficacy to a legally marketed device. This rather limited and short review process is supplemented by postmarket surveillance (13).
The Manufacturer and User Facility Device Experience (MAUDE) database is the primary postmarket surveillance tool the FDA uses to monitor device performance and detect device-related safety issues. The MAUDE database includes medical device reports submitted to the FDA by mandatory reporters (manufacturers, importers, device user facilities) and voluntary reporters (health care professionals, patients, consumers). Manufacturers must submit reports when they become aware of information suggesting one of their devices may have caused or contributed to a death or serious injury or has malfunctioned with potential to cause death or serious injury.
Previous studies have used MAUDE data to analyze a variety of medical technology malfunctions, from coronary stent fracture (14) to health information technology (15–17). However, there are no previous studies using MAUDE data on therapeutic radiation devices. We sought to characterize all adverse events involving RODs and to compare these with events from non–radiation oncology devices.
MAUDE data of all adverse events through December 31, 2015, were obtained (18). Downloaded data include voluntary reports since June 1993, user facility reports since 1991, distributor reports since 1993, and manufacturer reports since August 1996 (18). The FDA 510(k) device preapproval database was also obtained (19), which contained all releasable 510(k) approvals from July 15, 1976, through December 31, 2015. Data files were imported and manipulated using R (20, 21).
MAUDE is a relational database; each report is a master record associated with one or more devices and one or more patients (though most records include one device and one patient only). Master records contain global information, such as date of event and manufacturer name and address. Each device record linked to a master record contains device-specific information, such as product code and 510(k) number. The outcome of interest was device associated with an adverse event report. Therefore, the master and device database tables were merged, returning complete data for each device that was associated with a report.
Each device is classified by a 3-letter product code. We identified 39 product codes regulated under a subsection of federal code specifically for therapeutic radiation devices (21 CFR §892, Subpart F), designating these as RODs (22). We also identified 4 more therapeutic radiation product codes by manual search outside of the aforementioned subsection, resulting in 43 product codes representing RODs (Table E1; available online at www.redjournal.org).
Among the 43 aforementioned product codes, we found 23 existed in the database. These devices were further classified into 4 categories (Table E3; available online at www.redjournal.org). “External beam” includes linear accelerators, cobalt-60 units, and associated accessories such as blocks and couches; “brachytherapy” includes source radionuclides, afterloading systems, and intravascular radiation; “planning system” refers to the product code MUJ (“System, Planning, Radiation Therapy Treatment”); and “simulation” refers to the product code KPQ (“System, Simulation, Radiation Therapy”).
Each device record in MAUDE is associated with one or more problem codes. We identified the 92 problem codes associated with RODs and grouped them into the following categories, shown in Table E4 (available online at www.redjournal.org): software, mechanical, electrical, other, user error, unknown, and dose delivery impact (eg, “radiation exposure, unintended”).
MAUDE data were merged with the 510(k) approval database by linking 510(k) number. Time since 510(k) approval was computed by subtracting the 510(k) approval date from the date the report was received.
Outcomes included whether or not the device was evaluated by the manufacturer, event type, remedial action, problem code, and device age (computed as date of report minus date of manufacture). Remedial action means any action other than routine maintenance or servicing of a device where such action is necessary to prevent recurrence of a reportable event.
We undertook a detailed analysis of “death” event reports. We characterized the number and time course of these events and device product categories involved. We reviewed the description of each event and determined the number of deaths involved, whether or not a true device malfunction causing death was described, and appropriate device problem classifications.
Descriptive statistics of database characteristics were obtained. Linear regression was performed for time-series data. Results for RODs were compared with other devices by the Pearson χ2 test for categorical data and the 2-sample Kolmogorov-Smirnov test for distributions (23). Two-tailed P<.05 was considered significant.
Database characteristics are shown in Table 1. There were 4234 RODs associated with adverse events. The number of such devices increased over each 5-year period, with the most in 2006 to 2010 (32.6%) and 2011 to 2015 (32.4%). There were 4,985,698 other devices associated with adverse reports. These also increased over time, with the majority reported from 2011 to 2015 (65.5%). RODs and other devices differed in their distribution across time (P<.001). Most reports for RODs involved external beam systems (50.8%), followed by brachytherapy devices (24.9%) and planning software (21.6%).
Report sources differed between devices (P<.001). While manufacturers submitted the majority of reports for both RODs and other devices, voluntarily submitted reports were more frequent for RODs than other devices (11.7% vs 2.1%).
The number of reports peaked in 2014 for other devices and in 2011 for RODs (Fig. 1A). The number of reports increased over time for all other devices (linear regression; β = 29,059 reports per year, R2=0.7, P<.001), external beam devices (β = 8.6 reports per year, R2=0.52, P<.001), and brachytherapy devices (β = 2.6 reports per year, R2=0.22, P=.01). Reports among planning and simulation systems did not change significantly over time. External beam reports peaked in 2011 (n=310), and brachytherapy reports were highest in 2002 (n=137). Planning system reports were highest in 1999 (n=167), and the most simulation system reports were in 2007 (n=16) (Fig. 1B).
Outcomes of adverse events are shown in Table 2. RODs were evaluated by the manufacturer more often (46.9%) than were other devices (33.0%, P<.001).
The most common adverse event among RODs and other devices was malfunction (62.4% and 60.9%, respectively). Injury was more common among other devices (33.7%). Deaths were rare (only 2.3% for both types of devices). The highest number of reports of deaths associated with RODs was in 2003 (n=28); for other devices, it was in 2014 (n=24,841) (Fig. 1B).
Remedial action was reported for 972 ROD reports (23.0%) and 474,783 other device reports (9.5%). The most common remedial action for RODs was notification (43.6%), while the most common remedial action for other devices was recall (37.9%) (Table 2, Fig. 2).
RODs were associated with 251 reports that recorded 339 problem codes. The top 3 categories were software (30.4%), mechanical (20.9%), and user error (20.4%) (Table 2).
Among 4234 RODs, 2258 (53.3%) had a date of manufacture recorded. For these devices, the adverse event occurred at an average device age of 544 days (range, 12–13,136 days). Among other devices, 3,047,057 (61.1%) had a date of manufacture recorded. The average device age at the time of the event was 753 days (range, 1–21,944 days). The distribution function for device age is shown in Figure 3A. The probability distributions differed significantly (D = 0.31, P<.001).
The FDA stopped requiring baseline 510(k) numbers on reports after 2008. Of the 4234 RODs, 361 (8.5%) had a valid 510(k) number recorded. For these devices, the adverse event occurred an average of 1450 days (range, 24–7683 days) after 510(k) approval. For other devices, 291,351 (5.8%) had 510(k) numbers, and the average time of event since 510(k) approval was 2605 days (range, 1–11,261 days). The distribution functions for time since 510(k) approval are shown in Figure 3A. The probability distributions differed significantly (D = 0.47, P<.001).
The average and standard deviation of years since 510(k) clearance for adverse events over time (1994–2008) are shown in Figure 3B. The time since 510(k) clearance for RODs tended to decrease over this time frame (β = −0.52 years/year, P =.09), while it increased for other devices (β = 0.44 years/year, P =.01).
There were 98 reports of patient deaths as adverse events for RODs (Table 2). Manual inspection revealed 2 duplicate reports, resulting in 96 unique reports of patient deaths. Most deaths occurred from 2001 to 2005 (45.8%), and most were associated with brachytherapy devices (72.4%). Of external beam reports, 0.7% were for deaths. The proportions of deaths for brachytherapy devices, planning systems, and simulation systems were 6.7%, 1%, and nil, respectively. These proportions differed significantly (P<.001).
A single death report sometimes described multiple deaths associated with an incident. The total number of deaths described within the 96 reports was 103. Of the total deaths, 22 (21.4%) were clearly attributable to a device. External beam device errors were responsible for 11 of these deaths (50%), planning systems for 9 (40.9%), brachytherapy devices for 2 (9.1%), and simulation systems for none. The remainder were due to natural causes or an expected toxicity or complication of treatment. Thirteen deaths (12.6%) were described from reports in the medical literature, none of which were directly attributable to a device.
Of the 96 reports, 18 referenced true device problems, with a total of 28 problems. Most were either user errors (32.1%) or overdose (32.1%).
To our knowledge, this is the first comprehensive analysis of adverse events among RODs from the FDA’s MAUDE database. Reports involving RODs comprised about 0.1% of all reports, and the number of reports increased over time, peaking in 2011. This peak may correspond to “increased awareness prompted by targeted interactions with industry” (7), increased media attention (9–11), and/or widespread adoption of volumetric arc therapy. Most ROD reports involved external beam devices, consistent with these devices being involved in a high percentage of all radiation oncology patient encounters.
Event outcomes differed between RODs and other devices. While most device events involved malfunction, patient injury was less common among RODs (16.3% vs 33.7% in other devices). This finding suggests that while ROD adverse events had potential for causing injury, they were caught before this happened. RODs were more often evaluated by the manufacturer, implying that ROD vendors may be more closely involved after device purchase and proactive in responding after a device issue. Manufacturers may not evaluate a device if the problem is already known (24), so it may also be that RODs are more likely to experience unknown problems. Adverse events involving RODs more often led to notifications with fewer recalls than other devices, representing immediate action taken at the time of report submission. The lower incidence of recalls at the time of event report could be an issue of reporting, though the FDA concluded that ROD manufacturers improved recall reporting from 2003 to 2012 (7).
Software-related issues were the most common among ROD problem codes. Other authors have raised concern about the incidence of software problems (25, 26). The FDA found linear accelerations to be the most-recalled medical device from 2003 to 2012, mostly because of software issues (7). Problems with interoperability, user interface, and dose calculation accounted for more than two-thirds of these linear accelerator recalls. Indeed, software design was tied as the top reason for all medical device recalls from 2010 to 2012 (7). In 2010, the FDA wrote to manufacturers of RODs regarding concerns about underdoses, overdoses, and misaligned exposures from therapeutic radiation. These data sparked a public workshop discussing safeguards and controls, as well as changes to premarket testing and premarket review of software (8). More recently, a 2015 ASTRO proceeding revealed software issues in RT are still very much of interest to the FDA (27).
User error was common, accounting for 20.4% of all ROD problems. Human fallibility has been shown to seriously affect the delivery of RT (28), which points to the need for better manufacturer–user training and better design. This issue has not escaped notice by the FDA, whose Human Factors Engineering–Usability Engineering team evaluates how people interact with technology and how user interface design affects these interactions. No ROD, however, made the recent short list of highest-priority devices for human factors review (29). Any premarket design, postmarket regulation, or quality-improvement effort must clearly address user error and human factors.
RODs experienced adverse events sooner after their manufacture compared with other devices, as well as sooner after their market approval. The time to adverse event since 510(k) clearance also tended to decrease during the evaluable period. This finding could reflect the relative age of most RODs in use in the United States compared with other devices; most linear accelerators tend to be replaced every 7 to 9 years, with software upgrades occurring even more often. These data also suggest that there may not be enough premarket testing of RODs prior to device approval. In part, this reflects problems within the current 510(k) system. The Institute of Medicine found the current 510(k) process inadequate and recommended its replacement with “an integrated premarket and postmarket regulatory framework that effectively provides a reasonable assurance of safety and effectiveness throughout the device life cycle” (30). Other criticisms of the process include reliance by the FDA on manufacturer data only, outsourced and inadequate reviews performed by third parties (13), and new devices becoming significantly different than the original equivalent device (“predicate creep”) (31). Our data suggest that the first 2 years from manufacture and first 5 years from 510(k) approval appear to be the most critical in which to focus safety efforts among RODs. Postmarket monitoring and close user–manufacturer training during this time may mitigate many adverse events.
Deaths, especially those directly attributable to device malfunction or misuse, were rare ROD events on par with the low incidence for all devices. Among deaths directly attributable to RODs, user error and overdoses were the most common problems. Because manufacturers are required to report any event they become aware of, including events in the medical literature, these types of reports were common despite their frequent lack of connection to an actual malfunction. Radioactive microspheres were responsible for the high proportion of brachytherapy devices involved in deaths. In nearly all cases, however, we found these were expected side effects or disease progression, unrelated to a specific device malfunction. Their reporting may be explained by one of the manufacturer’s reports: “Hepatorenal failure is listed in the product package insert and is a known side effect…However, this side effect was inadvertently excluded from the clinical trial protocol hence the assessment of this event as an unexpected adverse device effect requiring reporting” (32).
There are several limitations to the MAUDE database. Because MAUDE is a passive surveillance system, it may suffer from underreporting. Data submissions may be incomplete, inaccurate, untimely, or biased. Our recoding of identifiable misclassifications and organization of the data into broader categories should mitigate some of these problems but cannot prevent them entirely. Our 510(k) findings are limited by the low percentage of records containing this information. To our knowledge, there is no systemic bias regarding the availability of these data; however, caution is warranted in extrapolating the findings to all devices. There is also no information about frequency of device use; therefore, we cannot determine the prevalence or incidence of adverse events per device use or patient treatment course. Furthermore, while problem codes “describe device failures or issues related to the device” according to the FDA (33), these are not necessarily root causes and the specific causal factors are often indeterminable. Despite its limitations, we believe the MAUDE database can inform radiation oncology safety practices, and the FDA itself has relied on this information in focusing its efforts (7, 8).
There are many exciting developments to improve device safety on the horizon. The FDA will soon track “emerging signals” that a medical device may pose risks (34), and it is also revamping the MAUDE system (35). The Global Unique Device Identification Database is being phased in and will allow for more accurate reports of adverse events, more effective management of recalls, and reduction of medical errors (36), as well as more consistent availability of linked data such as 510(k) number. However, our data suggest RODs are substantially different from other devices, and a one-size-fits-all approach may not suffice. We agree with consensus reports that the complexity of processes in radiation oncology calls for a discipline-specific system (37). In the absence of a national mandatory reporting database specifically for radiation oncology, the Radiation Oncology Incident Learning System (RO-ILS) of ASTRO and the American Association of Physicists in Medicine has gained considerable traction in its first year (38). The system has already yielded valuable information beyond MAUDE’s capabilities. For example, 34% of 2014 quarter 4 RO-ILS events were near misses caught by checklists, time-outs, and patient vigilance, and radiation therapists discovered a majority of events (38). These data provide specific and actionable information within the broader scope of issues raised using MAUDE.
Compared with other devices, RODs experience adverse events sooner after manufacture and market approval. RODs are more likely to be evaluated by the manufacturer after an event but less likely to be recalled than other devices. Given the high rate of issues involving human interface among RODs—software problems, user errors, and notifications as remedial action—it is critical to involve professionals at every stage of RT delivery in quality improvement and reporting. We must ensure that rapid innovation in radiation oncology is applied not only to our devices and our technology but also to our systems for ensuring quality and safety. Information from centralized, mandated reporting databases such as MAUDE should inform ongoing efforts to improve patient safety.
Three decades of US Food and Drug Administration data show that adverse events involving radiation oncology devices differ significantly from other devices in adverse event type, remedial action, problem code, device age, time from 510(k) market clearance to event, and whether the device was evaluated by the manufacturer. The first 2 years from manufacture and first 5 years from 510(k) approval appear to be the most critical in which to focus safety efforts among radiation oncology devices.
This work was partially supported by the following grants: National Institutes of Health grants KL2TR00099 and UL1TR000100 (J.A.H.-G.), American Cancer Society Pilot Award ACS-IRG 70-002 (J.A.H.-G.), and National Cancer Institute cancer center specialized grant P30CA023100 (J.A.H.-G.).