PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Int J Radiat Oncol Biol Phys. Author manuscript; available in PMC 2018 January 1.
Published in final edited form as:
PMCID: PMC5193217
NIHMSID: NIHMS827734

Adverse Events Involving Radiation Oncology Medical Devices: Comprehensive Analysis of US Food and Drug Administration Data, 1991 to 2015

Abstract

Purpose

Radiation oncology relies on rapidly evolving technology and highly complex processes. The US Food and Drug Administration collects reports of adverse events related to medical devices. We sought to characterize all events involving radiation oncology devices (RODs) from the US Food and Drug Administration’s postmarket surveillance Manufacturer and User Facility Device Experience (MAUDE) database, comparing these with non–radiation oncology devices.

Methods and Materials

MAUDE data on RODs from 1991 to 2015 were sorted into 4 product categories (external beam, brachytherapy, planning systems, and simulation systems) and 5 device problem categories (software, mechanical, electrical, user error, and dose delivery impact). Outcomes included whether the device was evaluated by the manufacturer, adverse event type, remedial action, problem code, device age, and time since 510(k) approval. Descriptive statistics were performed with linear regression of time-series data. Results for RODs were compared with those for other devices by the Pearson χ2 test for categorical data and 2-sample Kolmogorov-Smirnov test for distributions.

Results

There were 4234 ROD and 4,985,698 other device adverse event reports. Adverse event reports increased over time, and events involving RODs peaked in 2011. Most ROD reports involved external beam therapy (50.8%), followed by brachytherapy (24.9%) and treatment planning systems (21.6%). The top problem types were software (30.4%), mechanical (20.9%), and user error (20.4%). RODs differed significantly from other devices in each outcome (P<.001). RODs were more likely to be evaluated by the manufacturer after an event (46.9% vs 33.0%) but less likely to be recalled (10.5% vs 37.9%) (P<.001). Device age and time since 510(k) approval were shorter among RODs (P<.001).

Conclusions

Compared with other devices, RODs may experience adverse events sooner after manufacture and market approval. Close postmarket surveillance, improved software design, and manufacturer–user training may help mitigate these events.

Introduction

Modern radiation oncology relies on rapidly evolving technology of staggering complexity. The American Association of Physicists in Medicine has made recommendations for safe operation of medical accelerators (15), and quality and safety efforts from groups including the American Society for Radiation Oncology (ASTRO) have become more comprehensive (6). However, technological evolution in radiation oncology allows opportunities for errors. Of all devices, linear accelerators were the most recalled from 2003 to 2012 according to the US Food and Drug Administration (FDA), most often because of software-related problems (7). The FDA wrote to manufacturers of radiation oncology devices (RODs) in 2010, concerned about underdosing, overdosing, and misaligned exposures (8). Radiation oncology has also received national attention in recent years because of catastrophic and avoidable errors (911).

Thus, critical questions arise given the field’s reliance on such sophisticated devices. How often do these devices malfunction, and why? How many device-related injuries or deaths have occurred? Are there meaningful differences in the characteristics of device-related adverse events within radiation oncology and compared with other fields?

The FDA divides medical devices into classes I through III. The vast majority of radiation therapy (RT) devices are class II, cleared through a process of premarket notification, or “510(k)” after the relevant section in the Food, Drug, and Cosmetic Act (21 USC §360) (12). This requires demonstration that the device is substantially equivalent in safety and efficacy to a legally marketed device. This rather limited and short review process is supplemented by postmarket surveillance (13).

The Manufacturer and User Facility Device Experience (MAUDE) database is the primary postmarket surveillance tool the FDA uses to monitor device performance and detect device-related safety issues. The MAUDE database includes medical device reports submitted to the FDA by mandatory reporters (manufacturers, importers, device user facilities) and voluntary reporters (health care professionals, patients, consumers). Manufacturers must submit reports when they become aware of information suggesting one of their devices may have caused or contributed to a death or serious injury or has malfunctioned with potential to cause death or serious injury.

Previous studies have used MAUDE data to analyze a variety of medical technology malfunctions, from coronary stent fracture (14) to health information technology (1517). However, there are no previous studies using MAUDE data on therapeutic radiation devices. We sought to characterize all adverse events involving RODs and to compare these with events from non–radiation oncology devices.

Methods and Materials

Data acquisition

MAUDE data of all adverse events through December 31, 2015, were obtained (18). Downloaded data include voluntary reports since June 1993, user facility reports since 1991, distributor reports since 1993, and manufacturer reports since August 1996 (18). The FDA 510(k) device preapproval database was also obtained (19), which contained all releasable 510(k) approvals from July 15, 1976, through December 31, 2015. Data files were imported and manipulated using R (20, 21).

MAUDE is a relational database; each report is a master record associated with one or more devices and one or more patients (though most records include one device and one patient only). Master records contain global information, such as date of event and manufacturer name and address. Each device record linked to a master record contains device-specific information, such as product code and 510(k) number. The outcome of interest was device associated with an adverse event report. Therefore, the master and device database tables were merged, returning complete data for each device that was associated with a report.

Product code classification

Each device is classified by a 3-letter product code. We identified 39 product codes regulated under a subsection of federal code specifically for therapeutic radiation devices (21 CFR §892, Subpart F), designating these as RODs (22). We also identified 4 more therapeutic radiation product codes by manual search outside of the aforementioned subsection, resulting in 43 product codes representing RODs (Table E1; available online at www.redjournal.org).

Device records were then inspected for any misclassifications or typographical errors. These records were manually recoded (Table E2; available online at www.redjournal.org).

Among the 43 aforementioned product codes, we found 23 existed in the database. These devices were further classified into 4 categories (Table E3; available online at www.redjournal.org). “External beam” includes linear accelerators, cobalt-60 units, and associated accessories such as blocks and couches; “brachytherapy” includes source radionuclides, afterloading systems, and intravascular radiation; “planning system” refers to the product code MUJ (“System, Planning, Radiation Therapy Treatment”); and “simulation” refers to the product code KPQ (“System, Simulation, Radiation Therapy”).

Problem classification

Each device record in MAUDE is associated with one or more problem codes. We identified the 92 problem codes associated with RODs and grouped them into the following categories, shown in Table E4 (available online at www.redjournal.org): software, mechanical, electrical, other, user error, unknown, and dose delivery impact (eg, “radiation exposure, unintended”).

Data merging

MAUDE data were merged with the 510(k) approval database by linking 510(k) number. Time since 510(k) approval was computed by subtracting the 510(k) approval date from the date the report was received.

Outcomes

Outcomes included whether or not the device was evaluated by the manufacturer, event type, remedial action, problem code, and device age (computed as date of report minus date of manufacture). Remedial action means any action other than routine maintenance or servicing of a device where such action is necessary to prevent recurrence of a reportable event.

Deaths

We undertook a detailed analysis of “death” event reports. We characterized the number and time course of these events and device product categories involved. We reviewed the description of each event and determined the number of deaths involved, whether or not a true device malfunction causing death was described, and appropriate device problem classifications.

Statistical analysis

Descriptive statistics of database characteristics were obtained. Linear regression was performed for time-series data. Results for RODs were compared with other devices by the Pearson χ2 test for categorical data and the 2-sample Kolmogorov-Smirnov test for distributions (23). Two-tailed P<.05 was considered significant.

Results

Database characteristics

Database characteristics are shown in Table 1. There were 4234 RODs associated with adverse events. The number of such devices increased over each 5-year period, with the most in 2006 to 2010 (32.6%) and 2011 to 2015 (32.4%). There were 4,985,698 other devices associated with adverse reports. These also increased over time, with the majority reported from 2011 to 2015 (65.5%). RODs and other devices differed in their distribution across time (P<.001). Most reports for RODs involved external beam systems (50.8%), followed by brachytherapy devices (24.9%) and planning software (21.6%).

Table 1
Characteristics of RODs versus all other medical devices in reported adverse events

Report sources differed between devices (P<.001). While manufacturers submitted the majority of reports for both RODs and other devices, voluntarily submitted reports were more frequent for RODs than other devices (11.7% vs 2.1%).

Events over time

The number of reports peaked in 2014 for other devices and in 2011 for RODs (Fig. 1A). The number of reports increased over time for all other devices (linear regression; β = 29,059 reports per year, R2=0.7, P<.001), external beam devices (β = 8.6 reports per year, R2=0.52, P<.001), and brachytherapy devices (β = 2.6 reports per year, R2=0.22, P=.01). Reports among planning and simulation systems did not change significantly over time. External beam reports peaked in 2011 (n=310), and brachytherapy reports were highest in 2002 (n=137). Planning system reports were highest in 1999 (n=167), and the most simulation system reports were in 2007 (n=16) (Fig. 1B).

Fig. 1
A, Reports over time by event type for radiation oncology devices (RODs) versus other devices. B, Reports over time by product code for RODs. Significant linear regressions are shown in the legend. The slope (β) indicates reports per year.

Event outcomes

Device evaluation

Outcomes of adverse events are shown in Table 2. RODs were evaluated by the manufacturer more often (46.9%) than were other devices (33.0%, P<.001).

Table 2
Medical device report outcomes

Event type

The most common adverse event among RODs and other devices was malfunction (62.4% and 60.9%, respectively). Injury was more common among other devices (33.7%). Deaths were rare (only 2.3% for both types of devices). The highest number of reports of deaths associated with RODs was in 2003 (n=28); for other devices, it was in 2014 (n=24,841) (Fig. 1B).

Remedial action

Remedial action was reported for 972 ROD reports (23.0%) and 474,783 other device reports (9.5%). The most common remedial action for RODs was notification (43.6%), while the most common remedial action for other devices was recall (37.9%) (Table 2, Fig. 2).

Fig. 2
Remedial actions for radiation oncology devices versus other devices.

Problem codes

RODs were associated with 251 reports that recorded 339 problem codes. The top 3 categories were software (30.4%), mechanical (20.9%), and user error (20.4%) (Table 2).

Device age

Among 4234 RODs, 2258 (53.3%) had a date of manufacture recorded. For these devices, the adverse event occurred at an average device age of 544 days (range, 12–13,136 days). Among other devices, 3,047,057 (61.1%) had a date of manufacture recorded. The average device age at the time of the event was 753 days (range, 1–21,944 days). The distribution function for device age is shown in Figure 3A. The probability distributions differed significantly (D = 0.31, P<.001).

Fig. 3
A, Probability distributions for device age and time since 510(k) approval for radiation oncology devices versus other devices. B, Years since 510(k) approval over time. The points show the average number of years since 510(k) approval for reports in ...

Time since 510(k) approval

The FDA stopped requiring baseline 510(k) numbers on reports after 2008. Of the 4234 RODs, 361 (8.5%) had a valid 510(k) number recorded. For these devices, the adverse event occurred an average of 1450 days (range, 24–7683 days) after 510(k) approval. For other devices, 291,351 (5.8%) had 510(k) numbers, and the average time of event since 510(k) approval was 2605 days (range, 1–11,261 days). The distribution functions for time since 510(k) approval are shown in Figure 3A. The probability distributions differed significantly (D = 0.47, P<.001).

The average and standard deviation of years since 510(k) clearance for adverse events over time (1994–2008) are shown in Figure 3B. The time since 510(k) clearance for RODs tended to decrease over this time frame (β = −0.52 years/year, P =.09), while it increased for other devices (β = 0.44 years/year, P =.01).

Deaths

There were 98 reports of patient deaths as adverse events for RODs (Table 2). Manual inspection revealed 2 duplicate reports, resulting in 96 unique reports of patient deaths. Most deaths occurred from 2001 to 2005 (45.8%), and most were associated with brachytherapy devices (72.4%). Of external beam reports, 0.7% were for deaths. The proportions of deaths for brachytherapy devices, planning systems, and simulation systems were 6.7%, 1%, and nil, respectively. These proportions differed significantly (P<.001).

A single death report sometimes described multiple deaths associated with an incident. The total number of deaths described within the 96 reports was 103. Of the total deaths, 22 (21.4%) were clearly attributable to a device. External beam device errors were responsible for 11 of these deaths (50%), planning systems for 9 (40.9%), brachytherapy devices for 2 (9.1%), and simulation systems for none. The remainder were due to natural causes or an expected toxicity or complication of treatment. Thirteen deaths (12.6%) were described from reports in the medical literature, none of which were directly attributable to a device.

Of the 96 reports, 18 referenced true device problems, with a total of 28 problems. Most were either user errors (32.1%) or overdose (32.1%).

Discussion

To our knowledge, this is the first comprehensive analysis of adverse events among RODs from the FDA’s MAUDE database. Reports involving RODs comprised about 0.1% of all reports, and the number of reports increased over time, peaking in 2011. This peak may correspond to “increased awareness prompted by targeted interactions with industry” (7), increased media attention (911), and/or widespread adoption of volumetric arc therapy. Most ROD reports involved external beam devices, consistent with these devices being involved in a high percentage of all radiation oncology patient encounters.

Event outcomes differed between RODs and other devices. While most device events involved malfunction, patient injury was less common among RODs (16.3% vs 33.7% in other devices). This finding suggests that while ROD adverse events had potential for causing injury, they were caught before this happened. RODs were more often evaluated by the manufacturer, implying that ROD vendors may be more closely involved after device purchase and proactive in responding after a device issue. Manufacturers may not evaluate a device if the problem is already known (24), so it may also be that RODs are more likely to experience unknown problems. Adverse events involving RODs more often led to notifications with fewer recalls than other devices, representing immediate action taken at the time of report submission. The lower incidence of recalls at the time of event report could be an issue of reporting, though the FDA concluded that ROD manufacturers improved recall reporting from 2003 to 2012 (7).

Software-related issues were the most common among ROD problem codes. Other authors have raised concern about the incidence of software problems (25, 26). The FDA found linear accelerations to be the most-recalled medical device from 2003 to 2012, mostly because of software issues (7). Problems with interoperability, user interface, and dose calculation accounted for more than two-thirds of these linear accelerator recalls. Indeed, software design was tied as the top reason for all medical device recalls from 2010 to 2012 (7). In 2010, the FDA wrote to manufacturers of RODs regarding concerns about underdoses, overdoses, and misaligned exposures from therapeutic radiation. These data sparked a public workshop discussing safeguards and controls, as well as changes to premarket testing and premarket review of software (8). More recently, a 2015 ASTRO proceeding revealed software issues in RT are still very much of interest to the FDA (27).

User error was common, accounting for 20.4% of all ROD problems. Human fallibility has been shown to seriously affect the delivery of RT (28), which points to the need for better manufacturer–user training and better design. This issue has not escaped notice by the FDA, whose Human Factors Engineering–Usability Engineering team evaluates how people interact with technology and how user interface design affects these interactions. No ROD, however, made the recent short list of highest-priority devices for human factors review (29). Any premarket design, postmarket regulation, or quality-improvement effort must clearly address user error and human factors.

RODs experienced adverse events sooner after their manufacture compared with other devices, as well as sooner after their market approval. The time to adverse event since 510(k) clearance also tended to decrease during the evaluable period. This finding could reflect the relative age of most RODs in use in the United States compared with other devices; most linear accelerators tend to be replaced every 7 to 9 years, with software upgrades occurring even more often. These data also suggest that there may not be enough premarket testing of RODs prior to device approval. In part, this reflects problems within the current 510(k) system. The Institute of Medicine found the current 510(k) process inadequate and recommended its replacement with “an integrated premarket and postmarket regulatory framework that effectively provides a reasonable assurance of safety and effectiveness throughout the device life cycle” (30). Other criticisms of the process include reliance by the FDA on manufacturer data only, outsourced and inadequate reviews performed by third parties (13), and new devices becoming significantly different than the original equivalent device (“predicate creep”) (31). Our data suggest that the first 2 years from manufacture and first 5 years from 510(k) approval appear to be the most critical in which to focus safety efforts among RODs. Postmarket monitoring and close user–manufacturer training during this time may mitigate many adverse events.

Deaths, especially those directly attributable to device malfunction or misuse, were rare ROD events on par with the low incidence for all devices. Among deaths directly attributable to RODs, user error and overdoses were the most common problems. Because manufacturers are required to report any event they become aware of, including events in the medical literature, these types of reports were common despite their frequent lack of connection to an actual malfunction. Radioactive microspheres were responsible for the high proportion of brachytherapy devices involved in deaths. In nearly all cases, however, we found these were expected side effects or disease progression, unrelated to a specific device malfunction. Their reporting may be explained by one of the manufacturer’s reports: “Hepatorenal failure is listed in the product package insert and is a known side effect…However, this side effect was inadvertently excluded from the clinical trial protocol hence the assessment of this event as an unexpected adverse device effect requiring reporting” (32).

There are several limitations to the MAUDE database. Because MAUDE is a passive surveillance system, it may suffer from underreporting. Data submissions may be incomplete, inaccurate, untimely, or biased. Our recoding of identifiable misclassifications and organization of the data into broader categories should mitigate some of these problems but cannot prevent them entirely. Our 510(k) findings are limited by the low percentage of records containing this information. To our knowledge, there is no systemic bias regarding the availability of these data; however, caution is warranted in extrapolating the findings to all devices. There is also no information about frequency of device use; therefore, we cannot determine the prevalence or incidence of adverse events per device use or patient treatment course. Furthermore, while problem codes “describe device failures or issues related to the device” according to the FDA (33), these are not necessarily root causes and the specific causal factors are often indeterminable. Despite its limitations, we believe the MAUDE database can inform radiation oncology safety practices, and the FDA itself has relied on this information in focusing its efforts (7, 8).

There are many exciting developments to improve device safety on the horizon. The FDA will soon track “emerging signals” that a medical device may pose risks (34), and it is also revamping the MAUDE system (35). The Global Unique Device Identification Database is being phased in and will allow for more accurate reports of adverse events, more effective management of recalls, and reduction of medical errors (36), as well as more consistent availability of linked data such as 510(k) number. However, our data suggest RODs are substantially different from other devices, and a one-size-fits-all approach may not suffice. We agree with consensus reports that the complexity of processes in radiation oncology calls for a discipline-specific system (37). In the absence of a national mandatory reporting database specifically for radiation oncology, the Radiation Oncology Incident Learning System (RO-ILS) of ASTRO and the American Association of Physicists in Medicine has gained considerable traction in its first year (38). The system has already yielded valuable information beyond MAUDE’s capabilities. For example, 34% of 2014 quarter 4 RO-ILS events were near misses caught by checklists, time-outs, and patient vigilance, and radiation therapists discovered a majority of events (38). These data provide specific and actionable information within the broader scope of issues raised using MAUDE.

Conclusions

Compared with other devices, RODs experience adverse events sooner after manufacture and market approval. RODs are more likely to be evaluated by the manufacturer after an event but less likely to be recalled than other devices. Given the high rate of issues involving human interface among RODs—software problems, user errors, and notifications as remedial action—it is critical to involve professionals at every stage of RT delivery in quality improvement and reporting. We must ensure that rapid innovation in radiation oncology is applied not only to our devices and our technology but also to our systems for ensuring quality and safety. Information from centralized, mandated reporting databases such as MAUDE should inform ongoing efforts to improve patient safety.

Summary

Three decades of US Food and Drug Administration data show that adverse events involving radiation oncology devices differ significantly from other devices in adverse event type, remedial action, problem code, device age, time from 510(k) market clearance to event, and whether the device was evaluated by the manufacturer. The first 2 years from manufacture and first 5 years from 510(k) approval appear to be the most critical in which to focus safety efforts among radiation oncology devices.

Supplementary Material

supplement

Acknowledgments

This work was partially supported by the following grants: National Institutes of Health grants KL2TR00099 and UL1TR000100 (J.A.H.-G.), American Cancer Society Pilot Award ACS-IRG 70-002 (J.A.H.-G.), and National Cancer Institute cancer center specialized grant P30CA023100 (J.A.H.-G.).

Footnotes

Conflict of interest: Drs Hattangadi-Gluth, Cervino, and Moore hold research funding from Varian Medical Systems, unrelated to the current study.

Supplementary material for this article can be found at www.redjournal.org.

References

1. Purdy J, Biggs P, Bowers C, et al. Medical accelerator safety considerations: Report of AAPM Radiation Therapy Committee Task Group No. 35. Med Phys. 1993;20:1261–1275. [PubMed]
2. Kutcher GJ, Coia L, Gillin M, et al. Comprehensive QA for radiation oncology: Report of AAPM Radiation Therapy Committee Task Group 40. Med Phys. 1994;21:581–618. [PubMed]
3. Nath R, Biggs PJ, Bova FJ, et al. AAPM code of practice for radiotherapy accelerators: Report of AAPM Radiation Therapy Task Group No. 45. Med Phys. 1994;21:1093–1121. [PubMed]
4. Das IJ, Cheng CW, Watts RJ, et al. Accelerator beam data commissioning equipment and procedures: Report of the TG-106 of the Therapy Physics Committee of the AAPM. Med Phys. 2008;35:4186–4215. [PubMed]
5. Klein EE, Hanley J, Bayouth J, et al. Task Group 142 report: Quality assurance of medical accelerators. Med Phys. 2009;36:4197–4212. [PubMed]
6. American Society for Radiation Oncology. Safety is No Accident: A Framework for Quality Radiation Oncology and Care. Arlington, VA: American Society for Radiation Oncology; 2012.
7. US Food and Drug Administration, Center for Devices and Radiological Health. Medical device recall report FY2003 to FY2012. Available at: http://www.fda.gov/downloads/aboutfda/centersoffices/officeofmedicalproductsandtobacco/cdrh/cdrhtransparency/ucm388442.pdf. Accessed January 23, 2016.
8. US Food and Drug Administration, Center for Devices and Radiological Health. Letter to manufacturers of linear accelerator radiation therapy treatment planning system ancillary devices. Available at: http://www.fda.gov/Radiation-EmittingProducts/NewsEventsRadiationEmittingProducts/ucm207835.htm. Accessed January 23, 2016.
9. Bogdanich W. While Technology Surges, Radiation Safeguards Lag. New York Times; Jan 27, 2010. p. A1.
10. Bogdanich W, Rebelo K. A pinpoint beam strays invisibly, harming instead of healing. New York Times; Dec 29, 2010. p. A1.
11. Bogdanich W. Radiation offers new cures, and ways to do harm. New York Times; Jan 23, 2010. p. A1.
12. Food, Drug, and Cosmetic Act, 21 USC §360.
13. Hattangadi JA, O’Reilly JT, Recht A. US Food and Drug Administration regulation of medical devices and radiation oncology: Can reform improve safety? J Oncol Pract. 2012;8:53–56. [PMC free article] [PubMed]
14. Everett KD, Conway C, Desany GJ, et al. Structural mechanics predictions relating to clinical coronary stent fracture in a 5 year period in FDA MAUDE database. Ann Biomed Eng. 2016;44:391–403. [PMC free article] [PubMed]
15. Magrabi F, Ong M, Runciman W, et al. Patient safety problems associated with heathcare information technology: An analysis of adverse events reported to the US Food and Drug Administration. AMIA Annu Symp Proc. 2011;2011:853–857. [PMC free article] [PubMed]
16. Myers RB, Jones SL, Sittig DF. Review of reported clinical information system adverse events in US Food and Drug Administration databases. Appl Clin Inform. 2011;2:63–74. [PMC free article] [PubMed]
17. Ash JS, Berg M, Coiera E. Some unintended consequences of information technology in health care: The nature of patient care information system-related errors. J Am Med Inform Assoc. 2004;11:104–112. [PMC free article] [PubMed]
18. US Food and Drug Administration, Center for Devices and Radiological Health. Manufacturer and User Facility Device Experience Database - (MAUDE) Available at: http://www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/PostmarketRequirements/ReportingAdverseEvents/ucm127891.htm. Accessed January 23, 2016.
19. US Food and Drug Administration, Center for Devices and Radiological Health. Downloadable 510(k) files. Available at: http://www.fda.gov/MedicalDevices/ProductsandMedicalProcedures/DeviceApprovalsandClearances/510kClearances/ucm089428.htm. Accessed January 23, 2016.
20. R Core Team. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Core Team; 2015. Available at: https://www.r-project.org. Accessed August 1, 2015.
21. Dowle M, Srinivasan A, Short T, et al. data.table: Extension of Data.frame. R package version 1.9.6. 2015 Available at: http://CRAN.R-project.org/package=data.table. Accessed August 1, 2015.
22. 21 CFR §892, Subpart F.
23. Lilliefors HW. On the Kolmogorov-Smirnov test for normality with mean and variance unknown. J Am Stat Assoc. 1967;62:399–402.
24. US Food and Drug Administration, Center for Devices and Radiological Health. Guidance documents (medical devices and radiation-emitting products) - MEDWATCH medical device reporting code instructions. Available at: http://www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/ucm106737.htm. Accessed January 23, 2016.
25. Pecoraro F, Luzi D. Detecting software failures in the MAUDE database: A preliminary analysis. Stud Health Technol Inform. 2013;192:1098. [PubMed]
26. Simone LK. Software-related recalls: An analysis of records. Biomed Instrum Technol. 2013;47:514–522. [PubMed]
27. Pishko G, Oliveri CM, Zhang J, et al. A three-layered approach to radiation therapy software recall analysis. Int J Radiat Oncol Biol Phys. 2015;93:E498–E499.
28. Leunens G, Verstraete J, Van den Bogaert W, et al. Human errors in data transfer during the preparation and delivery of radiation treatment affecting the final result: “Garbage in, garbage out” Radiother Oncol. 1992;23:217–222. [PubMed]
29. US Food Drug Administration. List of Highest Priority Devices for Human Factors Review. Silver Spring, MD: US Food and Drug Administration; 2016.
30. Committee on the Public Health Effectiveness of the FDA 510(k) Clearance Process. Medical Devices and the Public’s Health—The FDA 510(k) Clearance Process at 35 Years. Washington, DC: Institute of Medicine; 2011.
31. Hines JZ, Lurie P, Yu E, et al. Left to their own devices: Breakdowns in United States medical device premarket review. PLoS Med. 2010;7:1–8. [PMC free article] [PubMed]
32. US Food and Drug Administration, Center for Devices and Radiological Health. MAUDE adverse event report: MDS NORDION Therasphere yttrium-90 glass microspheres. Available at: https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfmaude/detail.cfm?mdrfoi__idZ1275543. Accessed January 23, 2016.
33. US Food and Drug Administration, Center for Devices and Radiological Health. Event Problem Codes - Coding Concepts. http://www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/PostmarketRequirements/ReportingAdverseEvents/EventProblemCodes/ucm134747.htm. Accessed January 23, 2016.
34. US Food Drug Administration. Public Notification of Emerging Postmarket Medical Device Signals (“Emerging Signals”) Silver Spring, MD: US Food and Drug Administration; 2015.
35. November Research Group. FDA launches November Research Group’s PRIMO software platform as next generation product vigilance solution. Available at: http://www.prweb.com/releases/2015/09/prweb12963410.htm. Accessed January 23, 2016.
36. US Food and Drug Administration, Center for Devices and Radiological Health. Unique device identification - UDI. Available at: http://www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/UniqueDeviceIdentification. Accessed January 23, 2016.
37. Ford EC, Fong de Los Santos L, Pawlicki T, et al. Consensus recommendations for incident learning database structures in radiation oncology. Med Phys. 2012;39:7272–7290. [PubMed]
38. American Society for Radiation Oncology, American Association of Physicists in Medicine. RO-ILS year in review 2015. Available at: https://www.astro.org/uploadedFiles/Main_Site/Clinical_Practice/Patient_Safety/Radiation_Oncology_Incident_Learning_System/ROILSYIR2015.pdf. Accessed January 23, 2016.