Medical technology assessment may amplify conflicts of interest related to investments, reimbursement for clinical services, research support, political and lobbying influences, and other factors affecting all stakeholders: manufacturers, stockholders, physicians, patients, insurance carriers, and even the FDA itself. Fortunately, several existing medical technology assessment groups provide complementary critical analyses of the efficacy and safety of new medical devices (Tables , , and ). Among the government-sponsored entities other than the FDA that have contributed to the research and dissemination of technology assessments, the most prominent is the Centers for Medicare and Medicaid Services (CMS), which often initiates coverage of medical devices with an evidence-development process ultimately aimed at synthesizing large amounts of data from various clinical experiences.36
Other groups include the IOM, the National Institutes of Health, the US Preventive Services Task Force, and the Agency for Healthcare Research and Quality (AHRQ), which itself includes a network for Developing Evidence to Inform Decisions about Effectiveness (DeCIDE), Centers for Education and Research on Therapeutics (CERTs), and several evidence-based practice centers (EPCs) funded by the AHRQ that not only review relative effectiveness and safety, but also identify areas requiring further study. A number of foreign federal and private technology assessment agencies (e.g., NICE in the UK) also make important contributions.
Major Evidence-Based Sources of Information on Medical Technology
Selection of Technology Assessment Entities
Selection of Professional Societies
Private Payers and Technology Assessment
Private payers rely on a mix of both proprietary and publicly available technology assessments for their policy development; some have their own proprietary technology assessment process or purchase proprietary technology assessments produced by independent, often for-profit, companies. An example of an independent, nonprofit organization that publishes technology assessments is the Institute of Clinical Systems Improvement (ICSI), with 56 members funded by all 6 of the health plans in Minnesota. The Blue Cross/Blue Shield Association (BCBSA) produces technology assessments through an EPC, the Technology Evaluation Center (TEC). TEC assessments are available to the public at their website and are distributed to all of the Blues plans as a common starting point for medical policy coverage development. Technology assessment organizations, whether developed by specific payers or independent companies, variably engage specialty societies or local physicians for their input. Payers typically convene a committee to consider either privately or publicly the findings of an assessment; consumers, specialists, and practicing physicians may serve as contributors to the assessment itself, as members of an assessment committee, or as majority or minority voting members on a technology assessment. There is some concern that private payers are vulnerable to financial pressures. A pointed summary from the 2004 Consensus Conference of the American Society for Bariatric Surgery (ASBS) suggested that insurance providers are “unwilling to invest in expensive treatments that yield no profits until after 4 years because 4 years is the median time for an individual to remain with an insurance company.”37
Whether such views are substantiated, it remains important to acknowledge the potential for vested interests.
Professional societies oversee the utilization of new technologies that fall under their domain, and they frequently provide checks and balances on other decision-making entities. Their Web sites can serve as a forum for hotly debated, complex topics that affect patient safety (Table ).38,39
Indeed, professional societies have often taken the lead in endorsing evidence-based reviews. For example, the American College of Physicians (ACP) announced its collaboration with the American Academy of Family Physicians (AAFP), the American Academy of Pediatrics (AAP), and the American Osteopathic Association (AOA) to release joint principles for a “Patient-Centered Medical Home,” a program in which evidence-based medicine serves clinical decision making.40
The American Thoracic Society (ATS) has been a strong advocate of applying and refining standard criteria for conducting systematic reviews with the aim of developing practice guidelines.41,42
Professional societies not only provide a format for discussion, they also work toward improving the overall quality of debate. However, as with other evaluative entities, they are not immune to pressures and financial conflicts of interest that may influence their recommendations.43,44
Two examples of professional society recommendation serve to highlight the potential for member self-interest to influence the evaluation of novel medical devices. In May of 2006, an article in the lay press questioned patient testimonials on a manufacturer’s Web site extolling a new procedure, balloon dilation of the sinuses, a minimally invasive alternative to sinus surgery. This challenge was officially echoed in a position statement by the American Rhinologic Society (ARS), claiming that in spite of FDA approval, “... the scientific literature has no data on the technology for long-term safety, indications, efficacy and outcomes.”45–47
What the ARS did not say is that some of their members might experience a decline in income if the balloon dilation procedure were established as the standard of care over surgery. Likewise, intensity-modulated radiation therapy (IMRT), an alternative to conventional 3D-conformal radiation therapy (3D-CRT), has been broadly embraced by the radiation oncology professional organizations for treatment of prostate cancer despite the lack of any comparative studies in humans and no FDA review process of specific applications of IMRT for prostate cancer. The embrace of IMRT may have been in part driven by its favorable reimbursement compared to that of standard therapies for prostate cancer.48
The safety and effectiveness of medical devices are not yet comprehensively evaluated by the FDA, other federal entities, payers, or professional societies. Therefore, a stance of critical self-analysis will remain crucial while advocating innovation. Technology assessments by objective, independent, not-for-profit entities are necessary so that deficiencies in the clinical evidence and potential threats to patient safety are recognized in a manner that counterbalances market forces driving diffusion and demand.
A Case Study: The California Technology Assessment Forum
The California Technology Assessment Forum (CTAF) is an independent, nonprofit entity whose mission is to provide transparent, objective reviews of new medical devices to better educate the public, clinicians, and policymakers (see http://www.ctaf.org
). The CTAF reviews are intended to complement the role played by professional societies and the FDA. Recent topics have included full-field digital mammography for breast cancer screening, gene expression profiling for the diagnosis of heart transplant rejection, wireless capsule endoscopy, and device-controlled breathing as a treatment for hypertension.
CTAF contracts with general internists at academic medical centers to provide about 15 systematic reviews of new and emerging medical technologies per year. Topics are selected through a consensus process based on impact of the technology and availability of relevant clinical data by an ad hoc committee consisting of representatives from the CTAF panel (see below), the generalist physician consultants and the Blue Shield of CA health plan. Potential topics may be brought to the attention of the committee by all potential stakeholders: the health plan, industry, professional societies, consumer groups, or CTAF staff. Although the subject of these reviews is often highly specialized and technical, CTAF believes that generalist physicians are better suited to provide objective, evidence-based assessments than are subspecialists who, in spite of their superior technical knowledge, may be more likely to have preexisting biases or opinions. Three times per year, CTAF invites manufacturers, patients, clinicians, and payers to openly debate the safety and effectiveness of 4 to 5 new devices; these discussions typically include input ranging from anecdotal patient experience to expert testimony from the principal investigators of the pivotal randomized controlled trials.
CTAF reviews are systematic analyses of the peer-reviewed literature that synthesize the results of all published clinical trials, case series, and randomized clinical trials, with an emphasis on comparative trials. The MEDLINE database, Cochrane clinical trials database, Cochrane reviews database, and the Database of Abstracts of Reviews of Effects (DARE) are searched using relevant key words from 1966 up to the present. In addition, the bibliographies of systematic reviews and key articles are manually searched for additional references. Abstracts of citations are reviewed and all relevant articles reviewed in full. Each assessment is researched and written by the lead consultant with consultation and input from the other consultants as needed and are, generally, 25,000 to 30,000 words in length.
The consultant must determine whether each new technology selected for review meets 5 criteria (see Table ) according to a consistent, reproducible approach with a visible line of reasoning. The technology assessment should specify critical patient-centered outcomes, as well as identify which data are available that will ideally evaluate these outcomes, and where there is insufficient data. CTAF reviews are sent to subspecialty experts and to professional societies for critical appraisal and comment. Members from both of these groups are invited to the CTAF meetings to contribute to the discussion in a public forum. Other interested parties from the government (e.g., CMS), health care organizations, academic centers, and the community are also invited to attend and participate in the discussion. Efforts are also made to invite those who are advocates of a particular device (such as manufacturers and inventors), as well as others who may be against it. The open meeting is a unique feature of the CTAF assessment protocol.
California Technology Assessment Forum Assessment Criteria
The voting panel includes experts from a variety of disciplines: ethicists, consumer advocates, practicing clinicians, methodologists, and others. The consultants present a brief overview of the peer-reviewed literature to the panel and then recommend that the technology either meets or does not meet the 5 criteria for approval. If the recommendation cannot be accepted as stated, the panel then entertains and votes on alternatives. Decisions are made by a simple majority vote; consensus is not required. Approximately 25% of new technologies are approved (i.e., found to meet all 5 CTAF criteria for efficacy and safety). There is no formal appeal process, but technologies will be assessed again when more data become available and frequently are approved at that time. CTAF does not have the ability to formally track the costs or impact of new technology it has reviewed.