Presently, there are several general impediments and some barriers specific to lung diseases and sleep disorders for the accomplishment of CER. The group identified these barriers, considered how they affect the ability to perform CER, and discussed potential strategies to overcome these obstacles.
Identification of Patients and Therapies for Studies
Many pulmonary disorders and some sleep disorders meet the criterion of a rare disease, defined by the NIH as a disorder that affects fewer than 200,000 people in the United States, which will present a challenge for CER. For example, cystic fibrosis (CF) is a chronic and debilitating disorder that often results in premature morbidity and mortality, but it only affects an estimated 30,000 individuals. In addition, regional variability exists in the prevalence of some pulmonary disorders, including interstitial lung diseases (27
). The low prevalence and geographical heterogeneity of these diseases may hamper the ability to enroll these patients into pragmatic clinical trials.
The development of large national registries of patients with specific pulmonary diseases and sleep disorders might assist in the identification of patients for enrollment into clinical trials. Registries have generally been used for relatively rare diseases, such as CF. The information in the CF registry created more than 40 years ago by the CF Foundation allows caregivers and researchers to identify new health trends, design clinical trials for potential therapies, and identify patients for future trials. Few such registries currently exist for lung diseases and sleep disorders. To facilitate CER, it is necessary to develop more disease-specific registries, including developing methodologies and approaches for capturing key representative data on common complex pulmonary and sleep disorders.
Identification of large patient populations for CER in more common lung disorders can also be difficult, and systematic methods to aid recruitment are important. Clinical trial networks can help in identification of appropriate patients. The DLD has established a number of disease-specific networks that created the necessary infrastructure of clinical and associated support centers to enable the rapid development and conduct of multiple clinical protocols to evaluate the efficacy of promising diagnostic and therapeutic approaches. These clinical networks, which currently study asthma, interstitial lung disease, ARDS, and COPD, may be properly positioned to conduct CER if efforts are made to expand the patient pools and interventions outside of academic settings.
More recently, a large network of academic institutions supported by NIH Clinical and Translational Science Awards have led the development of a Web site for patients who are interested in participating in clinical research. The goal of this program, called ResearchMatch (30
), is to link suitable patients with researchers who are looking for study participants. The NIH registry ClinicalTrials.gov
is another resource to help connect patients with appropriate studies.
Another barrier to the performance of CER is the lack of proven therapies for many pulmonary and sleep diseases. For example, there is a paucity of proven medical therapies for ARDS and interstitial lung diseases. Large pragmatic trials that compare different therapies may not yet be practical for certain lung and sleep diseases. The identification of treatments with proven efficacy with smaller efficacy trials may be necessary before trials of the size and magnitude necessary for CER should be performed.
Administrative data sets, including claims data files kept by health plans and government payers, are a potentially powerful tool for CER (31
). Clinical information obtained during routine clinical care provides a unique opportunity to examine the relative effectiveness of treatments. The information derived from such records are coded data (International Classification of Diseases-9 codes), including diagnostic and procedure codes, dispensing records for medications, and free text entries. Newer technology, such as natural language processing, will be needed to take full advantage of the variety of different types of data available, such as free text that occurs in electronic medical records.
The Agency for Healthcare Research and Quality (AHRQ) has funded several large projects to build and expand the nation's clinical electronic infrastructure to generate prospective, patient-centered outcomes for CER on therapeutics and tests. The infrastructure could be scaled up to include other organizations and data systems, with the goal of sustainable support for CER. The development of new methods and data governance approaches needed to enhance the national infrastructure for CER will be spearheaded by the Electronic Data Methods Forum, a new initiative led for AHRQ by AcademyHealth. Descriptions of these AHRQ-sponsored projects are available on the program Web sites (32
Effectiveness studies that compare different mechanisms of health care delivery may also benefit directly from clinically derived data that can identify populations for study and/or allow assessment of the delivery and effectiveness of care. However, for investigators to make valid inferences from studies using such administrative data, they need to understand the validity of all the data elements to be used, including variables on exposures, outcomes, and covariates.
The quality of administrative data depends on limitations and changes in the specificity of the criteria for diagnoses and on temporal changes related to financial reimbursement and regional coding practices. Internation Classification of Diseases-9 codes require additional investigation to evaluate and improve the reliability and validity of these measures in heterogeneous clinical settings even for common conditions, such as COPD, asthma, sleep apnea, and acute lung injury. Furthermore, laboratory values require a reference standard to be meaningful, because results are known to vary across laboratories and time. Defining common reference standards is important for other elements of the medical record, including medications in which these elements may be described by name or coded according to other common standards, such as National Drug Classification or Systematized Nomenclature of Medicine codes. Harmonization of data records may need to occur not only across administrative data sets but also across electronic medical records (EMRs) from different settings. This will require transforming native data from disparate EMRs into data that have common meaning across information platforms. This issue is being addressed using standardized data dictionaries, such as those developed by the Health Maintenance Organization Research Network (34
), the Mini-Sentinel Initiative sponsored by the Food and Drug Administration (35
), and the COPD DataHub, CONCERT (26
). For example, research in ontologies also may improve data integration efforts. Recent informatics research for sleep disorders research has led to the development of ontologic-driven data-mapping procedures that have provided an efficient approach for integrating data across databases and institutions (R. Mueller and colleagues, unpublished results).
A related concept is that certain CER questions may be best addressed by merging data from existing administrative or clinical databases (including EMRs) with data from prospectively collected research databases. Research databases often better standardize the definitions of data elements than do administrative or clinical databases. However, the formats of such databases may vary greatly and the coding systems may not easily harmonize between databases. Continued work on the development of standardized coding systems and ontological frameworks may facilitate the harmonization of data across many sources.
Although there are a number of questions regarding the validity and reliability of administrative data, another major barrier is access to the data. To understand the comparative effectiveness of care, up-to-date near real-time data are essential and becoming more common. Integrating data from multiple sources, including administrative data sets and EMRs, for CER purposes would significantly improve the ability to compare treatment and diagnostic strategies across multiple systems of health care. Because issues of privacy are determined by local institutional review boards (IRBs), there is significant heterogeneity in the willingness to permit sharing of data. There is currently no national standard for sharing data between health care providers, health care organizations, academic and nonacademic institutions, and federal entities. In research that relies on the sharing and pooling of data for individual use or reuse between such organizations, the process can be exceedingly inefficient and time consuming, with approvals required from multiple individuals within each organization, including IRB officials and security and privacy compliance officers. Despite having previously shared data between organizations, the process often must be replicated for each additional investigation, adding to the inefficiency and delaying research progress. Given the need to be able to perform comparative effectiveness research in an efficient and timely fashion, a national emphasis is needed to reduce this barrier. Emphasis needs to be placed on developing national standards to facilitate the sharing of data by agencies such as Centers for Medicare and Medicaid Services (CMS), NIH, and Department of Veterans Affairs and among academic and nonacademic partners. Such data sharing could involve deidentified or limited data sets in which personal identifiers such as names, birth dates, and dates of medical services are removed or transformed to protect patient privacy. These standards need to emphasize the relative benefit of CER in relationship to risk of disclosure associated with the use of clinically derived administrative data. Innovative strategies include distributed data network approaches wherein identifiable data elements remain under the control of the respective institutions and their disclosure or data sharing policies (36
). With appropriate governance arrangements, permissions for disclosure, access, and use of these data can be controlled by the originating health care system.
Investigator-initiated funding for CER is proportionately and in absolute terms far less than it is for other biomedical research. The need for CER to inform the nation's health care enterprise has motivated the use of targeted contracts, task orders, and other government-specified work. AHRQ has developed centers and networks, including Evidence-Based Practice Centers, Centers for Education and Research on Therapeutics, and the Developing Evidence to Inform Decisions about Effectiveness Network (38
). These types of structures are important. In addition, the workshop participants emphasized the need for investigator-initiated peer-reviewed CER research via NIH mechanisms. Without funding to support innovative, peer-reviewed CER studies, the scientific quality of CER could suffer.
CER will benefit from multidisciplinary teams. These are not common and often do not include expertise in implementation or improvement research. In addition to the contract-based work done at AHRQ CER centers, programs and centers should be funded that assemble multiple disciplines and conduct investigator-initiated studies. Networks of investigators that have been established based on clinical trials, clinical and translational science awards (CTSA), or linkages of electronic data sets will be valuable resources. Approaches should be developed to encourage collaboration from investigators currently outside these networks.
The ultimate impact of CER will depend on its implementation of treatment and strategies in real-world settings by the health care industry. Partnerships with stakeholders are needed, both to conduct research and to translate CER findings into practice. Although AHRQ, NIH, and the new Patient-Centered Outcomes Research Institute presumably will be the major funders of CER, other stakeholders will be needed to provide other resources crucial for CER implementation, including access to health care systems, providers, and patients; electronic data; and medical records. For example, health plans and provider groups might be able to provide access to claims and electronic medical records, pharmaceutical companies might conduct and provide clinical trials using novel designs, and information technology vendors or consultants could provide and develop data collection and interface methods to support CER. In addition, these partners could also provide settings and opportunities for training CER researchers. Potential barriers to these collaborations include economic interests that could be affected by the results and implementation of CER, cultural and political differences among the research community and stakeholders, organizational hurdles including legal concerns about patient privacy, and the lack of channels for communication and collaboration.
The National CTSA Consortium sponsored a CER forum in December 2010 at the NIH so that representatives from federal agencies, industry, foundations, and other stakeholders could discuss such high-impact CER topics as infrastructure, education/training/workforce development, methods development, community and practice engagement, and health information technology. The meeting outcomes included identifying challenges, opportunities, and next steps, including a recommendation that priority should be given to funding studies that have explicit plans to work with other stakeholders in the health care community. More information can be found at http://www.ctsaweb.org/
Engagement of Community Participants
The CER reports by the Institute of Medicine (2
) and Federal Coordinating Council for Comparative Effectiveness Research (5
) emphasized the need for community input when choosing relevant questions and for recruiting sites, patients, and physicians into studies. These efforts also should address disparities in health and health care among communities. Engagement of the community has not been the tradition in academic medical research. This is an important part of CER and is mandated for work done at the AHRQ CER centers. Not only will such engagement enhance the relevance of questions addressed by CER but also it will engage members of the community in the research and will educate the public so it can maximally benefit from the findings of CER. Community engagement is a substantially new approach, there is relatively little experience in this area, and it is time consuming. Although expertise is sparse, it is growing through the efforts of Centers for Disease Control (CDC), NIH-supported community-based participatory research grants, and the Community Engagement Components of the NIH CTSAs. Support for developing the procedures and practices for engaging the public in CER is needed to further leverage these and encourage research in a wide variety of settings.
Needs for Training
The competencies needed by a researcher in CER include some that are distinct. Currently there is a critical need to strengthen a national cadre of investigators with the skills to conduct CER (39
). Research training is a central role of CTSAs and they are involved in CER-specific education and career development. Investigators who conduct CER benefit from training in specific fields, especially epidemiology and biostatistics. More specifically, a recent report of the CTSA Consortium CER Workforce Development workgroup identified special skills and education relevant to CER, including research ethics and logistics related to the complex issues involved in community-engaged research, pragmatic clinical trials, biomedical informatics, electronic health records research, large database research, practice-based network research, and decision analysis/cognitive sciences, health economics/cost-effectiveness, and health services research (39
). CER training and career development should be supported by training grants. Investigators who conduct CER may have Master in Public Health degrees, PhDs, or similar training.
Research institutions as well as investigators may view CER as representing a large paradigm shift. Research-intensive academic centers may be most familiar with traditional efficacy studies and randomized trials, and may not understand or value the role CER has in the spectrum of clinical research. IRBs may be uncomfortable with the flexibility needed in pragmatic clinical trials or with alternative approaches to seeking informed consent from patients in real-world settings. Thus, programs to orient institutions and IRBs to CER could help accelerate support for CER. Furthermore, in multicenter studies, the need for IRB approval at multiple sites may sometimes create long delays. One solution is to arrange for IRBs to be able to cede to one another, an approach used in some projects led by members of the Health Maintenance Organization Research Network.