OBJECTIVE: To describe the rationale, development, content, and results of the AHSR-sponsored conference on developing a research agenda focused on improving the quality of care for children. DATA SOURCES AND METHODS: Planning documents, background papers, and conference proceedings. PRINCIPAL FINDINGS: The conference developed the research agenda focused on (1) monitoring the health of children; (2) evaluating the efficacy and effectiveness of health services for children; (3) assessing the quality of healthcare provided to children; (4) improving the quality of healthcare within health systems; (5) assessing the performance of community systems for children; (6) exploring the impact of different financial incentives on the provision of pediatric healthcare; and (7) developing and disseminating clinical practice guidelines and other information to physicians, families, and consumers. Specific issues and research questions in each area are also presented. Strategies for implementing the research agenda are presented and include: (1) expanding the child health services research workforce; (2) developing child healthcare quality improvement research centers; (3) conducting research in specific high-priority areas; (4) focusing research on improving the health of vulnerable populations; (5) improving child health data and collection systems at the national level; (6) developing better community health monitoring for children; (7) building and supporting research networks and a consortium of research users; and (8) developing a coordinated interagency federal effort to advance this agenda and to provide accountability for its completion. CONCLUSION: The proposed research agenda should be a national priority so that all Americans can be assured that children are receiving the best quality of care that the United States can provide.
The measurement of healthcare provider performance is becoming more widespread. Physicians have been guarded about performance measurement, in part because the methodology for comparative measurement of care quality is underdeveloped. Comprehensive quality improvement will require comprehensive measurement, implying the aggregation of multiple quality metrics into composite indicators.
To present a conceptual framework to develop comprehensive, robust, and transparent composite indicators of pediatric care quality, and to highlight aspects specific to quality measurement in children.
We reviewed the scientific literature on composite indicator development, health systems, and quality measurement in the pediatric healthcare setting. Frameworks were selected for explicitness and applicability to a hospital-based measurement system.
We synthesized various frameworks into a comprehensive model for the development of composite indicators of quality of care. Among its key premises, the model proposes identifying structural, process, and outcome metrics for each of the Institute of Medicine's six domains of quality (safety, effectiveness, efficiency, patient-centeredness, timeliness, and equity) and presents a step-by-step framework for embedding the quality of care measurement model into composite indicator development.
The framework presented offers researchers an explicit path to composite indicator development. Without a scientifically robust and comprehensive approach to measurement of the quality of healthcare, performance measurement will ultimately fail to achieve its quality improvement goals.
To review recent health policies related to measuring child health care quality, the selection processes of national child health quality measures, the nationally recommended quality measures for child mental health care and their evidence strength, the progress made toward developing new measures, and early lessons learned from these national efforts.
Methods used included description of the selection process of child health care quality measures from 2 independent national initiatives, the recommended quality measures for child mental health care, and the strength of scientific evidence supporting them.
Of the child health quality measures recommended or endorsed during these national initiatives, only 9 unique measures were related to child mental health.
The development of new child mental health quality measures poses methodologic challenges that will require a paradigm shift to align research with its accelerated pace.
ADHD; child mental health; clinical validity; depression; quality improvement research; quality measures
Healthcare quality improvement has the potential to reduce the striking disparities in health outcomes among patients with systemic lupus erythematosus (SLE). Donabedian’s framework for assessment of healthcare quality, which divides factors impacting quality into structures, processes and outcomes, provides a theoretical framework for research and interventions in quality improvement. This review applies Donabedian’s model to current research describing quality of care in SLE, highlighting structures and processes that may lead to improved outcomes. Work remains to be done to develop meaningful metrics to assess quality and to understand the structures and processes that improve outcomes. Quality indicators have emerged as an important tool to measure quality, but further validation is required to define their validity and feasibility in clinical practice, as well as their association with improved outcomes. Implementation science also shows promise as a means to create meaningful systematic improvements in healthcare quality for patients with SLE.
access to care; healthcare quality; health disparities; health outcomes; systemic lupus erythematosus
OBJECTIVES--To develop structure, process, and outcome indicators within a quality rating index for audit of public health medicine. DESIGN--Development of an audit matrix and indicator of quality through a series of group discussions with public health physicians, from which self administered weighted questionnaires were constructed by a modified Delphi technique. SETTING--Five Scottish health boards. SUBJECTS--Public health physicians in the five health boards. MAIN MEASURES--Indicators of quality and a quality rating index for seven selected service categories for each of seven agreed roles of public health medicine: assessment of health and health care needs in information services, input into managerial decision making in health promotion, fostering multisectoral collaboration in environmental health services, health service research and evaluation for child services, lead responsibility for the development and/or running of screening services, and public health medicine training and staff development in communicable disease. RESULTS--Indicators in the form of questionnaires were developed for each topic. Three types of indicator emerged: "global," "restricted," and "specific." A quality rating index for each topic was developed on the basis of the questionnaire scores. Piloting of indicators showed that they are potentially generalisable; evaluation of the system is under way across all health boards in Scotland. CONCLUSION--Measurable indicators of quality for public health medicine can be developed.
In September 2000, the Agency for Healthcare Quality and Research and the American Academy of Pediatrics Center for Child Health Research sponsored a meeting of experts and knowledgeable stakeholders to identify 1) the special information needs of pediatric care and 2) health service research questions related to the use of information technology in children's health care. Technologies that support the care of children must address issues related to growth and development, children's changing physiology, and the unique diseases of children and interventions of pediatric care. Connectivity and data integration are particular concerns for child health care workers. Consumer health information needs for this population extend beyond the needs of one individual to the needs of the family. Recommendations of the attendees include rapid implementation of features in electronic health information systems that support pediatric care and involvement of child health experts in policy making, standards setting, education, and advocacy. A proposed research agenda should address both effectiveness and costs of information technology, with special consideration for the needs of children, the development and evaluation of clinical decision support in pediatric settings, understanding of the epidemiology of iatrogenic injury in childhood, supplementation of vocabulary standards with pediatrics-specific terminology, and improvement in health care access for children, using telemedicine.
Large employers formed the National Committee for Quality Assurance (NCQA) to ensure value to healthcare purchasers. Value in healthcare is a function of quality divided by costs. Through NCQA's role as an accrediting agency for healthcare organizations and the development of performance measures, Health Plan Employer Data and Information Set (HEDIS), gains toward defining the value of health services have materialized. An analysis of the impact of HEDIS data collection on physician practices and the influence of HEDIS data on employer, employee, and governmental health plan selections is examined. This study consisted of a general review, from 1993 to 2001, of HealthStar databases, PubMed databases, and the NCQA website. NCQA accreditation is accepted as an important industry milestone for health plans, credentials verification organizations, and physician organizations. The data for HEDIS is collected from health plan administrative data repositories, whereas health plan members' clinical data may be collected by chart abstraction in physician offices. Data collection in physician offices consumes administrative resources from physician practices and health plans. As commercial and governmental insurers move toward greater adoption of HEDIS measures, complex implications are created for physician practices and vulnerable populations. There are lingering questions regarding the improvements in quality of care for medically underserved populations and physician practice costs attributable to HEDIS.
The current healthcare quality improvement infrastructure is a product of a century long experience of cumulative efforts. It began with an acknowledgement of the role of quality in healthcare, and gradually evolved to encompass the prioritization of quality improvement and the development of systems to monitor, quantify, and incentivize quality improvement in healthcare. We review the origins and the evolution of the US healthcare quality movement, identify existing initiatives specific to musculoskeletal care, outline significant challenges and opportunities, and propose recommendations for the future. Elements noted to be associated with successful healthcare quality improvement efforts include the presence of physician leadership, infrastructural support, and prioritization of healthcare quality within the culture of the organization. Issues that will require continued work include the development of a valid and reliable evidence base, accurate and replicable performance measurement and data collection methods, and development of a standard set of specialty specific performance metrics, with accurate provider attribution, risk adjustment and reporting mechanisms.
Quality; Healthcare; Musculoskeletal; Orthopedics; History; Improvement; Performance measurement; Standards; Reporting; Reimbursement; Cost; Value; Public; Private; Government; Evidence-base; Structure; Process; Outcome; Monitoring and evaluation
Awareness of the importance of strengthening investments in child health and monitoring the quality of services in the pediatric field is increasing. The Pediatric Quality Indicators developed by the US Agency for Healthcare Research and Quality (AHRQ), use hospital administrative data to identify admissions that could be avoided through high-quality outpatient care. Building on this approach, the purpose of this study is to perform an empirical examination of the ‘pediatric gastroenteritis admission rate’ indicator in Italy, under the assumption that lower admission rates are associated with better management at the primary care level and with overall better quality of care for children.
Following the AHRQ process for evaluating quality indicators, we examined age exclusion/inclusion criteria, selection of diagnostic codes, hospitalization type, and methodological issues for the ‘pediatric gastroenteritis admission rate’. The regional variability of hospitalizations was analyzed for Italian children aged 0–17 years discharged between January 1, 2009 and December 31, 2011. We considered hospitalizations for the following diagnoses: non-bacterial gastroenteritis, bacterial gastroenteritis and dehydration (along with a secondary diagnosis of gastroenteritis). The data source was the hospital discharge records database. All rates were stratified by age.
In the study period, there were 61,130 pediatric hospitalizations for non-bacterial gastroenteritis, 5,940 for bacterial gastroenteritis, and 38,820 for dehydration. In <1-year group, the relative risk of hospitalization for non-bacterial gastroenteritis was 24 times higher than in adolescents, then it dropped to 14.5 in 1- to 4-year-olds and to 3.2 in 5- to 9-year-olds. At the national level, the percentage of admissions for bacterial gastroenteritis was small compared with non-bacterial, while including admissions for dehydration revealed a significant variability in diagnostic coding among regions that affected the regional performance of the indicator.
For broadest application, we propose a ‘pediatric gastroenteritis admission rate’ that consists of including bacterial gastroenteritis and dehydration diagnoses in the numerator, as well as infants aged <3 months. We also suggest adjusting for age and including day hospital admissions. Future evaluation by a clinical panel at the national level might be helpful to determine appropriate application for such measures, and make recommendations to policy makers.
Health services research; Quality of care; Quality indicators; Pediatrics; Gastroenteritis
Quality and essential health information is considered one of the most cost-effective interventions to improve health for a developing country. Healthcare portals have revolutionalized access to health information and knowledge using the Internet and related technologies, but their usage is far from satisfactory in India. This article describes a health portal developed in India aimed at providing one-stop access to efficiently search, organize and share maternal child health information relevant from public health perspective in the country.
The portal ‘Repository on Maternal Child Health’ was developed using an open source content management system and standardized processes were followed for collection, selection, categorization and presentation of resource materials. Its usage is evaluated using key performance indicators obtained from Google Analytics, and quality assessed using a standardized checklist of knowledge management. The results are discussed in relation to improving quality and access to health information.
The portal was launched in July 2010 and provides free access to full-text of 900 resource materials categorized under specific topics and themes. During the subsequent 18 months, 52,798 visits were registered from 174 countries across the world, and more than three-fourth visits were from India alone. Nearly 44,000 unique visitors visited the website and spent an average time of 4 minutes 26 seconds. The overall bounce rate was 27.6%. An increase in the number of unique visitors was found to be significantly associated with an increase in the average time on site (p-value 0.01), increase in the web traffic through search engines (p-value 0.00), and decrease in the bounce rate (p-value 0.03). There was a high degree of agreement between the two experts regarding quality assessment carried out under the three domains of knowledge access, knowledge creation and knowledge transfer (Kappa statistic 0.72).
Efficient management of health information is imperative for informed decision making, and digital repositories have now-a-days become the preferred source of information management. The growing popularity of the portal indicates the potential of such initiatives in improving access to quality and essential health information in India. There is a need to develop similar mechanisms for other health domains and interlink them to facilitate access to a variety of health information from a single platform.
Health information; Information management; Access; Health portal; Digital repository; Repository on maternal child health; India
Deficiencies in the quality of health care are major limiting factors to the achievement of the Millennium Development Goals for child and maternal health. Quality of patient care in hospitals is firmly on the agendas of Western countries, but has been slower to gain traction in developing countries, despite evidence that there is substantial scope for improvement, that hospitals have a major role in child survival and that inequities in quality may be as important as inequities in access. There is now substantial global experience of strategies and interventions that improve the quality of care for children in hospitals with limited resources. WHO has developed a toolkit containing adaptable instruments, including a framework for quality improvement, evidence-based clinical guidelines in the form of the Pocketbook of Hospital Care for Children, teaching material, assessment and mortality audit tools. These tools have been field-tested by doctors, nurses and other child health workers in many developing countries. This collective experience was brought together in a global WHO meeting in Bali in 2007. This article describes how many countries are achieving improvements in quality of paediatric care, despite limited resources and other major obstacles, and how the evidence has progressed in recent years from documenting the nature and scope of the problems to describing the effectiveness of innovative interventions. The challenges remain to bring these and other strategies to scale, and to support research into their use, impact and sustainability in different environments.
Children; Quality of health care
OBJECTIVE: To provide a framework for a research agenda-setting conference on quality of care for children. DATA SOURCES/STUDY SETTING: Literature review. CONCLUSIONS: Research on quality of care for children has lagged behind its counterpart for adults. Defining key issues and questions in seven topic areas-the health of children; the efficacy and effectiveness of health services for children; assessing quality of care; improving quality of care within health services delivery systems; assessing and improving quality at the community level; getting financial incentives right; and disseminating information about the results of research investigations-will facilitate the development of an effective research strategy. Ultimately, enhanced information in these areas will lead to improvements in the processes and outcomes of care for children.
As providers and systems move towards meaningful use of electronic health records, the once distant vision of data reuse for automated quality reporting may soon become a reality. To facilitate consistent and reliable reporting and benchmarking beyond the local level, standardization of both electronic health record content and quality measures is needed at the concept level. This degree of standardization requires local and national advancement and coordination. The purpose of this paper is to review national efforts that can be leveraged to guide local information modeling and terminology work to support automated quality reporting. Moreover, efforts at Partners HealthCare to map electronic health record content to inpatient quality metrics, terminology standards and to align local efforts with national initiatives are reported. We found that forty-one percent (41%) of the elements needed to populate the inpatient quality measures are represented within the draft documentation content and an additional 29.5% are represented within other Partners HealthCare (PHS) electronic applications. Recommendations are made to support data reuse based on established national standards and identified gaps. Our work indicates that value exists in individual healthcare systems engaging in local standardization work by adopting established methods and standards where they exist. A process is needed, however, to ensure that local work is shared and available to inform national standards.
quality; electronic health record; meaningful use; terminology standards
The overall goal of our research agenda is to contribute to improved quality of healthcare by identifying factors that foster or inhibit the use of healthcare information by patients to make informed healthcare decisions. We propose to study the natural history of the use of healthcare information by women with breast cancer to support decisions about health care. To do so in this paper we propose a conceptual model developed based on an extensive literature review and critique that describes patients' health information use over the disease course. It will guide our further investigation of the complex relationships among patients' personal circumstances, the progress of their medical treatment, and their satisfaction and empowerment as informed decision-makers. The model will help policy makers and health professionals identify the best means to provide patients with useful information, and help all stakeholders in health care acquire information needed to improve healthcare quality.
The National Program for Quality Indicators in Community Healthcare in Israel (QICH) was developed to provide policy makers and consumers with information on the quality of community healthcare in Israel. In what follows we present the most recent results of the QICH indicator set for 2009 and an examination of changes that have occurred since 2007.
Data for 28 quality indicators were collected from all four health plans in Israel for the years 2007-2009. The QICH indicator set examined six areas of healthcare: asthma, cancer screening, cardiovascular health, child health, diabetes and immunizations for older adults.
Dramatic increases in the documentation of anthropometric measures were observed over the measurement period. Documentation of BMI for adolescents and adults increased by 30 percentage points, reaching rates of 61% and 70%, respectively, in 2009. Modest increases (3%-7%) over time were observed for other primary prevention quality measures including immunizations for older adults, cancer screening, anemia screening for young children, and documentation of cardiovascular risks. Overall, rates of recommended care for chronic diseases (asthma, cardiovascular disease and diabetes) increased over time. Changes in rates of quality care for diabetes were varied over the measurement period.
The overall quality of community healthcare in Israel has improved over the past three years. Future research should focus on the adherence to quality indicators in population subgroups and compare the QICH data with those in other countries. In addition, one of the next steps in assessing and further improving healthcare quality in Israel is to relate these process and performance indicators to health outcomes.
Community healthcare; indicators; Israel; National Health Insurance law; quality; QICH
The Ethical Force Program is a collaborative effort to create performance measures for ethics in health care. This report lays out areas of consensus that may be amenable to performance measurement on protecting the privacy, confidentiality and security of identifiable health information.
Iterative consensus development process.
The program's oversight body and its expert panel on privacy include national leaders representing the perspectives of physicians, patients, purchasers, health plans, hospitals, and medical ethicists as well as public health, law, and medical informatics experts.
METHODS AND MAIN RESULTS
The oversight body appointed a national Expert Advisory Panel on Privacy and Confidentiality in September 1998. This group compiled and reviewed existing norms, including governmental reports and legal standards, professional association policies, private organization statements and policies, accreditation standards, and ethical opinions. A set of specific and assessable expectations for ethical conduct in this domain was then drafted and refined through 7 meetings over 16 months. In the final 2 iterations, each expectation was graded on a scale of 1 to 10 by each oversight body member on whether it was: (1) important, (2) universally applicable, (3) feasible to measure, and (4) realistic to implement. The expectations that did not score more than 7 (mean) on all 4 scales were reconsidered and retained only if the entire oversight body agreed that they should be used as potential subjects for performance measurement. Consensus was achieved on 34 specific expectations. The expectations fell into 8 content areas, addressing the need for transparency of policies and practices, consent for use and disclosure of identifiable information, limitations on information that can be collected and by whom, individual access to one's own health records, security requirements for storage and transfer of information, provisions to ensure ongoing data quality, limitations on how identifiable information may be used, and provisions for meaningful accountability.
This process established consensus on 34 measurable ethical expectations for the protection of privacy and confidentiality in health care. These expectations should apply to any organization with access to personally identifiable health information, including managed care organizations, physician groups, hospitals, other provider organizations, and purchasers. Performance measurement on these expectations may improve accountability across the health care system.
health policy; health care information
In a PLOS Medicine Review, Jennifer Requejo, Holly Newby and Jennifer Bryce discuss the five-step process that underlies the generation of data for global monitoring of intervention coverage for maternal and child health and describe the problems associated with selecting appropriate coverage indicators for global monitoring.
Global monitoring of intervention coverage is a cornerstone of international efforts to improve reproductive, maternal, newborn, and child health. In this review, we examine the process and implications of selecting a core set of coverage indicators for global monitoring, using as examples the processes used by the Countdown to 2015 for Maternal, Newborn and Child Survival and the Commission on Accountability for Women's and Children's Health. We describe how the generation of data for global monitoring involves five iterative steps: development of standard indicator definitions and measurement approaches to ensure comparability across countries; collection of high-quality data at the country level; compilation of country data at the global level; organization of global databases; and rounds of data quality checking. Regular and rigorous technical review processes that involve high-level decision makers and experts familiar with indicator measurement are needed to maximize uptake and to ensure that indicators used for global monitoring are selected on the basis of available evidence of intervention effectiveness, feasibility of measurement, and data availability as well as programmatic relevance. Experience from recent initiatives illustrates the challenges of striking this balance as well as strategies for reducing the tensions inherent in the indicator selection process. We conclude that more attention and continued investment need to be directed to global monitoring, to support both the process of global database development and the selection of sets of coverage indicators to promote accountability. The stakes are high, because these indicators can drive policy and program development at the country and global level, and ultimately impact the health of women and children and the communities where they live.
Timely and reliable data on causes of death are fundamental for informed decision-making in the health sector as well as public health research. An in-depth understanding of the quality of data from vital statistics (VS) is therefore indispensable for health policymakers and researchers. We propose a summary index to objectively measure the performance of VS systems in generating reliable mortality data and apply it to the comprehensive cause of death database assembled for the Global Burden of Disease (GBD) 2013 Study.
We created a Vital Statistics Performance Index, a composite of six dimensions of VS strength, each assessed by a separate empirical indicator. The six dimensions include: quality of cause of death reporting, quality of age and sex reporting, internal consistency, completeness of death reporting, level of cause-specific detail, and data availability/timeliness. A simulation procedure was developed to combine indicators into a single index. This index was computed for all country-years of VS in the GBD 2013 cause of death database, yielding annual estimates of overall VS system performance for 148 countries or territories.
The six dimensions impacted the accuracy of data to varying extents. VS performance declines more steeply with declining simulated completeness than for any other indicator. The amount of detail in the cause list reported has a concave relationship with overall data accuracy, but is an important driver of observed VS performance. Indicators of cause of death data quality and age/sex reporting have more linear relationships with simulated VS performance, but poor cause of death reporting influences observed VS performance more strongly. VS performance is steadily improving at an average rate of 2.10% per year among the 148 countries that have available data, but only 19.0% of global deaths post-2000 occurred in countries with well-performing VS systems.
Objective and comparable information about the performance of VS systems and the utility of the data that they report will help to focus efforts to strengthen VS systems. Countries and the global health community alike need better intelligence about the accuracy of VS that are widely and often uncritically used in population health research and monitoring.
Mortality; Causes of death; Vital statistics; Civil registration; Vital registration; Data quality; Health information systems
Increasing numbers of Americans are living with multiple chronic conditions (MCCs) and disabilities. Addressing health care needs of persons with MCCs or disabilities presents challenges on many levels. For health services researchers, priorities include (1) considering MCCs and disabilities in comparative effectiveness research (CER) and assessing quality of care; and (2) identifying and evaluating the data needed to conduct CER, performance measure development, and other research to inform health policy and public health decisions concerning persons with MCCs or disabilities. Little information is available to guide CER or treatment choices for persons with MCCs or disabilities, however, because they are typically excluded from clinical trials that produce the scientific evidence base. Furthermore, most research funding flows through public and private agencies oriented around single organ systems or diseases. Likely changes in the data landscape—notably wider dissemination of electronic health records (EHRs) and moving toward updated coding nomenclatures—may increase the information available to monitor health care service delivery and quality for persons with MCCs and disabilities. Generating this information will require new methods to extract and code information about MCCs and functional status from EHRs, especially narrative texts, and incorporating coding nomenclatures that capture critical dimensions of functional status and disability.
Chronic conditions; disability; functional status; comparative effectiveness research; quality measurement; health information technology; coding nomenclatures
Well-working health information systems are considered vital with the quality of health data ranked of highest importance for decision making at patient care and policy levels. In particular, health facilities play an important role, since they are not only the entry point for the national health information system but also use health data (and primarily) for patient care.
A multiple case study was carried out between March and August 2012 at the antenatal care (ANC) clinics of two private and one public Kenyan hospital to describe clinical information systems and assess the quality of information. The following methods were developed and employed in an iterative process: workplace walkthroughs, structured and in-depth interviews with staff members, and a quantitative assessment of data quality (completeness and accurate transmission of clinical information and reports in ANC). Views of staff and management on the quality of employed information systems, data quality, and influencing factors were captured qualitatively.
Staff rated the quality of information higher in the private hospitals employing computers than in the public hospital which relies on paper forms. Several potential threats to data quality were reported. Limitations in data quality were common at all study sites including wrong test results, missing registers, and inconsistencies in reports. Feedback was seldom on content or quality of reports and usage of data beyond individual patient care was low.
We argue that the limited data quality has to be seen in the broader perspective of the information systems in which it is produced and used. The combination of different methods has proven to be useful for this. To improve the effectiveness and capabilities of these systems, combined measures are needed which include technical and organizational aspects (e.g. regular feedback to health workers) and individual skills and motivation.
health information system; clinical information system; data quality; information and communication technology; antenatal care; Kenya
In recent years, pediatric health care has embraced the concept of quality improvement to improve patient outcomes. As quality improvement efforts are implemented, network collaboration (where multiple centers and practices implement standardized programs) is a popular option. In a collaborative network, improvement in the conduct of structural, process and outcome quality measures can lead to improvements in overall health, and benchmarks can be used to assess and compare progress. In this review article, we provided an overview of the quality improvement movement and the role of quality indicators in this movement. We reviewed current quality improvement efforts in pediatric inflammatory bowel disease (IBD), as well as other pediatric chronic illnesses. We discussed the need to standardize the development of quality indicators used in quality improvement networks to assess medical care, and the validation techniques which can be used to ensure that process indicators result in improved outcomes of clinical significance. We aimed to assess current quality improvement efforts in pediatric IBD and other diseases, such as childhood asthma, childhood arthritis, and neonatal health. By doing so, we hope to learn from their successes and failures and to move the field forward for future improvements in the care provided to children with IBD.
Inflammatory bowel disease; Colitis; Ulcerative; Crohn’s disease; Child; Adolescent; Quality of health care; Review
Randomized trials have long been the gold-standard for evaluating clinical practice. There is growing recognition that rigorous studies are similarly needed to assess the effects of policy. However, these studies are rarely conducted. We report on the Quality Improvement Demonstration Study (QIDS), an example of a large randomized policy experiment, introduced and conducted in a scientific manner to evaluate the impact of large-scale governmental policy interventions.
In 1999 the Philippine government proposed sweeping reforms in the National Health Sector Reform Agenda. We recognized the unique opportunity to conduct a social experiment. Our ongoing goal has been to generate results that inform health policy. Early on we concentrated on developing a multi-institutional collaborative effort. The QIDS team then developed hypotheses that specifically evaluated the impact of two policy reforms on both the delivery of care and long-term health status in children. We formed an experimental design by randomizing matched blocks of three communities into one of the two policy interventions plus a control group. Based on the reform agenda, one arm of the experiment provided expanded insurance coverage for children; the other introduced performance-based payments to hospitals and physicians. Data were collected in household, hospital-based patient exit, and facility surveys, as well as clinical vignettes, which were used to assess physician practice. Delivery of services and health status were evaluated at baseline and after the interventions were put in place using difference-in-difference estimation.
We found and addressed numerous challenges conducting this study, namely: formalizing the experimental design using the existing health infrastructure; securing funding to do research coincident with the policy reforms; recognizing biases and designing the study to account for these; putting in place a broad data collection effort to account for unanticipated findings; introducing sustainable policy interventions based on the reform agenda; and providing results in real-time to policy makers through a combination of venues.
QIDS demonstrates that a large, prospective, randomized controlled policy experiment can be successfully implemented at a national level as part of sectoral reform. While we believe policy experiments should be used to generate evidence-based health policy, to do this requires opportunity and trust, strong collaborative relationships, and timing. This study nurtures the growing attitude that translation of scientific findings from the bedside to the community can be done successfully and that we should raise the bar on project evaluation and the policy-making process.
Participants in the Cochrane Collaboration conduct and periodically update systematic reviews that address the question, “What works?” for healthcare interventions. The Cochrane Library makes available quality-appraised systematic reviews that address this question. No coordinated effort has been undertaken to conduct and periodically update systematic reviews that address the other types of questions asked by healthcare managers and policy makers, to adapt existing reviews to highlight decision-relevant information (including the factors that may affect assessments of a review’s local applicability) or to facilitate their retrieval through a “one-stop shopping” portal. Researchers interested in evaluating new methodological developments, health services and policy researchers interested in conducting and adapting systematic reviews, and research funders all have a role to play in making systematic reviews more useful for healthcare managers and policy makers.
In public health research and practice, quality of life is increasingly acknowledged as a valid and appropriate indicator of service need and intervention outcomes. Health-related quality of life measures, including objective and subjective assessments of health, are particularly useful for evaluating efforts in the prevention of disabling chronic diseases. Such data can inform health policy, planning, and practice. Mechanisms for routinely monitoring quality of life of populations at the State and local levels are currently lacking, however. This article discusses the rationale for and concepts measured by four quality of life questions developed for the 1993 Behavioral Risk Factor Surveillance System, a State-based telephone surveillance system. To encourage quality of life surveillance by States, the Centers for Disease Control and Prevention's National Center for Chronic Disease Prevention and Health Promotion held two related workshops, one in December 1991 and the other in June 1992. The workshops convened experts in quality of life and functional status measurement and resulted in the formulation of items for the Behavioral Risk Factor Surveillance System on self-perceived health, recent physical and mental health, and recent limitation in usual activities. The criteria, including feasibility and generalizability, considered by the Centers for Disease Control and Prevention and the workshop participants in the selection and development of these items are discussed. A model that conceptualizes the relationship of quality of life domains measured by the four survey items is presented and validated with preliminary data from the 1993 Behavioral Risk Factor Surveillance System. Finally, how States can use these measures to track progress towards the Year 2000 goal of improving quality of life is discussed.
Although it is now widely endorsed that children should as far as possible rate their own health related quality of life (HRQL), there are situations where proxy information on child HRQL may be useful, especially where a child is too ill or young to provide their own HRQL assessment. There is limited availability of generic HRQL scales that have a parallel child and parent version and that are reliable, valid, brief, comprehensible and suitable for use in UK populations. The aims of this study were therefore to develop and validate a parent version of the anglicised Manchester-Minneapolis Quality of Life child form (MMQL-UK (CF)) and to determine the level of association between the child and parent versions of this form.
This study was undertaken concurrently with the anglicisation and validation of the MMQL, a measure of HRQL developed for use with children in North America. At that time, no parent version existed, so the MMQL form for children (MMQL-UK (CF)) was used as the basis for the development of the MMQL-UK parent form (PF). The sample included a control group of healthy children and their parents and five exemplar groups; children diagnosed with asthma, diabetes or inflammatory bowel disease and their parents, children in remission from cancer and their parents and children in public care and their carers. Consistency of the MMQL-UK (PF) components were assessed by calculating Cronbach's alpha. Validation of the parent questionnaire was undertaken by comparing MMQL-UK (PF) component scores with comparable components on the proxy PedsQL™ quality of life scales, comparing MMQL-UK (PF) component scores between parents of healthy and chronic disease children and by comparison of component scores from children and their parents or carers. Reproducibility and responsiveness were assessed by retesting parents by follow-up questionnaires.
A total of 874 children (completing MMQL-UK (CF)) and 572 parents or carers (completing MMQL-UK (PF)) took part in the study. The internal consistency of all the MMQL-UK (PF) components exceeding the accepted criterion of 0.70 and the construct validity was good with moderate correlations being evident between comparable components of the MMQL-UK (PF) and the proxy PedsQL™. Discriminant validity was demonstrated with significant differences being identified between parents of healthy children and those with chronic conditions. Intra-class correlations exceeded 0.65 for all MMQL-UK (PF) components demonstrating good reproducibility. Weak to moderate levels of responsiveness were demonstrated for all but social functioning. The MMQL-UK (PF) showed moderate parent-child correlation with the MMQL-UK (CF) for all components. The best correlations were seen for those components measuring the same construct (Pearson's r ranged from 0.31 to 0.61, p < 0.01 for equivalent components).
The MMQL-UK (PF) showed moderate to good correlations with the MMQL-UK (CF) component scores. The MMQL-UK (PF) will be of use when comparing child and parent/carer perception of the impact of a child's condition on their HRQL or where the child is too ill or young to provide their own report.