NICU safety culture, as measured by the Safety Attitudes Questionnaire (SAQ), varies widely. Associations with clinical outcomes in the adult ICU setting make the SAQ an attractive tool for comparing clinical performance between hospitals. Little information is available on the use of the SAQ for this purpose in the NICU setting.
To determine whether the dimensions of safety culture measured by the SAQ give consistent results when used as a NICU performance measure.
Cross-sectional survey of caregivers in twelve NICUs, using the six scales of the SAQ: teamwork climate, safety climate, job satisfaction, stress recognition, perceptions of management, and working conditions. NICUs were ranked by quantifying their contribution to overall risk-adjusted variation across the scales. Spearman Rank Correlation coefficients were used to test for consistency in scale performance. We then examined whether performance in the top four NICUs in one scale predicted top four performance in others.
There were 547 respondents in twelve NICUs. Of fifteen NICU-level correlations in performance ranking, two were greater than 0.7, seven were between 0.4 and 0.69, the six remaining were less than 0.4. We found a trend towards significance in comparing the distribution of performance in the top four NICUs across domains with a binomial distribution p = .051, indicating generally consistent performance across dimensions of safety culture.
A culture of safety permeates many aspects of patient care and organizational functioning. The SAQ may be a useful tool for comparative performance assessments among NICUs.
Infant; newborn; quality of care; benchmarking; safety culture
To assess the level of agreement when selecting quality measures for inclusion in a composite index of neonatal intensive care quality (Baby-MONITOR) between two panels: one comprised of academic researchers (Delphi) and another comprised of academic and clinical neonatologists (Clinician).
In a modified Delphi process, a panel rated twenty eight quality measures. We assessed clinician agreement with the Delphi panel by surveying a sample of forty eight neonatal intensive care practitioners. We asked the clinician group to indicate their level of agreement with the Delphi panel for each measure using a five- point scale (much too high, slightly too high, reasonable, slightly too low, and much too low). In addition, we asked clinicians to select measures for inclusion in the Baby-MONITOR based on a yes or no vote and a pre-specified two-thirds majority for inclusion.
Twenty three (47.9%) of the clinicians responded to the survey. We found high levels of agreement between the Delphi and clinician panels, particularly across measures selected for the Baby-MONITOR. Clinicians selected the same nine measures for inclusion in the composite as the Delphi panel. For these nine measures, 74% of clinicians indicated that the Delphi panel rating was ‘reasonable’.
Practicing clinicians agree with an expert panel on the measures that should be included in the Baby-MONITOR, enhancing face validity.
infant; newborn; quality of health care; measurement; composite indicator
Psychometric scales often change over time, complicating comparison of scores across different versions. The Impact of Cancer (IOC) scale was developed to measure quality of life of long-term cancer survivors. We sought to develop a method for scoring the earlier version, IOCv1, to obtain scores comparable to IOCv2, which is the recommended version.
Data from 1828 cancer survivors who had completed a questionnaire including all IOCv1 and IOCv2 items were randomly split into training, validation and test sets. The training and validation sets were used to develop and validate linear regression models for predicting each IOCv2 item missing from IOCv1. The models were then applied to the test set to obtain pseudo-IOCv2 scores, which were compared to observed scores to assess predictive performance of the models in independent data.
Observed and pseudo-IOCv2 scale scores were highly correlated in the test sample and had mean differences near zero. The models performed especially well in predicting summary scale scores, with correlations exceeding 0.98.
The approach facilitates comparison across samples of survivors surveyed using different versions of the IOC and may be useful to other investigators trying to compare participants surveyed using different versions of the same instrument.
impact of cancer; oncology; psychometric scales; quality of life; cancer survivorship
To measure the influence of varying mortality time frames on performance rankings among regional NICUs in a large state.
We carried out cross-sectional data analysis of VLBW infants cared for at 24 level 3 NICUs. We tested the effect of four definitions of mortality: (1) deaths between admission and end of birth hospitalization or up to 366 days; (2) deaths between 12 hours of age and end of birth hospitalization or up to 366 days; (3) deaths between admission and 28 days; and (4) deaths between 12 hours of age and 28 days. NICUs were ranked by quantifying their deviation from risk-adjusted expected mortality and splitting them into three tiers: top six, bottom six, and in-between.
There was wide inter-institutional variation in risk-adjusted mortality for each definition (observed minus expected Z-score range, -6.08 to 3.75). However, mortality-rate-based NICU rankings, and their classification into performance tiers, were very similar for all institutions for each of our time frames. Among all four definitions, NICU rank correlations were high (>0.91). Few NICUs changed relative to a neighboring tier with changes in definitions, and none changed by more than one tier.
The time frame used to ascertain mortality had little effect on comparative NICU performance.
Infant; newborn; quality of care; performance measurement; mortality
Variation in health care delivery and outcomes in NICUs may be partly explained by differences in safety culture.
To describe NICU caregiver assessments of safety culture, explore the variability within and between NICUs on safety culture domains, and test for association with caregiver characteristics.
We surveyed NICU caregivers in a convenience sample of 12 hospitals from a single health care system, using the Safety Attitudes Questionnaire (SAQ). The six scales of the SAQ include teamwork climate, safety climate, job satisfaction, stress recognition, perception of management, and working conditions. For each NICU we calculated scale means, standard deviations and percent positives (percent agreement).
We found substantial variation in safety culture domains among participating NICUs. A composite mean score across the six safety culture domains ranged from 56.3 to 77.8 on a 100-point scale and NICUs in the top four NICUs were significantly different from the bottom four (p < .001). Across the six domains, respondent assessments varied widely, but were least positive on perceptions of management (3–80% positive; mean 33.3%) and stress recognition (18–61% positive; mean 41.3%). Comparisons of SAQ scale scores between NICUs and a previously published cohort of adult ICUs generally revealed higher scores for NICUs. Physicians composite scores were 8.2 (p = .04) and 9.5 (p =.02) points higher than nurses and ancillary personnel.
Significant variation and scope for improvement in safety culture exists among this sample of NICUs. The NICU variation was similar to variation in adult ICUs, but NICU scores were generally higher than adult ICU scores. Future studies should validate whether safety culture as measured with the SAQ correlates with clinical and operational outcomes in the NICU setting.
Infant; newborn; quality of care; patient safety; safety culture
To develop and evaluate a new instrument that measures aspects of long-term survivorship not measured by existing tools.
In qualitative interviews, 47 long-term cancer survivors (LTS) detailed ways that cancer has impacted their lives. Content analysis resulted in the creation of 325 candidate items for inclusion in a new Impact of Cancer (IOC) instrument. Following expert review, item reduction and pilot testing, 81 items were administered with other established health status and quality of life (QOL) instruments to 193 LTS of breast, prostate, colorectal cancers and lymphoma. Internal consistency reliability and validity of newly-derived scales was assessed.
Factor analysis of items using a priori QOL domains resulted in the derivation of ten new and specific subscales: health awareness, body changes, health worries, positive and negative self-evaluation, positive and negative life outlook, social life interferences, relationships, and meaning of cancer. Internal consistency measurements for these subscales ranged from 0.67 to 0.89. Expected associations within and among the IOC subscales and standardized measures of health status and QOL were observed, as were some unexpected findings.
Psychometric analysis indicated that this initial version of the Impact of Cancer instrument measures distinct and relevant constructs for LTS. Future work is necessary to confirm the factor structure, responsiveness and further validation of the instrument.
Cancer survivors; Instrument development; Psychometric testing; Quality of life
Diagnostic errors in primary care are harmful but poorly studied. To facilitate understanding of diagnostic errors in real-world primary care settings using electronic health records (EHRs), this study explored the use of the Situational Awareness (SA) framework from aviation human factors research.
A mixed-methods study was conducted involving reviews of EHR data followed by semi-structured interviews of selected providers from two institutions in the US. The study population included 380 consecutive patients with colorectal and lung cancers diagnosed between February 2008 and January 2009. Using a pre-tested data collection instrument, trained physicians identified diagnostic errors, defined as lack of timely action on one or more established indications for diagnostic work-up for lung and colorectal cancers. Twenty-six providers involved in cases with and without errors were interviewed. Interviews probed for providers' lack of SA and how this may have influenced the diagnostic process.
Of 254 cases meeting inclusion criteria, errors were found in 30 (32.6%) of 92 lung cancer cases and 56 (33.5%) of 167 colorectal cancer cases. Analysis of interviews related to error cases revealed evidence of lack of one of four levels of SA applicable to primary care practice: information perception, information comprehension, forecasting future events, and choosing appropriate action based on the first three levels. In cases without error, the application of the SA framework provided insight into processes involved in attention management.
A framework of SA can help analyze and understand diagnostic errors in primary care settings that use EHRs.
diagnostic error; decision-making; patient safety; primary care; medical errors; human factors; cancer; electronic health records; diagnostic delays
Studies provide conflicting results about the impact of comorbid conditions on the quality of chronic illness care. We assessed the effect of comorbidity type (concordant, discordant, or both) on the receipt of guideline-recommended care among patients with diabetes.
Patients were assigned to 1 of 4 condition groups: diabetes-concordant (hypertension, ischemic heart disease, hyperlipidemia), and/or -discordant (arthritis, depression, chronic obstructive pulmonary disease) conditions, or neither. We evaluated hemoglobin (Hb) A1c, blood pressure, and low-density lipoprotein cholesterol (LDL-C) readings at index and measured overall good quality of diabetes care, including a 6-month follow-up interval. We assessed the effect of condition group on overall good quality of care with logistic regression and generalized ordered logistic regression.
We assigned 35,872 patients to the diabetes comorbid condition groups, ranging from 2.0% in the discordant-only group to 58.0% in the concordant-only group. Patients with both types of conditions were more likely than those with no comorbidities to receive overall good quality for glycemic [odds ratio (OR), 2.13; 95% confidence interval (CI), 1.86-2.41], blood pressure (OR, 1.62; 95% CI, 1.40-1.84) and LDL-C (OR, 3.57; 95% CI, 3.08-4.05) control within 6 months of an index visit. They were also more likely to receive overall good quality for all 3 quality measures combined (OR, 2.17; 95% CI, 1.96-2.39).
Patients with the greatest clinical complexity were more likely than less complex patients to receive high quality diabetes care, suggesting that increased complexity does not necessarily predispose chronically ill patients to receiving poorer care. However, caution should be used in treating certain patient groups, such as the elderly, for whom adherence to multiple condition-specific guidelines may lack benefit or cause harm.
diabetes mellitus; quality of health care; comorbidity
To determine how reliance on Veterans Affairs (VA) for medical care among veterans enrolled in Medicare is affected by medical conditions, access, and patient characteristics.
Data Sources/Study Setting
Department of Veterans Affairs.
We examined reliance on the VA for inpatient, outpatient, and overall medical care among all VA users in fiscal years 2003 and 2004 who were also enrolled in Medicare. We calculated the marginal effects of patient factors on VA reliance using fractional logistic regression; we also analyzed overall VA reliance separately for under-65 and age-65+ groups. The primary focus of this analysis was the relationship between aggregated condition categories (ACCs), which represent medical conditions, and reliance on the VA.
Mean VA reliance was significantly higher in the under-65 population than in the age-65+ group (0.800 versus 0.531). Lower differential distance to the VA, and higher VA-determined priority for health care, predicted higher VA reliance. Most individual ACCs were negatively associated with VA reliance, though substance abuse and mental health disorders were significantly associated with increased reliance on VA care. Conditions of the eyes and ears/nose/throat had positive marginal effect on VA reliance for the under 65, while diabetes was positive for age 65+. Among inpatients, veterans with ACCs for mental health conditions, eye conditions, amputations, or infectious and parasitic conditions had higher likelihood of a VA hospitalization than inpatients without these conditions.
Many dually enrolled Veterans use both Medicare and VA health care. Age, accessibility, and priority level for VA services have a clear relationship with VA reliance. Because dual use is common, coordination of care among health care settings for such patients should be a policy priority.
Access to health care; veterans; Medicare; health services research; health services
Electronic health records (EHR) enable transmission and tracking of referrals between primary-care practitioners (PCPs) and subspecialists. We used an EHR to examine follow-up actions on electronic referral communication in a large multispecialty VA facility.
We retrieved outpatient referrals to five subspecialties between October 2006 and December 2007, and queried the EHR to determine their status: completed, discontinued (returned to PCP), or unresolved (no action taken by subspecialist). All unresolved referrals, and random samples of discontinued and completed referrals were reviewed to determine whether subspecialists took follow-up actions (i.e., schedule appointments anytime in the future) within 30 days of referral-receipt. For referrals without timely follow-up, we determined whether inaction was supported by any predetermined justifiable reasons or associated with certain referral characteristics. We also reviewed if PCPs took the required action on returned information.
Of 61,931 referrals, 22,535 were discontinued (36.4%), and 474 were unresolved (0.8%). We selected 412 discontinued referrals randomly for review. Of these, 52% lacked follow-up actions within 30 days. Appropriate justifications for inaction were documented in 69.8% (150/215) of those without action and included lack of prerequisite testing by the PCP and subspecialist opinion that no intervention was required despite referral. We estimated that at 30 days, 6.3% of all referrals were associated with an unexplained lack of follow-up actions by subspecialists. Conversely, 7.4% of discontinued referrals returned to PCPs were associated with an unexplained lack of follow-up.
Although the EHR facilitates transmission of valuable information at the PCP-subspecialist interface, unexplained communication breakdowns in the referral process persist in a subset of cases.
referrals; primary care; sub-specialty care; electronic health records; patient safety; health information technology; communication; diagnostic errors; patient follow-up
Recent work suggests that perceptions of the impact of cancer on survivors’ lives are associated with physical and mental health and quality of life (QOL) outcomes. This study examines the association between the Impact of Cancer version 2 scales (IOCv2) and these outcomes in a large sample of survivors of adult non-Hodgkin lymphoma (NHL).
Participants completed a mailed survey to assess physical and mental health (SF-36), cancer-specific quality of life (FACT-G) and perceived impact of cancer (IOCv2). Hierarchical multiple regression models, in which demographic, clinical, psychosocial and IOCv2 measures were added sequentially, were employed to evaluate their contribution to explaining variance in SF-36 and FACT-G scores.
A total of 652 post-treatment NHL survivors participated. Survivors with comorbidities and negative appraisals of life threat and treatment intensity reported worse physical and mental health and QOL (all P<.05). After controlling for demographic and clinical characteristics, younger respondents reported better physical but worse mental health and QOL (all P<.01). Lower IOCv2 Negative Impact (all P<.001) and higher Positive Impact (all P<.05) scores were associated with better physical and mental health and QOL after controlling for demographic, clinical and psychosocial characteristics.
Findings suggest that perceptions of cancer’s impact on survivors’ lives may influence or be influenced by health status and functioning and QOL. Longitudinal research is needed to establish causality, which could lead to the development of interventions targeting survivors’ impact of cancer concerns, and ultimately to the enhancement of overall health and QOL.
Quality of life; health status; non-Hodgkin lymphoma; psychosocial oncology; cancer survivors
Despite compelling evidence of the benefits of treatment and well-accepted guidelines for treatment, hypertension is controlled in less than one-half of United States citizens.
This randomized controlled trial tests whether explicit financial incentives promote the translation of guideline-recommended care for hypertension into clinical practice and improve blood pressure (BP) control in the primary care setting. Using constrained randomization, we assigned 12 Veterans Affairs hospital outpatient clinics to four study arms: physician-level incentive; group-level incentive; combination of physician and group incentives; and no incentives (control). All participants at the hospital (cluster) were assigned to the same study arm. We enrolled 83 full-time primary care physicians and 42 non-physician personnel. The intervention consisted of an educational session about guideline-recommended care for hypertension, five audit and feedback reports, and five disbursements of incentive payments. Incentive payments rewarded participants for chart-documented use of guideline-recommended antihypertensive medications, BP control, and appropriate responses to uncontrolled BP during a prior four-month performance period over the 20-month intervention. To identify potential unintended consequences of the incentives, the study team interviewed study participants, as well as non-participant primary care personnel and leadership at study sites. Chart reviews included data collection on quality measures not related to hypertension. To evaluate the persistence of the effect of the incentives, the study design includes a washout period.
We briefly describe the rationale for the interventions being studied, as well as the major design choices. Rigorous research designs such as the one described here are necessary to determine whether performance-based payment arrangements such as financial incentives result in meaningful quality improvements.
Delays in colorectal cancer (CRC) diagnosis related to colonoscopy referrals are not well studied. We tested whether certain details of information transmitted through computerized provider order entry (CPOE)-based referrals affected timeliness of diagnostic colonoscopy for patients with newly diagnosed colorectal cancer (CRC).
We studied a 6-year cohort of all newly diagnosed patients with CRC at a large tertiary care Veterans Affairs hospital and its affiliated multispecialty clinics. Referring providers included primary care clinicians, resident trainees, and other specialists. From the colonoscopy referral preceding CRC diagnosis, we determined request date, type and frequency of diagnostic clues provided (symptoms, signs, test results), notation of urgency, and documented evidence of verbal contact between referring provider and consultant to expedite referral. We compared distributions of proportions of diagnostic clues between patients with > 60 and ≤ 60 day lag and examined predictors of lag time.
Of 367 electronic referrals identified with a median lag of 57 days, 178 (48.5%) had lag > 60 days. Referrals associated with longer lag times included those with “positive fecal occult blood test” (92 days, P<0.0001), “hematochezia” (75 days, P=0.02), “history of polyps” (221 days, P=0.0006), and when “screening” (versus specific symptoms) was given as reason for diagnostic colonoscopy (203 days, P=0.002). Independent predictors of shorter wait times included 3 diagnostic clues, notation of urgency, and documentation of verbal contact.
Attention to certain details of diagnostic information provided to consultants through CPOE-based referrals may help reduce delays in CRC diagnosis.
delayed cancer diagnosis; colorectal cancer; colonoscopy referrals; computerized order entry; electronic medical records; primary care
Moderately preterm infants (30–34 6/7 weeks’ gestational age) represent the largest population of NICU residents. Whether their clinical outcomes are associated with differences in NICU nurse-staffing arrangements has not been assessed. The objective of this study was to test the influence of patient-to-nurse ratios (PNRs) on outcomes of care provided to moderately preterm infants.
PATIENTS AND METHODS
Using data from a prospective, multicenter, observational cohort study of 850 moderately preterm infants from 10 NICUs in California and Massachusetts, we tested for associations between PNR and several important clinical outcomes by using multivariate random-effects models. To correct for the influence of NICU size, we dichotomized the sample into those with an average daily census of <20 or ≥20 infants.
Overall, we found few clinically significant associations between PNR and clinical outcomes of care. Mean PNRs were higher in large compared with small NICUs (2.7 vs 2.1; P < .001). In bivariate analyses, an increase in PNR was associated with a slightly higher daily weight gain (5 g/day), greater gestational age at discharge, any intraventricular hemorrhage, and severe retinopathy of prematurity. After controlling for case mix, NICU size, and site of care, an additional patient per nurse was associated with a decrease in daily weight gain by 24%. Other variables were no longer independently associated with PNR.
In this population of moderately preterm infants, the PNR was associated with a decrease in daily weight gain, but was not associated with other measures of quality. In contrast with findings in the adult intensive care literature, measured clinical outcomes were similar across the range of nurse-staffing arrangements among participating NICUs. We conclude that the PNR is not useful for profiling hospitals’ quality of care delivery to moderately preterm infants.
infant; newborn; health services research; nurse-staffing; outcomes
Pay-for-performance initiatives in medicine are proliferating rapidly. Neonatal intensive care is a likely target for these efforts because of the high cost, available databases, and relative strength of evidence for at least some measures of quality. Pay-for-performance may improve patient care but requires valid measurements of quality to ensure that financial incentives truly support superior performance. Given the existing uncertainty with respect to both the effectiveness of pay-for-performance and the state of quality measurement science, experimentation with pay-for-performance initiatives should proceed with caution and in controlled settings. In this article, we describe approaches to measuring quality and implementing pay-for-performance in the NICU setting.
pay-for-performance programs; quality improvement
To develop and explore the characteristics of a novel “nearest neighbor” methodology for creating peer groups for health care facilities.
Data were obtained from the Department of Veterans Affairs (VA) databases.
Statistical Methods and Findings
Peer groups are developed by first calculating the multidimensional Euclidean distance between each of 133 VA medical centers based on 16 facility characteristics. Each medical center then serves as the center for its own peer group, and the nearest neighbor facilities in terms of Euclidean distance comprise the peer facilities. We explore the attributes and characteristics of the nearest neighbor peer groupings. In addition, we construct standard cluster analysis-derived peer groups and compare the characteristics of groupings from the two methodologies.
The novel peer group methodology presented here results in groups where each medical center is at the center of its own peer group. Possible advantages over other peer group methodologies are that facilities are never on the “edge” of a group and group size—and thus group dispersion—is determined by the researcher. Peer groups with these characteristics may be more appealing to some researchers and administrators than standard cluster analysis and may thus strengthen organizational buy-in for financial and quality comparisons.
Peer groups; cluster analysis; nearest neighbor; Euclidean distance; quality of care comparisons
Follow-up of abnormal outpatient laboratory test results is a major patient safety concern. Electronic medical records can potentially address this concern through automated notification. We examined whether automated notifications of abnormal laboratory results (alerts) in an integrated electronic medical record resulted in timely follow-up actions.
We studied four alerts: hemoglobin A1c (HbA1c) ≥15%, positive hepatitis C antibody (HCV), prostate specific antigen (PSA) ≥15 ng/mL, and thyroid stimulating hormone (TSH) ≥ 15 mIU/L. An alert tracking system determined whether the alert was acknowledged (i.e. provider clicked on and opened the message) within two weeks of transmission; acknowledged alerts were considered read. Within 30 days of result transmission, record review and provider contact determined follow-up actions (e.g. patient contact, treatment etc.). Multivariable logistic regression models analyzed predictors for lack of timely follow-up.
Between May 2008 and December 2008, 78,158 tests (HbA1c, HCV, TSH and PSA) were performed, of which 1163 (1.48%) were transmitted as alerts; 10.2% of these (119/1163) were unacknowledged. Timely follow-up was lacking in 79 (6.8%) and was statistically not different for acknowledged and unacknowledged alerts (6.4% vs. 10.1%; p =.13). Two-hundred two alerts (17.4% of 1163) arose from unnecessarily ordered (redundant) tests. Alerts for a new versus known diagnosis were more likely to lack timely follow-up (OR: 7.35; 95% CI: 4.16-12.97) whereas alerts related to redundant tests were less likely to lack timely follow-up (OR: 0.24; 95% CI: 0.07-0.84).
Safety concerns related to timely patient follow-up remain despite automated notification of non-life threatening abnormal laboratory results in the outpatient setting.
diagnostic errors; abnormal diagnostic test results; electronic medical records; patient follow-up; patient safety; health information technology; communication; primary care
Delayed diagnosis of colorectal cancer (CRC) is among the most common reasons for ambulatory diagnostic malpractice claims in the United States. Our objective was to describe missed opportunities to diagnose CRC before endoscopic referral, in terms of patient characteristics, nature of clinical clues, and types of diagnostic-process breakdowns involved.
We conducted a retrospective cohort study of consecutive, newly diagnosed cases of CRC between February 1999 and June 2007 at a tertiary health-care system in Texas. Two reviewers independently evaluated the electronic record of each patient using a standardized pretested data collection instrument. Missed opportunities were defined as care episodes in which endoscopic evaluation was not initiated despite the presence of one or more clues that warrant a diagnostic workup for CRC. Predictors of missed opportunities were evaluated in logistic regression. The types of breakdowns involved in the diagnostic process were also determined and described.
Of the 513 patients with CRC who met the inclusion criteria, both reviewers agreed on the presence of at least one missed opportunity in 161 patients. Among these patients there was a mean of 4.2 missed opportunities and 5.3 clues. The most common clues were suspected or confirmed iron deficiency anemia, positive fecal occult blood test, and hematochezia. The odds of a missed opportunity were increased in patients older than 75 years (odds ratio (OR) = 2.3; 95% confidence interval (CI) 1.3–4.1) or with iron deficiency anemia (OR = 2.2; 95% CI 1.3–3.6), whereas the odds of a missed opportunity were lower in patients with abnormal flexible sigmoidoscopy (OR = 0.06; 95% CI 0.01–0.51), or imaging suspicious for CRC (OR = 0.3; 95% CI 0.1–0.9). Anemia was the clue associated with the longest time to endoscopic referral (median = 393 days). Most process breakdowns occurred in the provider–patient clinical encounter and in the follow-up of patients or abnormal diagnostic test results.
Missed opportunities to initiate workup for CRC are common despite the presence of many clues suggestive of CRC diagnosis. Future interventions are needed to reduce the process breakdowns identified.
Given the fragmentation of outpatient care, timely follow-up of abnormal diagnostic test results remains a challenge. We hypothesized that an EMR that facilitates the transmission and availability of critical imaging results through either automated notification (alerting) or direct access to the primary report would eliminate this problem.
We studied critical imaging alert notifications in the outpatient setting of a tertiary care VA facility from November 2007 to June 2008. Tracking software determined whether the alert was acknowledged (i.e. provider opened the message for viewing) within two weeks of transmission; acknowledged alerts were considered read. We reviewed medical records and contacted providers to determine timely follow-up actions (e.g. ordering a follow-up test or consultation) within 4 weeks of transmission. Multivariable logistic regression models accounting for clustering effect by providers analyzed predictors for two outcomes; lack of acknowledgment and lack of timely follow-up.
Of 123,638 studies (including X-rays, CT scans, ultrasounds, MRI and mammography), 1196 (0.97%) images generated alerts; 217 (18.1%) of these were unacknowledged. Alerts had a higher risk of being unacknowledged when ordering providers were trainees (OR, 5.58;95%CI, 2.86-10.89) and when dual (more than one provider alerted) as opposed to single communication was used (OR, 2.02;95%CI, 1.22-3.36). Timely follow-up was lacking in 92 (7.7% of all alerts) and was similar for acknowledged and unacknowledged alerts (7.3% vs. 9.7%;p=0.2). Risk for lack of timely follow-up was higher with dual communication (OR,1.99;95%CI, 1.06-3.48) but lower when additional verbal communication was used by the radiologist (OR, 0.12;95%CI: 0.04-0.38). Nearly all abnormal results lacking timely follow-up at 4 weeks were eventually found to have measurable clinical impact in terms of further diagnostic testing or treatment.
Critical imaging results may not receive timely follow-up actions even when providers receive and read results in an advanced, integrated EMR system. A multidisciplinary approach is needed to improve patient safety in this area.
diagnostic errors; abnormal diagnostic test results; electronic medical records; patient follow-up; patient safety; health information technology; communication; primary care
Inadequate follow-up of abnormal fecal occult blood test (FOBT) results occurs in several types of practice settings. Our institution implemented multifaceted quality improvement (QI) activities in 2004–2005 to improve follow-up of FOBT positive results. Activities addressed pre-colonoscopy referral processes and system-level factors such as electronic communication and provider education and feedback. We evaluated their effects on timeliness and appropriateness of positive FOBT follow-up and identified factors that affect colonoscopy performance.
Retrospective electronic medical record (EMR) review was used to determine outcomes pre- and post-QI activities in a multi-specialty ambulatory clinic of a tertiary care Veterans Affairs facility and its affiliated satellite clinics. From 1869 FOBT positive cases, 800 were randomly selected from time periods before and after QI activities. Two reviewers used a pretested standardized data collection form to determine whether colonoscopy was appropriate or indicated based on pre-determined criteria and if so, the timeliness of colonoscopy referral and performance pre- and post-QI activities.
In cases where a colonoscopy was indicated, the proportion of patients who received a timely colonoscopy referral and performance were significantly higher post implementation (60.5% vs. 31.7%, p<0.0001 and 11.4% vs. 3.4%, p =0.0005 respectively). A significant decrease also resulted in median times to referral and performance (6 vs. 19 days p<0.0001 and 96.5 vs. 190 days p<0.0001 respectively) and in the proportion of positive FOBT test results that had received no follow-up by the time of chart review (24.3%vs. 35.9%; p=0.0045). Significant predictors of absence of the performance of an indicated colonoscopy included performance of a non-colonoscopy procedure such as barium enema or flexible sigmoidoscopy (OR=16.9; 95% CI 1.9–145.1), patient non-adherence (OR=33.9; 95% CI 17.3–66.6), not providing an appropriate provisional diagnosis on the consultation (OR= 17.9; 95% CI 11.3–28.1) and gastroenterology service not rescheduling colonoscopies after an initial cancellation (OR= 11.0; 95% CI 5.1–23.7)
Multifaceted QI activities improved rates of timely colonoscopy referral and performance in an EMR system. However, colonoscopy was not indicated in over one third of patients with positive FOBTs, raising concerns about current screening practices and the appropriate denominator used for performance measurement standards related to colon cancer screening.
Although several types of computerized provider order entry (CPOE)-related errors may occur, errors related to inconsistent information within the same prescription (ie, mismatch between the structured template and the associated free-text field) have not been described, to our knowledge. We determined the nature and frequency of such errors and identified their potential predictive variables.
In this prospective study, we enrolled pharmacists to report prescriptions containing inconsistent communication over a 4-month period at a tertiary care facility. We also electronically retrieved all prescriptions written during the study period containing any comments in the free-text field and then randomly selected 500 for manual review to determine inconsistencies between free-text and structured fields. Of these, prescriptions without inconsistencies were categorized as controls. Data on potentially predictive variables from reported and unreported errors and controls were collected. For all inconsistencies, we determined their nature (eg, drug dosage or administration schedule) and potential harm and used multivariate logistic regression models to identify factors associated with errors and harm.
Of 55 992 new prescriptions, 532 (0.95%) were reported to contain inconsistent communication, a rate comparable to that obtained from the unreported group. Drug dosage was the most common inconsistent element among both groups. Certain medications were more likely associated with errors, as was the inpatient setting (odds ratio, 3.30; 95% confidence interval, 2.18–5.00) and surgical subspecialty (odds ratio, 2.45; 95% confidence interval, 1.57–3.82). About 20% of errors could have resulted in moderate to severe harm, for which significant independent predictors were found.
Despite standardization of data entry, inconsistent communication in CPOE poses a significant risk to safety. Improving the usability of the CPOE interface and integrating it with workflow may reduce this risk.
There is concern that performance measures, patient ratings of their care, and pay-for-performance programs may penalize health care providers of patients with multiple chronic co-existing conditions. We examined the impact of co-existing conditions on the quality of care for hypertension and patient perception of overall quality of their health care.
Methods and Results
141,609 veterans with hypertension were classified into 4 condition groups: those with hypertension-concordant (diabetes, ischemic heart disease, dyslipidemia) and/or discordant (arthritis, depression, chronic obstructive pulmonary disease) conditions, or neither. We measured blood pressure control at the index visit, overall good quality of care for hypertension including a follow-up interval, and patient ratings of satisfaction with their care. Association between condition type and number of co-existing conditions on receipt of overall good quality of care were assessed using logistic regression. Relationship between patient assessment and objective measures of quality was assessed. Of the cohort, 49.5% had concordant-only comorbidities, 8.7% had discordant-only comorbidities, 25.9% had both, and 16.0% had none. Odds of receiving overall good quality after adjusting for age were higher for those with concordant comorbidities (odds ratio [OR], 1.78; 95% confidence interval [CI], 1.70–1.87), discordant comorbidities (OR, 1.32; 95% CI 1.23–1.41) or both (OR, 2.25; 95% CI, 2.13–2.38), compared with neither. Findings did not change after adjusting for illness severity and/or number of primary care and specialty care visits. Patient assessment of quality did not vary by the presence of coexisting conditions and was not related to objective ratings of quality of care.
Contrary to expectation, patients with greater complexity had higher odds of receiving high quality care for hypertension. Subjective ratings of care did not vary with the presence or absence of the comorbid conditions. Our findings should be reassuring to those who care for the most medically complex patients and are concerned that they will be penalized by performance measures or patient ratings of their care.
Hypertension; Quality Indicators; Health Care; Process Assessment (Health Care); Comorbidity; Physician Incentive Plans
The measurement of healthcare provider performance is becoming more widespread. Physicians have been guarded about performance measurement, in part because the methodology for comparative measurement of care quality is underdeveloped. Comprehensive quality improvement will require comprehensive measurement, implying the aggregation of multiple quality metrics into composite indicators.
To present a conceptual framework to develop comprehensive, robust, and transparent composite indicators of pediatric care quality, and to highlight aspects specific to quality measurement in children.
We reviewed the scientific literature on composite indicator development, health systems, and quality measurement in the pediatric healthcare setting. Frameworks were selected for explicitness and applicability to a hospital-based measurement system.
We synthesized various frameworks into a comprehensive model for the development of composite indicators of quality of care. Among its key premises, the model proposes identifying structural, process, and outcome metrics for each of the Institute of Medicine's six domains of quality (safety, effectiveness, efficiency, patient-centeredness, timeliness, and equity) and presents a step-by-step framework for embedding the quality of care measurement model into composite indicator development.
The framework presented offers researchers an explicit path to composite indicator development. Without a scientifically robust and comprehensive approach to measurement of the quality of healthcare, performance measurement will ultimately fail to achieve its quality improvement goals.