This study of alcohol screening in the VA found that 61% of patients who screened positive for alcohol misuse on a mailed survey screened negative when screened clinically despite use of the same validated screening questionnaire, suggesting that many patients who could benefit from brief alcohol counseling are being missed by clinical screening in VA. Black race and the VA Network where clinical screening was conducted were the only factors other than survey AUDIT-C scores that were associated with discordance between clinical and survey AUDIT-C screens in fully-adjusted analyses.
Some discordance between clinical and survey screens is expected. Patients might be more motivated to report drinking honestly in clinical settings if they feel that their provider needs the information and/or it might be relevant to their health. In addition, some discordance is expected due to random measurement error, changes in patients’ drinking between two screens, or regression to the mean. However, the observed discordance between clinical and survey AUDIT-Cs cannot be accounted for by these factors. Discordance due to random measurement error and changes in drinking would be expected to occur in a normal distribution around the screening threshold similar to that observed across clinical screening scores (Fig. Panel a). However, the distribution of discordance across survey screening scores was not normal (Fig. , Panel b). Furthermore, discordance due to changes in drinking should increase as the time between screens increases, but no such association was observed. Discordance cannot solely reflect regression to the mean or decreased drinking at repeat screening because there was no association between the order of screens and discordance. Furthermore, randomized controlled trials suggest that repeated screening leads to lower reported consumption on later screens.26–28
This bias would be expected to result in lower AUDIT-C scores on the survey, which tended to follow the clinical screen, the opposite of the observed association.
Social desirability bias likely contributed to the observed discordance. Over twice as many patients who screened positive on confidential mailed surveys had discordant results compared to patients who had positive clinical screens (61% vs. 24%). If the latter is an estimate of the magnitude of “expected” discordance, 37% (95% CI 34-41%) of patients who screened positive on surveys had “excess” discordance. Patients may under-report alcohol consumption on clinical alcohol screening due to stigma or a desire to avoid discussing their drinking with providers.29
The fact that Black patients were more likely than White patients to have discordant screening results might reflect greater social desirability bias among these patients, although it could also reflect bias due to differences in the way the AUDIT-C was interpreted and/or administered across racial/ethnic subgroups.
Factors other than social desirability bias likely contributed to the magnitude of the observed discordance for several reasons. First, the AUDIT-C was validated in studies that used interviewers to administer screens,10
and in which patients were aware providers would receive screening results.8
Second, the variation in discordance across VA networks suggests that institutional factors contributed to the observed discordance. Anecdotal reports suggest that considerable variability exists across sites in the privacy of screening. Differences in training and/or decisions about who conducts screening might also contribute to variability; medical assistants or nurses may be more likely to follow screening instructions verbatim than primary care or mental health providers assessing alcohol use as part of the medical history.
Several limitations of this study should be noted. In order to study a large diverse national sample without alerting providers that the quality of screening was being evaluated, this study used AUDIT-Cs from confidential mailed surveys as a comparison standard. While in-depth interviews are often the ideal comparison standard, recruiting patients and providers for such studies biases results.30
In addition, primary data collection would have delayed assessment of the quality of screening. Finally, use of secondary data allowed us to evaluate regional variation in the quality of alcohol screening in a cost-efficient manner. For these reasons, the AUDIT-C from a mailed survey was the best available comparison standard for this translational study of alcohol screening implementation.
Other limitations relate to the study sample. Patients who returned the VA’s outpatient satisfaction survey are older and drink less than non-respondents,30
potentially under-estimating discordance since patients who screened positive for alcohol misuse had higher discordance. The study sample was also too small to evaluate facility-level variation. VA patients differ in important ways from other outpatients, and the VA health care system is currently atypical in that it uses performance incentives to achieve high rates of alcohol screening. Finally, this study did not collect data on alcohol screening procedures across VA networks, so it could not determine whether differences in implementation of alcohol screening account for the observed differences in discordance across VA networks.
Nevertheless, this study has important implications for health care systems implementing routine alcohol screening. Almost 7% of the total study sample screened positive for alcohol misuse on surveys but were missed by clinical screening, and 1.5% of the total sample had severe alcohol misuse that was missed. No known prior studies have evaluated the validity of alcohol screening when integrated into routine clinical care. This study found that use of validated questionnaires does not—by itself—ensure the quality of screening and suggests that the quality of clinical alcohol screening should be monitored, even when well-validated screening questionnaires are used. While it is unknown whether a similar issue affects screening for other health risk behaviors and mental health conditions, this issue merits evaluation. Self-administered measures of alcohol screening, whether on paper, online as in electronic health risk assessments (eHRA),31
or by interactive voice recording on the telephone (IVR),32
may be the most valid approaches to implementing alcohol screening. If alcohol screening is administered by clinicians they may need focused training to prepare them to screen in a valid manner.33,34
This study is another demonstration of the challenge of developing effective performance measures for preventive care.35
The current VA alcohol screening performance measure that sets targets for rates of alcohol screening creates incentives for documentation of screening results, but does not provide incentives for high quality screening that identifies patients with alcohol misuse. Furthermore, setting very high targets for screening could contribute to lower quality screening by encouraging providers to document screening when they do not have time to ask screening questions verbatim in a private setting. Future research must address the need for performance measures that create incentives for providers not only to screen, but to identify patients with alcohol misuse.
The VA has recently succeeded in achieving high rates of annual alcohol screening. Brief alcohol interventions are effective for patients with alcohol misuse,36
and patients with alcohol use disorders benefit from referral or repeated brief interventions, with and without lab monitoring or medications.37–40
This study suggests that mandating clinical use of a validated alcohol screening questionnaire does not ensure high quality screening. Three out of every five patients who screened positive for alcohol misuse on confidential mailed surveys were not identified by clinical screening. Put another way, 6,821 patients with alcohol misuse would be missed out of every 100,000 patients screened in VA. Moreover, significant variation across VA networks, after accounting for differences in patient characteristics, suggests organizational influences on the quality of alcohol screening. Together these findings indicate a need to focus on monitoring and improving the quality of alcohol screening in order to identify as many patients as possible who could benefit from brief alcohol interventions.