|Home | About | Journals | Submit | Contact Us | Français|
Sleep disturbance is a significant problem for adults presenting to primary care. Though it is recommended that primary care providers screen for sleep problems, a brief, effective screening tool is not available.
The aim of this preliminary study was to test the utility of item three of the Patient Health Questionnaire-9-item (PHQ-9) as a self-report screening test for sleep disturbance in primary care.
This was a cross-sectional survey of male VA primary care patients in Syracuse and Rochester, NY. Sensitivity and specificity statistics were calculated as well as positive and negative predictive value to determine both whether the PHQ-9 item-3 can be used as an effective sleep screen in primary care and at what PHQ item-3 cut score patients should be further assessed for sleep disturbance.
One hundred and eleven male, VA primary care patients over the age of 18 and without gross neurological impairment participated in this one-session, in-person study.
During the research session, patients completed several questionnaires, including a basic demographic questionnaire, the PHQ-9, and the Insomnia Severity Index (ISI).
PHQ-9 item 3 significantly correlated with the total score on the ISI (r=0.75, p<0.0001). A cut score of 1 on the PHQ-9 item 3, indicating sleep disturbance at least several days in the last two weeks, showed the best balance of sensitivity (82.5%) and specificity (84.5%) as well as positive (78.4%) and negative (91%) predictive value.
Item 3 of the PHQ-9 shows promise as a screener for sleep problems in primary care. Using this one-item of a popular screening measure for depression in primary care allows providers to easily screen for two important issues without unnecessarily adding significant burden.
Sleep disturbances, like insomnia and excessive daytime sleepiness (EDS), are highly prevalent in primary care.1,2 Insomnia is common among the general population, with numerous population-based studies reporting prevalence rates of approximately 30% for insomnia complaint and 10% for chronic insomnia,3 with rates in primary care even higher.1,4–6 Prevalence rates for EDS vary even more widely (from 1% to 30%) based on the operationalization of the term.7
The presence of insomnia, hypersomnia or EDS may constitute a symptom of a sleep disorder, a side effect of a medication, or a symptom of depression.7 Nonetheless, together and separately, these sleep disturbances are associated with their own significant morbidity and societal costs and often contribute to distress in patients. Experiencing sleep disturbances increases the likelihood of medical and psychiatric morbidity, contributes to greater healthcare utilization and accidents, and is associated with poor occupational performance and decreased quality of life.8–14 The economic impact of both insomnia and EDS is substantial, costing approximately $100 billion a year in the U.S..14–17
Although there is little research examining hypersomnia within primary care, insomnia and EDS are common complaints yet rarely mentioned during office visits18,19 or as the reason for making an appointment.20 Therefore, it has been suggested that screening for insomnia during routine primary care visits might be an effective secondary prevention effort,19,20 especially because there are several efficacious pharmacological and non-pharmacological treatments.3,21 Similarly, sleep disturbances related to EDS and/or hypersomnia continue to be under-diagnosed22 and, likely under-treated, despite the existence of effective treatments.23
The implementation of interventions requires primary care providers (PCPs) to have an efficient and effective way of identifying sleep disturbance in their patients.20 PCPs cannot be expected to implement the increasing number of recommended screenings each year unless efforts are made to ease this burden. Because it is currently recommended that PCPs routinely screen for depression in adults,24 and the Personal Health Questionnaire-925 (PHQ-9) is recommended as an effective screening measure for depression,26 it would simplify the process for providers if the PHQ-9 could also be used to screen for sleep disturbance. One item on the PHQ-9 assesses sleep; it asks individuals to indicate how often in the past two weeks they have had difficulty falling or staying asleep (i.e., an insomnia screening question) or sleeping too much (i.e., a hypersomnia screening question). The insomnia portion of this question has been used in several epidemiologic studies to identify insomnia7 and similar items exist on validated insomnia instruments.27 The hypersomnia portion of the PHQ-9 sleep item is not as well represented in epidemiologic studies.7
Therefore, the purpose of this preliminary study was to examine whether the sleep item on the PHQ-9 could be used as an adequate screener for sleep disturbance in a Veteran’s Affairs (VA) primary care setting with a focus on insomnia in particular. The VA is an ideal setting to examine this question because it requires annually screening patients for depression using the PHQ-2 and can follow-up any positive screen with a PHQ-9 questionnaire.28
Participants were recruited between February 2009 and May 2010 as part of a larger, cross-sectional, descriptive study investigating the prevalence of and relationship between physical and mental health risk factors (e.g. depression, smoking) in patients attending VA primary care clinics. Three recruitment methods were used: direct PCP referral, primary care waiting room referral, or through the Behavioral Health Assessment Center (BHAC). BHAC is an extension of the Syracuse and Rochester VA primary care clinics and employs health technicians who complete a comprehensive clinical assessment of patients’ mental health symptomatology over the telephone.29 Veterans are called for this assessment either because of a direct referral from their PCP or because they scored positive on an annual primary care screen for hazardous alcohol use (scores of 4 and above for men and 3 or above for women on the Alcohol Use Disorders Identification Test—Consumption (AUDIT-C)30). At the end of the phone call, the patient is asked if he/she is willing to be contacted regarding research participation. Patients expressing interest in research were then contacted via telephone and recruited for the study. As such, most of the patients in this sample were regular drinkers, although level of alcohol consumption was not an inclusion criterion for this study. Patients were eligible to participate if they were 18 years of age or older, did not exhibit gross neurological impairment, and attended a primary care clinic at the Syracuse VA Medical Center or the Rochester Outpatient Clinic.
A total of 575 patients were eligible to participate and were contacted regarding research participation in this or other studies being conducted contemporaneously. Of these, 220 participants declined research participation and 242 participated in other studies. Thus, the initial sample for this study consisted of 113 participants.
Participants completed a variety of questionnaires assessing demographic information and health behaviors during one, in-person research session. The session was conducted within either the Syracuse VA Medical Center or Rochester VA Outpatient primary care clinics and lasted approximately one hour. Participants were compensated 15 US-dollars for their time. Funding for this study was provided by the Center for Integrated Healthcare at the Syracuse VA Medical Center. All procedures were approved by the Syracuse VA Medical Center Institutional Review Board, and all participants provided written informed consent prior to participation.
Participants answered questions regarding their demographic information (e.g. age, race, marital status, education level, and annual household income).
This is a 9-item, self-report measure assessing the frequency of symptoms of depressed mood over the past 2 weeks (ranging from 0, or not at all, to 3, or nearly every day). Item #3 of this measure (PHQ item-3) asks the patient to rate how often he/she has had, “Trouble falling or staying asleep, or sleeping too much,” assessing the frequency of both insomnia and hypersomnia, which are symptoms of depression.27
This is a 7-item, self-report measure assessing the presence and severity of insomnia symptoms, as well as the effect of sleep disturbance on daily functioning. The ISI has been validated in adults31 and found to adequately assess the diagnostic criteria for insomnia.32 Items are rated on a Likert-type scale ranging from 0 to 4, with higher scores reflecting more sleep disturbance. Scores on this measure are summed and may be categorized into 4 levels of insomnia severity: 0–7 indicates no clinically significant insomnia, 8–14 indicates the presence of subthreshold insomnia symptoms, 15–21 indicates the presence of moderate insomnia symptoms, and 22–28 indicates severe insomnia symptoms.27,33 Thus, participants with a score of 8 or higher on this measure were classified as scoring positive for insomnia symptoms. We chose a cutoff of 8 to capture people with subthreshold symptoms because screening measures are useful not only to identify patients who already have a diagnosis, but those who have subthreshold symptoms, and who could still benefit from an intervention to prevent the full development of the disorder.
The AUDIT-C is a summed score of the first three items of the AUDIT34 a 10-item questionnaire used to assess symptoms of alcohol use disorders. Items are scored on a 0–4 point scale, with higher scores indicative of a greater likelihood of an alcohol use disorder. The AUDIT-C30 is the summed score of the consumption questions of the AUDIT, which assess quantity and frequency of drinking and binge episodes (6 or more drinks on one occasion).
The PC-PTSD is a 4-item questionnaire assessing the presence of the following post-traumatic symptoms in the past month: re-experiencing, avoidance, hypervigilance, and emotional numbing related to past trauma. This measure has demonstrated strong diagnostic efficiency35,36.
After confirming normality in variable distribution, Pearson correlations were computed across all variables pertinent to the analyses (e.g., PHQ-item 3, ISI total score, AUDIT-C, PTSD scores) and demographic variables (e.g., age, marital status, education, and income), which have been found to be associated with insomnia symptoms.3,37,38 (Categorical variables were dummy coded). Next, false negative (i.e., number of people who endorsed no sleep problems on the PHQ item-3 but who scored positive for insomnia symptoms on the ISI) and false positive (i.e., the number of people who endorsed sleeping problems on the PHQ item-3 but who did not screen positive for insomnia on the ISI) rates were calculated. Positive predictive values (i.e., dividing true positives (those scoring an 8 or higher on the ISI) by the sum of true and false positives multiplied by 100) and negative predictive values (dividing true negatives (those scoring a 7 or below on the ISI) by the sum of true and false negatives) were then calculated. Specificity was calculated as the number of participants who were correctly identified as having no sleep problems on the PHQ item-3 divided by the total number of participants without insomnia as indicated by a score less than 8 on the ISI, multiplied by 100. Thus, this statistic yields a percentage of people correctly identified by the PHQ item-3 as not having insomnia, or the PHQ item-3’s ability to identify true negatives. Sensitivity was calculated as the number of participants correctly identified as having insomnia by way of the PHQ item-3 divided by the total number of participants with insomnia as measured by the ISI multiplied by 100. Thus, this statistic yields a percentage of people correctly identified by the PHQ item-3 as having insomnia, or its ability to identify true positives. Lastly receiver operating curve (ROC) analysis was used to compare the PHQ item-3 with the presence or absence of sleep disturbance as assessed by the ISI.
Participants in this study had a mean age of 60.62 (SD=17.04), and the large majority of patients were male (98.2%, n=111). Due to the low number of females and the known association between gender and insomnia symptoms,39,40 the two females were excluded from the analyses. Thus, the final sample consisted of 111 males with a mean age of 60.80 years (SD=17.13), and was predominantly Caucasian (89.2%; n=99). All variables were normally distributed, and sample characteristics are presented in Table 1.
Pearson correlation analyses revealed a strong,41 significant, positive correlation between the PHQ item-3 and ISI total score (r=0.75, p<0.0001). Age significantly correlated with both the PHQ item-3 (r=−0.28, p<0.01) and the ISI (r=−0.35, p<0.001) such that younger individuals had more sleeping difficulties than older individuals. Correlations are presented in Table 2.
Summary statistics for positive and negative predictive value, sensitivity, and specificity are presented in Table 3. Of note, a cut score of 1, indicating sleep difficulty on several days or more in the last two weeks on the PHQ item-3 demonstrated the best balance of false negative and false positive rates with 7 and 11, respectively, as well as the best sensitivity (82.5%) and specificity (84.5%) rates. It also demonstrated the best balance of positive and negative predictive values at 78.4% and 91% respectively. The area under the ROC curve (AUC) for the PHQ item-3 plotted against an ISI positive score was 0.87 (95% CI=0.80 to 0.94) (see Fig. 1).
Correlation analyses revealed a negative relationship between age and sleep difficulties on both sleep indices, which is contrary to the literature stating older individuals tend to have more sleep difficulties.42,43 As such, we conducted a post hoc linear regression analysis to determine whether the fact that most participants in this sample were regular drinkers could account for the negative relationship between sleep difficulties and age. In this analysis, ISI total score served as the dependent variable with the PHQ item-3 serving as the independent variable. Age, AUDIT-C score, and the demographic variables served as covariates. Results revealed that the PHQ item-3 significantly predicted ISI total score (β=5.08, p<0.0001). After covarying for alcohol use, age was still a significant negative predictor of sleep problems (β=−0.06, p<0.05). Neither alcohol use nor demographic variables significantly predicted sleep difficulty.
We also examined whether PTSD symptoms could explain the negative relationship between age and sleep difficulty because PTSD symptoms have a known association with sleep difficulty44 and were significantly correlated with both sleep indices and age in our sample (see Table 2). Thus, we conducted an additional linear regression analysis using ISI total score as the dependent variable, PHQ item-3 as the independent variable, and age, AUDIT-C score, PC-PTSD score, and the demographic variables as covariates. After adding PTSD symptoms to the model, age no longer significantly predicted sleep difficulties (β=−0.03, p=0.34).
This study demonstrated that item 3 of the PHQ-9 is a potentially effective screener to identify sleep disturbance in VA primary care settings. Specifically, our results show that a cut score of 1 on the PHQ item-3 (indicating sleep problems several days in the last 2 weeks) provides the best balance of sensitivity and specificity, such that 82.5% of patients with sleep disturbance were correctly identified while 84.5% of patients without sleep difficulty were correctly ruled out using the PHQ item-3.
These results are particularly promising given the current recommended use of the PHQ-9 in primary care settings.26 The implication of these results is that PCPs may screen for sleep disturbances without implementing a separate, sleep-specific measure. Instead, should a patient endorse a 1 or higher on item 3 on the PHQ-9, it could signal to the provider to further assess the patient’s sleep and intervene if necessary. Sleep interventions are effective in both improving sleep in primary insomnia patients,3,13,45 as well as in patients who are experiencing insomnia and other sleep disturbances comorbid with other disorders.46–50 Furthermore, preliminary studies suggest that improving sleep quality may also improve symptoms of other physical and mental health disorders associated with insomnia,51–53 further emphasizing the importance of identifying and treating this, and potentially other common sleep disturbances.
It is of note that while the ISI has been found to adequately assess the diagnostic criteria for insomnia,27 it does not provide a valid diagnosis for insomnia or any other sleep problem. Additionally, the established ISI cutoff score of 8 includes insomnia symptoms that may be considered ‘mild’ or subthreshold.27 However, it was important to capture those patients with mild/subthreshold sleep problems with the ISI to test the PHQ item-3’s efficacy at identifying patients with subthreshold symptoms as well as a potential sleep disorder. The PHQ item-3 performed well in this capacity, as evidenced by its high level of sensitivity (82.5%), which is particularly important in a screening measure.54
Although this study is an important first step in the validation of the PHQ item-3 as an effective screening tool in primary care, our results should be interpreted in the context of some important limitations. Our sample consisted of male veterans from two Upstate New York VA clinics, where the majority of the sample was between 44 and 78 years of age, and drank regularly, which could reduce the generalizability of our findings. Future work in this area should include studies conducted in non-VA primary care clinics, in samples diverse enough to address generalizability across gender, age, race, and health care settings. Future studies should also include both drinkers and non-drinkers given the known relationship between alcohol and sleep.55
An additional important limitation is that PHQ item-3 assesses both insomnia and hypersomnia and we only validated it against an insomnia instrument. It remains to be seen to what extent false positives are related to endorsing the PHQ sleep item due to hypersomnia and whether these may represent true cases of hypersomnia requiring treatment. The latter is particularly important given that the most common sleep disorder with hypersomnia as a feature, sleep apnea, is a significant concern in primary care and associated with multiple health risks.56–58 If the PHQ item performs reasonably well in detecting cases of sleep apnea, it could prove to be a very efficient tool for identifying the two most common forms of sleep disorders (insomnia and sleep apnea).
Though not the focus of the paper, the significant negative relationship between age and sleep in this sample warrants further discussion. This finding is in contrast with a wealth of literature stating that, as people age, their sleep worsens.42,43 Alcohol intake did not account for this relationship in this sample, which is unexpected, given the close relationship between alcohol and sleep,55 and the fact that most people in our sample were regular drinkers. Rather, our post hoc analyses revealed that, once PTSD symptoms were added to the regression model, the significant relationship between age and sleep went away. This finding is not surprising given the significant correlation between PTSD symptoms and age in our sample and the literature demonstrating that PTSD symptoms disrupt normal sleep 44
Overall, this study is an important first step in the validation of the PHQ item-3 as an effective sleep screen for primary care. The PHQ item-3 is brief, and does not require an additional screen to primary care providers, who are already burdened with many annual screens. Particularly in settings already using the PHQ-9, this can be an incredibly efficient use of resources. Pulling out and focusing on the PHQ item-3 could help providers to more readily identify insomnia and potentially other sleep disturbances in their patients. Ultimately, identifying sleep disturbance is the gateway to interventions with demonstrated efficacy and to the common goal of improving their overall health and quality of life.
The views expressed in this article are those of the authors and do not reflect the official policy of the Veteran’s Affair’s department or other departments of the U.S. government. We acknowledge the Center for Integrated Healthcare, Syracuse VA Medical Center (VAMC) who contributed support to this research study.
This material is based upon work supported by the Department of Veterans Affairs, Veterans Health Administration, and the Center for Integrated Healthcare.
The material discussed in this paper has not been a part of any prior presentations.