|Home | About | Journals | Submit | Contact Us | Français|
Universal pain screening with a 0–10 pain intensity numeric rating scale (NRS) has been widely implemented in primary care.
To evaluate the accuracy of the NRS as a screening test to identify primary care patients with clinically important pain.
Prospective diagnostic accuracy study
275 adult clinic patients were enrolled from September 2005 to March 2006.
We operationalized clinically important pain using two alternate definitions: (1) pain that interferes with functioning (Brief Pain Inventory interference scale≥5) and (2) pain that motivates a physician visit (patient-reported reason for the visit).
22% of patients reported a pain symptom as the main reason for the visit. The most common pain locations were lower extremity (21%) and back/neck (18%). The area under the receiver operator characteristic curve for the NRS as a test for pain that interferes with functioning was 0.76, indicating fair accuracy. A pain screening NRS score of 1 was 69% sensitive (95% CI 60–78) for pain that interferes with functioning. Multilevel likelihood ratios for scores of 0, 1–3, 4–6, and 7–10 were 0.39 (0.29–0.53), 0.99 (0.38–2.60), 2.67 (1.56–4.57), and 5.60 (3.06–10.26), respectively. Results were similar when NRS scores were evaluated against the alternate definition of clinically important pain (pain that motivates a physician visit).
The most commonly used measure for pain screening may have only modest accuracy for identifying patients with clinically important pain in primary care. Further research is needed to evaluate whether pain screening improves patient outcomes in primary care.
Universal pain screening is an increasingly common practice, largely because of the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) requirement that accredited hospitals and clinics must routinely assess all patients for pain.1 Pain screening is intended to improve the quality of pain management by systematically identifying patients with pain in clinical settings.2
Pain symptoms are among the most common complaints in primary care and are diverse in terms of etiology, severity, and duration. Approximately 20% of primary care patients suffer from chronic pain;3 however, many pain symptoms in primary care are minor and transient.4 Universal screening in primary care would be useful if it accurately identified patients with clinically important pain who could potentially benefit from additional pain assessment and management.
JCAHO standards do not specify how pain should be assessed; rather, they allow organizations to develop their own pain assessment policies.5 The pain numeric rating scale (NRS), on which patients rate their current pain intensity from 0 (“no pain”) to 10 (“worst possible pain”), has become the most widely used instrument for pain screening. Although it was not developed or validated as a screening test, the NRS is ubiquitous as a screener in many health care environments. For example, its use is currently mandated throughout the Veterans Affairs health care system as part of its “Pain as the fifth vital sign” campaign.6
The NRS has potential advantages as a screening test. It is short, easy to administer, and has been validated as a measure of pain intensity in populations with known pain.7,8 However, no studies have evaluated its accuracy as a screening test to identify patients with clinically important pain. The U.S. Preventive Services Task Force (USPSTF) recommends that two criteria be met before a screening test is recommended for widespread use: (1) the test should be sufficiently accurate and capable of detecting a condition earlier than routine care and, (2) screening and early treatment should improve the likelihood of favorable patient outcomes.9 We designed this study to address the first criterion, that is, to evaluate the accuracy of the NRS as a screening test to identify primary care patients with clinically important pain.
We enrolled participants from September 2005 to March 2006 at the University of North Carolina (UNC) at Chapel Hill General Internal Medicine clinic. Adult patients presenting to clinic for a return visit were eligible to participate. We excluded patients who did not speak English because very few non-English speaking patients attend the clinic and because resource constraints prevented us from providing the survey in multiple languages. We obtained permission from physicians in the clinic to approach patients on their schedules and allowed physicians to opt out individual patients. Participants provided written informed consent. The UNC Biomedical Institutional Review Board approved the study protocol.
In accordance with UNC policy, patients are screened for current pain with the 0–10 NRS at the time of vital sign measurement. Patients then return to the waiting room or go to an examination room to wait for the physician. A research assistant approached patients during this waiting time and invited them to enroll in the study. Consecutive available patients were invited to participate during times when a research assistant was present in the clinic. To ensure adequate numbers of patients reporting pain, we oversampled those with pain screening NRS scores of≥1. Specifically, all eligible patients were invited to enroll until we enrolled enough patients with a pain screening score of zero to comprise approximately 20% of the target sample size. Thereafter, only patients with scores of≥1 were invited to participate.
To avoid alerting patients to the specific focus of the study, research assistants invited them to participate in a study of symptoms in primary care. After patients agreed to participate, a research assistant elicited the reasons for the visit.
Participants were asked to return to a designated room in clinic for the study interview immediately after completing their physician visit. Because previous work in the clinic has documented a high prevalence of low literacy,10 data were collected by face-to-face interview. One research assistant, a premedical student, conducted all interviews.
Nursing notes, dictated physician notes, and problem lists were abstracted from the electronic medical record after the interview was completed. Medical comorbidities were abstracted from the problem list using a list of 10 common conditions (arthritis, asthma/chronic lung disease, cancer, coronary artery disease, diabetes, heart failure, hypertension, liver disease, kidney disease, stroke).11
Pain screening NRS At the time of vital sign measurement, clinic staff members ask patients to rate the intensity of their current pain on a scale of 0 (“no pain”) to 10 (“worst possible pain”). Staff members then enter pain scores and other vital signs into the electronic medical record. Because this study was designed to evaluate the operating characteristics of pain screening in real practice, we did not retrain clinic staff or alter preexisting pain screening protocols. Based on previous studies and clinical practice, we categorized pain screening NRS scores as mild (1–3), moderate (4–6), or severe (7–10). Studies of chronic pain patients with different conditions have reached varying conclusions about the optimal cut points for mild, moderate, and severe pain on the 0–10 NRS, with 4 or 5 being the most commonly recommended lower limits for moderate pain and 7 or 8 for severe pain.12–17 We chose a score of 4 as the lower limit for moderate pain because it is most commonly accepted for clinical and administrative use.
Brief Pain Inventory (BPI) interference scale The BPI was administered during the study interview by a research assistant. We selected the BPI interference scale because it includes a generic measure of pain-related function that has been used in many types of pain conditions, is relatively short and easy to administer, and has been recommended as a core outcome measure for pain studies.18–20 The BPI measures pain-related functional impairment in 7 domains: general activity, mood, walking ability, normal work, relations with other people, sleep, and enjoyment of life. Possible scores for each domain range from 0 (“does not interfere”) to 10 (“interferes completely”). The overall BPI interference score is the mean of the 7 item scores. If one value was missing from a given scale, we imputed missing values using best subset regression. We did not impute values if more than one value was missing.The BPI includes an initial lead-in question about the presence of pain: “Throughout our lives, most of us have had pain from time to time (such as minor headaches, sprains, and toothaches). Have you had pain other than these everyday kinds of pain during the past week?” We were concerned that using this lead-in question alone might exclude too many patients from completing the full BPI. Therefore, we incorporated an additional question: “Have you had any pain at all, including minor aches or pains, during the past week?” Patients who answered “no” to both questions did not continue with the full BPI and were assigned a score of 0.
Reasons for the visit A research assistant asked two open-ended questions before patients saw the physician: (1) “what is the main reason for your visit today?” and (2) “what other concerns would you like to talk to the doctor about today?” Responses were classified as belonging to one of three categories: pain symptom, nonpain symptom, or other.
The purpose of this study was to evaluate the NRS as a screening test for clinically important pain in primary care. Because there is no gold standard definition of clinically important pain, we operationalized it using two alternate definitions: (1) pain that interferes with functioning and (2) pain that motivates a physician visit.
For our first definition (pain that interferes with functioning), we used a score of≥5 on the BPI interference scale as the primary reference standard. Although prior studies have used a BPI interference score of 4 or 5 as a cutoff,21,22 there is no consensus about what score constitutes substantial interference. Therefore, we used a score of≥5 (the midpoint) and conducted sensitivity analysis using different scores as alternative reference standards (BPI interference≥4; BPI interference≥3; any single BPI interference item≥5). For the second definition (pain that motivates a visit), we used our measure of reasons for the visit. Patients who reported a pain problem as the “main reason” or “other concern” were considered to have a pain reason for the visit.
Based on our conservative a priori assumption that 20% of the sample would have a BPI interference score≥5, we calculated that enrollment of 310 subjects would be necessary to estimate accuracy with a 95% confidence interval of ±0.10. An interim evaluation of data from the first 213 participants revealed that 46% had a BPI interference score≥5; therefore, we lowered our recruitment target to 275 participants.
We fit receiver operator characteristic (ROC) curves for the pain screening NRS compared to the reference standards for clinically important pain and calculated the area under the curve for each comparison. The area under the ROC curve is a measure of overall accuracy for tests with continuous results, where 0.5 indicates a worthless test and 1.0 a perfect (100% accurate) test.
We calculated sensitivity and specificity for dichotomized pain screening NRS cut points. We determined multilevel likelihood ratios for pain screening NRS categories of none (0), mild (1–3), moderate (4–6), and severe (7–10) using methods described by Peirce and Cornell.23 Likelihood ratios allow test results to be interpreted in different clinical populations.24,25 Multilevel, or stratum-specific, likelihood ratios are advocated for evaluation of diagnostic tests with continuous scores because they do not require scores to be forced into dichotomous positive/negative outcomes.23,26,27
All data analysis was conducted with sampling weights to adjust for oversampling of patients with pain screening NRS scores≥1. Stata Intercooled version 8.2 (Stata Corp., College Station, Tex) and Excel 2002 SP1 (Microsoft Corp., Redmond, Wash) were used.
Research assistants approached 548 patients with an invitation to participate. Of the 357 who initially agreed, 277 (78%) were interviewed, and 275 (77%) were included in the analysis (Fig. 1). One patient started but did not complete the interview. The remainder either declined to be interviewed after the physician visit or left the clinic without being interviewed. Those who gave a reason cited lack of time or the need to go elsewhere for further testing (e.g., laboratory or radiology). Patients who were not interviewed after initially agreeing to participate were more likely to have seen a resident physician but had similar pain scores, reasons for the visit, and demographics to completers.
The mean age of participants was 55 years (Table 1). They were socioeconomically diverse and had an average of 1.9 (95% CI 1.7–2.1) chronic medical conditions. Eighty percent saw the physician they considered to be their “regular doctor,” and 47% saw a resident physician.
Forty percent reported a pain symptom as a reason for the visit as either the main reason (22%) or a secondary concern (18%). Among participants with a pain screening NRS≥1, the mean score was 6.0 (95% CI 5.7–6.3). Most patients reported musculoskeletal pain; the most common primary pain locations were lower extremity (21%) and back/neck (18%). Two percent reported pain because of neuropathy. Fifty-five percent of the overall sample and 77% of patients with pain reported at least 1 pain symptom that had been persistent for 6 months or longer.
The percentage of participants with clinically important pain was 37% by the first definition (pain that interferes with functioning) and 40% by the second definition (pain that motivates a physician visit). Twenty-six percent met criteria for both definitions.
Accuracy for pain that interferes with functioning The area under the ROC curve for the pain screening NRS compared to the primary reference standard (BPI interference≥5) was 0.76, indicating fair accuracy. Results were not substantially different when the pain screening NRS was tested against alternative BPI interference thresholds (area under curve 0.76–0.77).The lowest possible cut point, a pain screening NRS score of 1, was 69% sensitive (95% CI 60–78) and 78% specific (95% CI 71–83) for functional interference. In other words, nearly a third of patients with pain-related functional interference had an NRS score of 0. The usual pain screening NRS cut point, 4, was slightly less sensitive (64%, 95% CI 54–72) with a specificity of 83% (95% CI 77–88).Multilevel likelihood ratios are shown in Table 2. The likelihood ratio for a score of 0 was less than 1, which reduces the probability of pain that interferes with functioning. Scores in the 1–3 range do not provide additional information because the likelihood ratio is near 1.0, whereas scores of 4–6 and 7–10 increase the probability of functional interference. For example, if the pretest probability of pain-related functional interference for a given clinic patient was 40%, the posttest probabilities corresponding to scores of 0, 1–3, 4–6, and 7–10 would be 21, 40, 64, and 79%, respectively.
Accuracy for pain that motivates a visit The area under the ROC curve for the pain screening NRS compared to the second reference standard (pain reason for the visit) was 0.78, again indicating fair accuracy. Twenty-one percent of patients who reported pain as the main reason for the visit, and 28% of those with any pain reason for the visit had a pain screening NRS score of zero. Accordingly, the sensitivity of a pain screening NRS score of 1 was 71% (95% CI 62–79), and the specificity was 81% (95% CI 74–86). A pain screening NRS score of 4 had a sensitivity of 63% (95% CI 54–72) and specificity of 85% (95% CI 79–90). Likelihood ratios were similar to those obtained using the first reference standard (Table 2).
We found that the pain screening NRS had only modest accuracy for identifying patients with clinically important pain in an academic primary care clinic. Even a pain screening cutoff score of 1 missed nearly a third of patients with clinically important pain.
Why did the pain screening NRS miss patients with clinically important pain? First, it is possible that this simple measure cannot be expected to identify all clinically important pain in primary care. Pain is a multidimensional experience, and this dimensionality has important implications for its measurement.8,28 In settings where pain is often chronic and complex, the simple pain screening NRS may fail to identify patients with pain-related suffering driven by functional limitations, illness worry, or other factors.
There are more focused potential explanations for the poor performance of the pain screening NRS, including the time frame and wording of the question. Because it focuses on current pain, the NRS might miss intermittent symptoms. In addition, we found that “pain” was not the preferred word for some patients to describe their subjective experience. For example, one participant reported difficulty answering the questions because “I feel great discomfort, but it is different than pain.”
To our knowledge, this is the first published study to evaluate the accuracy of pain screening in primary care. Our study has several strengths. First, we evaluated the most commonly used pain screening measure under real primary care clinical conditions. In addition, we collected detailed prospective information about pain and recruited a diverse group of primary care patients with a broad spectrum of pain and other medical problems.
This study also has several potential limitations. First, there is no well-established gold standard for clinically important pain. However, we believe our strategy for operationalizing clinically important pain is well supported by the available literature and clinical experience. Functional impairment has been used many times previously to classify pain severity.12–17 Pain as a reason for the visit was chosen as the second definition because of its patient-centeredness and clinical relevance. Whereas we recognize that not all clinically important pain will be included in either of these definitions, we believe a pain screening test should at least identify patients with pain that is functionally impairing or motivates a visit. Second, selection bias is a potential concern. Participants were enrolled when the research assistant was available. In addition, 34% of invited patients declined to participate, and 14% initially agreed but did not complete the interview. We do not believe these factors biased our results. Clinic procedures and staffing on study enrollment days did not differ from other days, and enrollment was not limited to certain times or days of the week. Patients who failed to complete the interview had similar demographic characteristics, pain scores, and reasons for the visit as completers. In addition, demographics of our study participants closely matched those of the clinic population. Finally, our findings are limited to a single academic Internal Medicine clinic; thus, our results cannot be generalized to all primary care settings.
To date, there is little empirical support for the hypothesis that routine assessment and documentation of pain will improve pain management in primary care. A recent retrospective study found no improvement in quality of pain care at a Veterans Affairs general medicine clinic after implementation of the “Pain as the 5th Vital Sign” campaign, which uses a pain intensity NRS to screen for pain.29
Universal pain screening may have substantial costs in primary care, where numerous acute, chronic, and preventive care priorities compete for limited physician and nursing time.30 For example, primary care physicians do not have time to complete even the preventive services that have been rigorously evaluated and recommended by the USPSTF.31 In this resource limited environment, mandated initiatives like universal pain assessment may have unintended effects on patient care, clinic efficiency, clinician and patient satisfaction, and medicolegal risk.
In conclusion, the practice of universal pain screening has become widespread despite a lack of published research evaluating the accuracy and effectiveness of pain screening strategies. Our results suggest that the most commonly used measure for pain screening may have only modest accuracy for identifying patients with clinically important pain in primary care. Further research is needed to determine whether pain screening improves patient outcomes in primary care.
We thank Kristen Rake and Halle Amick for data collection and management, Joanne Garrett and Bob Konrad for assistance with data analysis, and Mike Pignone for his insightful critique and suggestions.
Dr. Krebs was supported by the Robert Wood Johnson Clinical Scholars Program; Dr. Carey was supported in part by NIH grant 1RO1AR051970; and Dr. Weinberger was supported by an HSR&D Career Scientist Award from the Department of Veterans Affairs.
Conflict of Interest The authors have no conflict of interest to report. The Robert Wood Johnson Foundation provided funding for this study through the Clinical Scholars Program but had no role in study design, analysis, interpretation, or drafting of the manuscript.
Earlier versions of this work were presented at the 2005 national meeting of the Robert Wood Johnson Clinical Scholars Program and the 2006 national meeting of the Society of General Internal Medicine.