|Home | About | Journals | Submit | Contact Us | Français|
Symptoms suggestive of acute sinusitis are a common reason for patients to visit primary care providers. Since objective measures of outcome have not been shown to be related to patient reported outcomes, measures of treatment success have focused on symptom relief and improved health-related quality of life (HRQL). Assessing the appropriate role of treatment — for example, antibiotics for patients with acute sinusitis — requires valid, reliable, and responsive measures of outcome. We identified symptom scores and HRQL instruments for adults with sinusitis and assessed their performance characteristics.
Articles identified through computer searches of the medline, premedline, and embase databases, the Cochrane Library, and internet documents; inquiries to experts in sinusitis and outcomes assessment; and review of reference lists.
Studies that used HRQL instruments or evaluated the performance characteristics of symptom scores in adults with sinusitis, published in English after 1966.
Two reviewers independently extracted data on study design, setting, and patient characteristics; instrument length and format; and instrument validity, reliability, responsiveness to change, and interpretability. Study quality was assessed using a 10-point score.
Of 1,340 articles in the original search, 29 articles using 16 HRQL instruments and 5 symptoms scores met inclusion and exclusion criteria. The overall quality of these studies was low; only 4 studies scored higher than 4 of 10 points. Four studies included patients with acute sinusitis, but only 2 included exclusively acute sinusitis patients. Three instruments have been shown to meet basic requirements for validity, reliability, and responsiveness: the Chronic Sinusitis Survey, the Rhinosinusitis Outcome Measure-31, and the Sinonasal Outcome Test-16. No instrument has been validated in a primary care setting or for patients with acute sinusitis.
Few validated measures of sinusitis-specific HRQL are available. The 3 instruments shown to be valid, reliable, and responsive have been assessed in patients with chronic sinusitis. No measure has been validated in primary care settings or for patients with acute sinusitis. A lack of valid, responsive outcome measures may limit current treatment recommendations for patients with acute sinusitis.
Acute upper respiratory infections including sinusitis are the most common reason for symptomatic ambulatory care visits to physicians.1 Sinusitis accounts for 12 to 17 million annual visits to physicians in the United States2,3 and for 12% of antibiotics prescribed to adults in the United States.4
In the assessment of patients with symptoms of acute sinusitis, objective outcome measures, for example, radiographs or microbiologic studies, are of limited value. Diagnostically, plain x-rays and computed tomography have poor specificity for bacterial sinusitis5,6 and bacteriological evaluation in routine clinical care is not practical. Additionally, baseline severity on computed tomography is poorly predictive of the resolution of symptoms.7 Since objective measures of disease severity and outcome are not generally helpful in the management of sinusitis, clinicians and researchers have developed instruments to measure patient symptoms and health-related quality of life (HRQL).
Health-related quality of life has been defined as the component of overall quality of life that is determined primarily by the person's health and that can be influenced by clinical interventions.8 HRQL is self-determined and is comprised of health status, functional status, and overall quality of life.9 Included within these broad categories are physical health, psychological health, physical functioning, social functioning, role functioning, and general well-being.10,11
Health-related quality of life instruments have several advantages over symptom scores. First, HRQL instruments translate symptoms into broader concerns that are important to patients. Second, HRQL instruments measure the impact of symptoms, regardless of what specific symptoms are present. Third, generic HRQL instruments allow comparison across diagnoses, whereas symptom scores are limited to a single target condition.
In order to be useful, HRQL instruments and symptom scores need to have good psychometric performance characteristics.12 These performance characteristics include validity, reliability, and responsiveness to change.13,14 An instrument is valid if it measures what it is purported to measure; reliable if repeated measures under the same conditions give similar results; and responsive if an instrument can detect important changes in the condition, even if those changes are small. Furthermore, changes should be interpretable, so that differences in the score of an instrument can be understood and assigned qualitative meaning.15,16
Even with excellent performance characteristics, symptom scores and HRQL measures are not likely to predict whether a patient with symptoms of acute sinusitis has bacterial or viral sinusitis. Nevertheless, such instruments may be useful for identifying individuals with severe symptoms or decreased HRQL for whom empiric antibiotic treatment is reasonable. Symptom scores and HRQL measures should serve also as the primary outcome when evaluating the efficacy of therapy.6
We are unaware of a review that identifies and compares the performance characteristics of HRQL instruments or symptom scores for adults with acute sinusitis. To assess the performance characteristics of available HRQL instruments and symptom scores for adults with acute sinusitis, we performed a systematic review of the medical literature from 1966 to the present.
We searched medline, premedline, embase, and the Cochrane Library from January 1966 to March 2002 to identify articles about HRQL instruments and symptom scores for patients with rhinosinusitis. We used the subject headings sinusitis and rhinitis, and the text word rhinosinusitis. These terms were combined with the subject headings quality of life, health status, health status indicators, sickness impact profile, severity of illness index, outcome assessment (health care), activities of daily living, and questionnaires. For each subject heading, we included all subheadings and qualifiers. We also searched textbooks on HRQL and consulted experts in primary care, otorhinolaryngology, and quality of life measurement to identify additional instruments, articles, and scientific abstracts. We searched the listings of HRQL instruments on the World Wide Web for those regarding sinusitis.
Titles of articles were screened for suitability, and abstracts deemed appropriate were retrieved. Abstracts were reviewed independently by 2 authors. We retrieved articles that either reviewer felt met inclusion criteria. References of retrieved articles were examined to identify additional studies.
Studies were included if they (1) used HRQL instruments in adults with sinusitis, or (2) evaluated the performance characteristics of symptom scores for adults with sinusitis. We included studies of patients with either acute or chronic sinusitis because we anticipated instruments might be useful for either condition. We conceived of an HRQL instrument broadly as a questionnaire that included a non-symptom-related, but potentially health-related domain. We included only English-language articles.
We abstracted information directly into a database specifically designed for this project. Information was abstracted on the study goal, study type, clinical setting, method of diagnosis, treatments provided, patient populations, patient characteristics, and follow-up. We evaluated how well quality of life was defined and assessed in each study using 10 yes/no questions proposed by Gill and Feinstein for the evaluation of quality of life studies.17 Each question was given 1 point for a scale that ranged from 0 to 10, representing the sum of questions answered affirmatively.
For each HRQL or symptom measure used in the study, we abstracted information on the number of items, response formats, subscales or domains, score derivation, and respondent burden. We abstracted information on the performance characteristics of each HRQL measure and symptom score, including validity, reliability, responsiveness, and interpretability.
Validity assessment of each instrument included evaluation of content validity, criterion validity, and construct validity. Content validity was the appropriateness of an instrument for a particular task and was generally assessed by having experts and/or patients with the target condition review an instrument. Criterion validity was the comparison of a novel instrument to an established gold standard. Because there was no practical criterion standard for the diagnosis of sinusitis, for purposes of this review, we considered either radiographic or microbiologic assessment to be criterion standards. Construct validity was demonstrated when an instrument behaved in accordance with underlying theories. We assessed 2 aspects of construct validity: whether the measure was compared to other measures of a similar construct (convergent validity) and whether the measure was able to differentiate between persons with dissimilar conditions (discriminant validity).14
To evaluate reliability, we determined whether the internal consistency and test-retest reliability of an instrument was assessed. Internal consistency was most commonly measured using Cronbach's α, a form of split-half reliability.18 Test-retest reliability was determined by administering a measure to patients with unchanged clinical condition and comparing initial and follow-up scores using a correlation coefficient.
Responsiveness was examined to see if change, most commonly evaluated using effect size or standardized response mean, was calculated for each instrument.19,20 If standardized response mean was not reported, we attempted to calculate it using reported P values and the number of subjects. Because P values may have been reported using the “<” or “≤” sign, actual standardized response means could be higher than that derived from such a P value. Therefore, calculated standardized response means are indicated by the words “at least” in Table 2.
Finally, interpretability and determination of the minimally important clinical difference were assessed by comparing changes in the score of an instrument to patient assessment of the change in their overall status.16
All abstraction was done independently by 2 authors. Disagreements over inclusion were resolved through review and discussion. Calculation of κ was done using SAS 8.1 (SAS Institute, Inc., Cary, NC).
This study was funded in part by Aventis Pharmaceuticals through a Quality Care Research Fund award. The study sponsor had no role in the study design; in the collection, analysis, or interpretation of data; in the writing of the report; or in the decision to submit the report for publication.
We identified 1,340 articles, with all but 6 articles identified by electronic database retrieval (Fig. 1). Based on title or non-English language, 976 articles were excluded. Abstracts of the remaining 364 articles were reviewed by 2 reviewers. Evaluator agreement on study inclusion was moderate to excellent (κ = 0.80; 95% confidence interval, 0.72 to 0.88). Most abstracts were excluded for not involving patients with rhinosinusitis, not using an HRQL measure, or not evaluating a symptom score. Twenty-nine studies including 16 HRQL instruments and 5 symptom scores met our inclusion and exclusion criteria.21–49 One article by Stewart et al.38 contained information previously published in an article by Bhattacharyya et al.50 These 2 articles are considered a single study. We found no unpublished data for inclusion.
Among the 29 studies that met our inclusion and exclusion criteria, study designs included 12 prospective cohort studies,21,23,25,26,30,32,34,36,37,39,42,48 6 cross-sectional studies,28,29,33,35,38,49 6 randomized interventions,24,27,31,40,41,45 3 retrospective studies,22,44,46 1 nonrandomized, comparative study,43 and 1 prospective intervention.47
Studies included a variety of interventions. Thirteen studies included surgical interventions,22,25,26,30,32,34,36,37,39,42,44,46,48 9 studies included primarily medical interventions,21,24,27,31,40,41,43,45,47 1 study included both medical and surgical interventions,23 and 6 were observational.28,29,33,35,38,49
Among the randomized interventions, 4 were placebo controlled. One of the placebo-controlled trials was of nasal irrigation,24 1 was of recombinant human granulocyte colony stimulating factor,27 1 was of intranasal antibiotics,31 and 1 was of intranasal corticosteroids.41 There were no placebo-controlled trials of antibiotics that met our inclusion criteria. The remaining 2 randomized interventions were comparative trials between 2 antibiotics.40,45
Description of patient characteristics varied across studies in terms of what information was reported. Among the 25 of 29 studies that reported subject age, mean subject age ranged from 34 to 53 years old.21–33,35–42,45–47,49 In the 25 studies that reported patient gender, the percentage of women ranged from 0% to 78% of subjects.21–42,45,47,49 Only 5 of 29 studies included a description of the racial or ethnic composition of subjects.21,38,40,41,45 In these 5 studies, whites made up between 69% and 91% of subjects and African Americans 7% to 10%.
Almost all studies identified study participants from specialty settings. Twenty-five studies were performed in ear, nose, and throat (ENT) clinics or among patients undergoing ENT surgery.21–23,25–39,42–44,46–49 One study did not state the setting.40 One study had a mix of patients from ENT and primary care offices.41 One study drew patients from the community using newspaper ads.24 Only 1 study was based exclusively in a primary care setting.45
In terms of subject diagnosis, 13 studies were in patients with exclusively chronic sinusitis.24–27,31–33,35,37,39,47–49 Eight studies were performed in patients with a combination of ENT diagnoses.21–23,34,38,43,44,46 Four studies did not clearly state the patient diagnosis. Three of these studies were in patients undergoing sinus surgery,30,36,42 and 1 study included patients with “rhinosinusitis or rhinitis.”29 Two studies had a mix of patients with chronic sinusitis and acute sinusitis.28,41 Only 2 studies included exclusively patients with acute sinusitis.40,45
The overall quality of studies was low as assessed by the questions proposed by Gill and Feinstein. Out of 10 possible points, scores ranged from 1 to 8 points, with only 4 studies scoring higher than 4 points.21–24 There was no clear relationship between the quality score and the study type, setting, or intervention.
Characteristics of the 16 HRQL instruments and 5 symptom scores are shown in Table 1. The number of items ranged from 3 to 62. Most instruments were self-administered either in the office or at home and consisted of questions with categorical response frames. Respondent burden was not stated for 17 instruments. For the remaining 4 instruments, respondent burden ranged from 2 to 20 minutes. Ten instruments had subscales or subdomains that generally included symptoms, emotional well-being, physical functioning, and medication use.
Performance characteristics for the 16 HRQL instruments and 5 symptom scores are summarized in Table 2, in order of decreasing quality scores of the original articles. Although questions in this quality score address the conceptual framework of an instrument, the quality score also appeared to subjectively relate to assessment of content validity, reliability, responsiveness, and interpretability.
Overall, evidence was generally lacking regarding the performance characteristics of the HRQL instruments and symptom scores. Among the 21 measures, 2 had formal evaluations of at least some component of validity, reliability, responsiveness to change, and interpretability (the Rhinosinusitis Outcome Measure-31 and the Sinonasal Outcomes Test-16). Another 2 measures had evaluation of 3 of 4 components (the Chronic Sinusitis Survey – Duration-based form and the Chronic Sinusitis Survey – Severity-based form). All other measures had been evaluated for 2 or fewer of these components. The most common performance characteristic evaluated in 14 measures was construct validity, either convergent (11 measures) or discriminant (9 measures). Most commonly absent were assessment of interpretability (reported for only 2 measures), reliability (5 measures), and content validity (6 measures). Responsiveness to change was reported in 10 measures, but only 4 measures had reported it in terms of standardized response mean or effect size.
Four measures of outcome have had basic aspects of validity, reliability, and responsiveness to change assessed. Of these, each has been studied in patients with chronic sinusitis, and 3 of these measures have been shown to have acceptable performance characteristics: the Chronic Sinusitis Survey – Duration-based form, the Rhinosinusitis Outcome Measure-31, and the Sinonasal Outcomes Test-16. A fourth measure of outcome, the Chronic Sinusitis Survey – Severity-based form, has undergone detailed psychometric evaluation, but has been found to have poor convergent validity with subscales from the Short Form-36 and to be less reliable than the Duration-based form. As a result, the developers of the Chronic Sinusitis Survey recommend use of the Duration-based form.32
Of these 3 validated measures, 1 has reported a Cronbach's α of more than 0.9, and another reported an α of 0.89, indicating that they have sufficient reliability to assess response over time among individuals. Finally, for the Chronic Sinusitis Survey – Duration-based form and the Sinonasal Outcomes Test-16, the magnitude of the response to change within individuals over time appears adequate for surgically treated patients, but may be lower for medically treated patients.
No measure has been adequately evaluated in patients with acute sinusitis. Among the performance characteristics examined, only content validity has been partially demonstrated for 2 instruments in patients with acute sinusitis. First, Witsell et al. performed a multicenter, observational study of patients in ENT clinics using the Medical Outcomes Trust Short Form-12 to measure HRQL.28 Two summary scores from the Short Form-12, the Physical Component Summary and the Mental Component Summary scores, appeared to be lower in patients with acute sinusitis compared to published control populations, but no statistical comparison was reported.28 Second, Dolor et al. performed a randomized, placebo-controlled trial of intranasal fluticasone in ENT and primary care patients with both chronic sinusitis and recurrent acute sinusitis.41 Sinonasal Outcomes Test-20 scores appeared to improve in concordance with the Physical and Mental Component Summary scores of the Short Form-12. No statistical test was done to formally compare these 2 instruments, and results were not stratified based on diagnosis or setting, making it difficult to draw conclusions specifically about patients with recurrent acute sinusitis. No other performance characteristics have been described for any other measure of outcome for patients with acute sinusitis.
In our systematic review of measures of outcome for patients with sinusitis, we identified 3 HRQL instruments that have acceptable performance characteristics for patients with chronic sinusitis. The responsiveness of these instruments appears adequate in surgically treated patients, but may be inadequate for patients treated medically. For patients with acute sinusitis, no measure of outcome has met even minimal validation requirements. Finally, no performance characteristics have been determined for any measure of outcome for use in nonspecialty, primary care settings.
A lack of validated measures of outcome for acute sinusitis may limit current treatment recommendations. Two recent meta-analyses of antibiotic use for patients with acute sinusitis found a marginal benefit to antibiotics.6,51 The 5 placebo-controlled studies included in these meta-analyses used heterogeneous outcome measures that included a 4-point system combining x-ray evaluation and patient information,52 a 3-point global clinical rating,53 a 5-point patient self-assessment,54 the absence of symptoms at 2 weeks,55 or use of the McGill-Melzack Pain Questionnaire.56 None of the measures reported in these studies have been validated for use in patients with acute sinusitis. Because the measures of outcome are unvalidated, antibiotics for acute sinusitis may actually be more or less effective than previously reported.
Given the subjective nature of outcomes for patients with sinusitis, it is possible that the marginal benefit identified with antibiotics is due to the use of measures with poor responsiveness. Antibiotics for acute sinusitis could be more effective than previously reported if outcome measures that were more responsive to change were available.
Virtually all patients with acute sinusitis would be expected to return to their baseline status within a relatively short period of time, approximately 2 weeks.57 If a measure of outcome cannot detect such rapid changes in clinical status, then important differences between treatment groups could be missed when such differences truly exist. Several of the placebo-controlled trials of antibiotics for acute sinusitis used outcome assessment at a single point in time. Such cross sectional assessment may fail to detect divergence between groups occurring at points between baseline and follow-up, giving false-negative results.
Conversely, antibiotics for acute sinusitis may be less effective than previously reported. For example, the unvalidated sinusitis symptom scores that have been used may be too narrow in scope. If sinusitis-specific symptoms improve, but not to the point where a patient is able to return to work, or if more general symptoms persist (e.g., fatigue), a symptom score would show improvement where a HRQL measure would not. Similarly, if a patient's symptoms resolve, but the patient is unable to return to work for a treatment-related reason (e.g., diarrhea from antibiotics), again, a symptom score would show improvement where a HRQL measure would not. Use of a simple symptom score might bias results in favor of treatment.
This review has several limitations that should be acknowledged. There is no standard instrument to assess the quality of studies that evaluate subjective patient-reported outcome measures. We used a series of 10 questions proposed by Gill and Feinstein to evaluate the rigor with which quality of life was defined and assessed in each study.17 By this metric, we found that studies evaluating sinusitis measures were of generally low quality. These low scores may reflect the original authors' lack of familiarity with quality of life or HRQL as outcome measures. Additionally, most of the articles examined were primarily focused on evaluating a procedure or treatment rather than quality of life itself. Additionally, some authors may have believed that symptoms rather than HRQL are the preeminent outcome in patients with sinusitis. However, Witsell et al.,28 using the Short Form-12, show that patients with acute sinusitis have measurable decrements in HRQL, arguing that even a simple, generic HRQL measure can detect changes in health status for patients with acute sinusitis.
For conditions like acute sinusitis that lack convenient objective clinical measures of outcome, there is a need for patient-reported subjective measures that integrate symptoms and their impact on HRQL. Such a measure of outcome should be valid, reliable, and have excellent responsiveness to change characteristics. The results of such a measure should be interpretable, so that changes observed by researchers and clinicians can be used to inform patients about the expected magnitude of impact of a proposed therapeutic intervention. Studies validating such a measure of outcome for adults with acute sinusitis should be performed in a variety of settings, including primary care, with well-defined patient populations. Only with a validated, interpretable measure of outcome for patients with acute sinusitis can well-founded conclusions be made about the efficacy of specific treatments.
This study was supported by a grant from Aventis Pharmaceuticals through a Quality Care Research Fund award. Dr. Linder was supported in part by National Research Service Award 5T32PE11001-12.