PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptNIH Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Arch Clin Neuropsychol. Author manuscript; available in PMC Nov 17, 2011.
Published in final edited form as:
PMCID: PMC3219058
NIHMSID: NIHMS335860

Neuropsychological dysfunction in patients with end-stage pulmonary disease: lung transplant evaluation

Abstract

There has been a relative absence of studies that have examined the neuropsychological profiles of potential lung transplant candidates. Neuropsychological data are presented for 134 patients with end-stage pulmonary disease who were being evaluated as potential candidates for lung transplantation. Neuropsychological test results indicated that a significantly greater proportion of the patients exhibited impaired performances on a number of Selective Reminding Test (SRT) tasks as compared to the expected population frequency distributions for these measures. The highest frequencies of impairment were observed on the SRT’s Immediate Free Recall (46.43%), Long-term Retrieval (41.67%), and Consistent Long-term Retrieval (51.19%) variables. On the Minnesota Multiphasic Personality Inventory-2 (MMPI-2)/Minnesota Multiphasic Personality Inventory-Adolescent (MMPI-A), patients’ mean clinical profile revealed elevations on Scales 1 (Hypochondriasis) and 3 (Conversion Hysteria). This profile indicated that they were experiencing an array of symptomatology ranging from somatic complaints to lethargy and fatigue, and that they may have been functioning at a reduced level of efficiency. Findings are discussed in light of patients’ end-stage pulmonary disease and factors possibly contributing to their neuropsychological test performances. Implications for clinical practice and future research are also provided.

Keywords: Neuropsychology, Neurocognitive, Pulmonary disease, End-stage, Lung transplant

1. Introduction

Within the past 20 years, there has been a relative paucity of studies that have examined the neuropsychological profiles of patients with chronic pulmonary disease and related levels of hypoxemia. The majority of the studies that have been conducted have involved patients with chronic obstructive pulmonary disease (COPD). Overall, individuals in these studies have been found to display a diversity of deficient neurocognitive performances (as compared to controls in most studies) on tasks assessing flexible thought (Grant, Heaton, McSweeny, Adams, & Timms, 1982; Prigatano, Parsons, Wright, Levin, & Hawryluk, 1983), attention (Stuss, Peterkin, Guzman, Guzman, & Troyer, 1997), abstraction/reasoning abilities (Grant et al., 1982; Prigatano et al., 1983), perceptual–motor integration (Grant et al., 1982), simple motor skills (Grant et al., 1982), memory (Borak, Sliwinski, Tobiasz, Gorecka, & Zielinski, 1996; Grant et al., 1982; Prigatano et al., 1983; Stuss et al., 1997), and speed of performance/processing (Prigatano et al., 1983; Stuss et al., 1997).

In a large study that merged the neuropsychological data from two investigations involving COPD patients with mild (Prigatano et al., 1983) and advanced (Grant et al., 1982) hypoxemia, Grant et al. (1987) found that as patients’ levels of hypoxemia increased, they tended to display progressively poorer scores on neuropsychological measures assessing three factors obtained via a factor analysis: (1) perceptual learning–problem solving; (2) alertness–psychomotor speed; and (3) simple motor functions. Similarly, Stuss et al. (1997) found evidence of increasing neuropsychological deficits with more severe hypoxia in their sample of 18 patients with COPD.

The few published reports that have examined the psychological profiles of individuals with pulmonary disease via the Minnesota Multiphasic Personality Inventory (MMPI) have also typically involved patients with COPD. For example, McSweeny, Grant, Heaton, Adams, and Timms (1982) administered the MMPI to 203 patients with hypoxemic COPD and 73 controls. According to the authors, the primary “emotional disturbance” reported on this inventory was reactive depression, followed by a generalized dissatisfaction with life and preoccupation with somatic complaints. Prigatano, Wright, and Levin (1984) also administered the MMPI to 100 patients with mild hypoxemia and COPD. As a group, these patients exhibited significant clinical elevations on the Hypochondriasis, Depression, and Hysteria scales and slight elevations on the Psychasthenia and Schizophrenia scales, with a reactive depression as the most common pattern of findings for these patients.

There has also been a relative absence of investigations that have examined patients with end-stage pulmonary disease. Our research group has reported neuropsychological prevalence data for patients with end-stage cystic fibrosis (n = 18; Crews, Jefferson, Broshek, Barth, & Robbins, 2000), alpha-1-antitrypsin deficiency (n = 4; Jefferson, Crews, Broshek, Barth, & Robbins, in press), and COPD (n = 47; Crews et al., 2001). Overall, these investigations have revealed a diversity of neurocognitive impairments in these patients including deficient speed of processing abilities, shift of set and maintenance of response strategies, and auditory–verbal and visual memory. The most prevalent deficits identified (in more than 20% of the individuals tested) in these studies were on measures included in the Selective Reminding Test (SRT) assessing immediate free recall, long-term storage and retrieval strategies, consistent long-term retrieval, and cued and delayed free recall of noncontextual verbal material, as well as elevated numbers of intrusion errors.

The Minnesota Multiphasic Personality Inventory-2 (MMPI-2)/Minnesota Multiphasic Personality Inventory-Adolescent (MMPI-A) profiles of the patients in these “end-stage” studies (Crews et al., 2000, 2001; Jefferson et al., in press) revealed that the most consistent clinical elevations were displayed on Scales 1 (Hypochondriasis) and 3 (Conversion Hysteria), followed by Scale 2 (Depression). Thus, these patients tended to acknowledge a diversity of somatic complaints that worsened during times of stress. They were likely functioning at a reduced level of efficiency and complained of feeling tired and lethargic. Depressive symptomatology was also reported by many of the patients.

Since completion of these end-stage studies, data have been collected on 65 additional patients diagnosed with a variety of end-stage pulmonary diseases (see Patients for a listing of specific diagnoses). The purpose of this investigation was to consolidate the data from our previous end-stage studies with these 65 new cases and examine the neuropsychological profiles associated with end-stage pulmonary disease (regardless of the specific pulmonary diagnosis) in a relatively large sample of patients who were potential lung transplant candidates.

2. Method

2.1. Patients

Patients consisted of 59 males and 75 females with end-stage pulmonary disease who had been referred for neuropsychological evaluations as part of the lung transplant screening protocol at a large, tertiary care medical center. Pulmonary disease diagnoses included: cystic fibrosis, COPD (e.g., emphysema, chronic bronchitis, alpha-1-antitrypsin deficiency), idiopathic and interstitial pulmonary fibrosis, idiopathic pulmonary failure, sarcoidosis, and pulmonary hypertension. The physiological/medical criteria that operationally defined a patient as suffering from end-stage pulmonary disease and in need of a lung transplant included either a forced expiratory volume (FEV1) <25% predicted or a paO2 <60mmHg on room air. Individuals who had a paO2 <55mmHgwere given supplemental oxygen therapy in an effort to reverse any hypoxia. Patients’ histories were unremarkable for active, or clinically significant, neurological disorders/diseases, traumatic brain injuries, learning disabilities, or substance abuse. Patients ranged in age from 14 to 69 years with a mean age of 48.27 (S.D. = 13.96) years. Additional patient demographic data are provided in Table 1.

Table 1
Demographic characteristics and frequencies of neuropsychological test impairments of patients with end-stage pulmonary disease

2.2. Procedures

To evaluate the cognitive and behavioral functioning of patients, series of neuropsychological tests were administered. Neuropsychological measures were selected, which have been associated with a diversity of functions (intelligence, executive control/cognitive flexibility, memory/learning, psychological well-being) that were hypothesized to promote compliance with often complicated, and changing, pre- and post-lung transplant treatment regimens and, ultimately, overall health and well-being. Both verbal and visual memory measures were included in this battery in an effort to identify patients’ most efficient/effective mode(s) of learning/recall so that these strategies could be utilized to maximize their compliance with their treatment regimens. The neuropsychological battery was also designed to be relatively brief since the individuals assessed were suffering from end-stage pulmonary disease and, thus, were often unable to tolerate extensive batteries, or were limited by time constraints (e.g., by other lung transplant-related medical appointments, travel back to their hometowns, etc.).

All testing procedures were conducted by experienced neuropsychological test technicians under the supervision of licensed clinical neuropsychologists. The standardized administration and scoring procedures for each test were strictly followed. After brief rapport-building sessions and clinical interviews, the following battery was administered: portions of the Wechsler Adult Intelligence Scale—Revised (WAIS-R; Wechsler, 1981) or the Wechsler Intelligence Scale for Children—III (Wechsler, 1991); Trail Making Test (TMT, Parts A and B; Reitan, 1979; Reitan & Wolfson, 1993);Wisconsin Card Sorting Test (WCST; Heaton, Chelune, Talley, Kay, & Curtiss, 1993); portions of the Wechsler Memory Scale—Revised/III (WMS-R/III; Wechsler, 1987, 1997); the SRT (Buschke & Fuld, 1974) or the Rey Auditory Verbal Learning Test (RAVLT; Rey, 1964); and the MMPI-2 (Hathaway & McKinley, 1989) or the MMPI-A (Butcher, Graham, Williams, & Kaemmer, 1992). Although the majority of patients in this study completed the entire battery of measures, partial batteries were completed on individuals who were either too ill to tolerate the entire battery or who were limited by time constraints.

To interpret each individual’s neuropsychological test performances, scores were compared to published normative data for each test, while taking into consideration such demographic variables as gender, age, and educational level. Normative data and/or interpretations were obtained from the following sources: Graham (1993); Greene (1991); Heaton et al. (1993); Heaton, Grant, and Matthews (1991); Spreen and Strauss (1998); and Wechsler (1981, 1987, 1991, 1997). Neuropsychological test scores that fell more than one standard deviation below their respective normative means were judged to be impaired.

3. Results

The demographic characteristics and frequencies of neuropsychological test impairments for patients are provided in Table 1.

On WAIS-R/WISC-III estimates of intellectual functioning (i.e., Vocabulary, Similarities, and Block Design subtests), patients’ mean age-corrected scaled scores fell within the average range.

To investigate whether the current sample exhibited greater-than-expected frequencies of cognitive impairment on any of the neuropsychological measures, patients’ impairment rates were compared, via separate Tests for Significant Differences Between Two Proportions, to the normalized population frequency distribution of each measure with a similar (to the current study) criterion for impairment defined as scores at or below one standard deviation below the mean (i.e., frequency of impairment = 16%). Results of these analyses revealed significantly greater-than-expected frequencies of cognitive impairment on the SRT variables of Immediate Free Recall (z = 4.59, P < .001), Long-term Retrieval (z = 3.87, P < .001), Long-term Storage (z = 2.06, P < .04), Consistent Long-term Retrieval (z = 5.19, P < .001), Intrusion errors (z = 2.22, P < .03), and Delayed Free Recall (z = 2.89, P < .004). Alternatively, lower-than-expected frequencies of impairment were found on the SRT variables of Short-term Recall (z = −2.41, P < .02) and Random Long-term Retrieval (z = −2.76, P < .006). No additional significant differences were found for any of the remaining neuropsychological measures utilized with this sample of patients.

The mean MMPI-2/A profile for these patients with end-stage pulmonary disease is provided in Figure 1. The mean T scores for all three validity scales were within normal limits, which suggested a valid mean clinical profile. Regarding their MMPI-2/A clinical scales, patients’ mean code type was found to be a 1–3 (i.e., clinically elevated mean T scores of 67.56 and 66.56, respectively). All other mean clinical scale T scores for this sample of patients were within normal limits (i.e., T scores < 65).

Fig. 1
Mean MMPI-2/A profile for patients with end-stage pulmonary disease.

4. Discussion

The present study examined the neuropsychological profiles of 134 patients with end-stage pulmonary disease who were being evaluated as potential candidates for lung transplantation. The primary findings from this study indicated that the frequencies of impairment exhibited by the patient group on 8 out of 10 of the SRT variables differed significantly from the expected normalized population frequency distributions for these measures when one standard deviation below the mean was utilized as the criterion for impaired cognitive performances. However, when a Bonferroni-corrected alpha level (Winer, 1971; .05/25 = .002) was used to correct for multiple comparisons, only three of the SRT variables’ frequencies of impairment remained significantly greater than the expected population frequency distributions. These SRT variables included: Immediate Free Recall, Long-term Retrieval, and Consistent Long-term Retrieval. Furthermore, as noted in Table 1, 46.43% of these patients with end-stage pulmonary disease exhibited impaired immediate recall of a list of “selectively presented,” unrelated words over successive (up to 12) trials, while 41.67% of these individuals displayed difficulties retrieving these words from memory, and 51.19% experienced impairments of their consistent/ uninterrupted retrieval of these words over trials.

These findings appear generally consistent with those obtained in smaller studies conducted by the present authors of patients with end-stage cystic fibrosis (Crews et al., 2000), alpha-1-antitrypsin deficiency (Jefferson et al., in press), and COPD (Crews et al., 2001). The similarities between these past studies and the present one were not overly surprising given that the data (on a total of 69 patients) from our previous investigations were consolidated with data from 65 new cases to form the current patient sample.

In contrast to the frequencies of impairment exhibited by patients on the SRT Immediate Free Recall, Long-term Retrieval, and Consistent Long-term Retrieval variables, patients displayed lower rates of impairments on tasks assessing immediate (and delayed) free recall of contextual verbal material (i.e., WMS-R/III Logical Memory I and II subtests). These findings suggest more intact immediate (and delayed) free recall of contextual versus noncontextual/unrelated verbal material in this sample of patients with end-stage pulmonary disease.

The patients in this study also exhibited lower rates of impairment on subtests (i.e., WMS-R/III Visual Reproduction I and II) assessing immediate and delayed free recall of visual/figural material as compared to their rates of impairment on the SRT immediate and delayed free recall measures. These findings are suggestive of relatively more intact immediate (and delayed) visual/figural memory in this sample of patients as compared to their memory for noncontextual verbal material. It should also be noted that almost twice as many patients exhibited difficulties recalling visual/figural designs after 30 min (i.e., delayed recall) as compared to their recall immediately after presentation.

On another measure involving noncontextual verbal material (i.e., RAVLT), patients displayed lower frequencies of impaired immediate and delayed free recall as compared to their immediate and delayed memory performances on the SRT. This finding suggested that the SRT, as compared to the RAVLT, may be a more sensitive index of impaired immediate and delayed free recall of noncontextual verbal material in patients with end-stage pulmonary disease.

Although approximately 8–20% of the patients in the present study displayed impairments on measures (i.e., Trail Making Test, WCST) purported to be sensitive to executive functioning, no significant differences were found between the frequencies of impairment exhibited by this sample on these measures and the expected population frequency distributions, when one standard deviation below the mean was utilized as the criterion of impaired cognitive performance. Thus, the number of patients who displayed impairments of their cognitive processing speed, visuomotor scanning abilities, and cognitive flexibility/sequencing skills on the Trail Making Test was not significantly greater than would be expected in the “normal” population frequency distributions. Similarly, the proportions of the current sample that exhibited impaired maintenance of response strategies and concept formation/reasoning skills were also not significantly different from the expected population frequency distributions for the WCST.

On the MMPI-2/A, the mean profile of the patients with end-stage pulmonary disease revealed significant elevations on Scales 1 (Hypochondriasis) and 3 (Conversion Hysteria). The group’s mean MMPI-2/A clinical profile suggested that they acknowledged a diversity of somatic symptoms that likely worsened during times of stress (Graham, 1993; Greene, 1991). They also endorsed items suggestive of a reduced level of efficiency and complained of feeling weak, tired, and lethargic. Taken together, such profiles were not unexpected in patients who often experience multiple medical/physical complications secondary to their end-stage pulmonary disease. Individuals with these profiles may also be somewhat dependent and feel as if life is treating them unfairly (Greene, 1991). They may utilize denial and repression in an attempt to cope and lack insight that psychological distress may be contributing to their difficulties (Graham, 1993; Greene, 1991). In contrast, these patients may be somewhat overcontrolled and present themselves as responsible and conforming (Graham, 1993; Greene, 1991). It should also be noted that the current sample’s mean MMPI-2/A profile was similar to our past studies of patients suffering from end-stage cystic fibrosis (i.e., 1–3–2 mean clinical profile; Crews et al., 2000) and COPD (i.e., 1–3 mean clinical profile; Crews et al., 2001). Furthermore, the MMPI-2 findings in this study appear to support the psychological findings of previous investigations involving patients with COPD that have found symptomatology such as reactive depressions, generalized dissatisfaction with life, and preoccupation with somatic complaints, as well as clinical elevations on the Hypochondriasis, Depression, and Hysteria scales of the MMPI (McSweeny et al., 1982; Prigatano et al., 1984).

In sum, the findings from this study revealed that this sample of patients with end-stage pulmonary disease exhibited significantly greater frequencies of neurocognitive impairment (as compared to the expected population frequency distributions) only on SRT tasks that particularly involve the “selective reminding,” retrieval, and immediate recall of an unrelated word list that is presented aurally. While it remains unknown whether these results are more strongly related to the sensitivity of the SRT (vs. other measures) to identify certain types of neurocognitive (i.e., verbal memory) dysfunction and/or if end-stage pulmonary disease somehow differentially impairs the functional brain system responsible for certain aspects of verbal memory to a greater degree than other cognitive systems, it is likely that patients’ neurocognitive test performances were negatively impacted by their progressive declines in pulmonary functioning and resulting chronic hypoxia (see Lezak, 1995). Furthermore, as pulmonary disease advances, hypercapnea (increased blood carbon dioxide levels) may result secondary to ineffective exchanges of gases (Stuss et al., 1997). Stuss et al. (1997) have provided some data gathered from COPD patients suggesting that hypercapnea may have a more deleterious effect on neurocognitive functioning as compared to reduced oxygen delivery. Grant et al. (1982) have also theorized that pulmonary disease patients may suffer from other diseases, which, in combination with their pulmonary disease, may compromise their cerebral functioning. Similarly, Prigatano et al. (1983) have hypothesized that chronic pulmonary disease may enhance preexisting vascular disease and accelerate the aging process resulting in decreased cerebral blood flow and oxygen consumption and deficient neurocognitive performances. Finally, since psychological distress has been widely shown to negatively impact neurocognitive functioning (see Lezak, 1995), patients’ neurocognitive deficits may have also been exacerbated by their reported levels of psychological distress as reflected in their elevated MMPI-2/A clinical profiles.

The findings from this study appear to have important implications for clinical practice. Although as a group, patients demonstrated their most frequent neurocognitive impairments on tasks involving immediate free recall and retrieval of noncontextual verbal material, notable numbers of patients also exhibited impairments on measures assessing contextual verbal and nonverbal memory, as well as on tasks purported to be sensitive to executive functioning. Thus, based on the diversity of neurocognitive and psychological dysfunctions observed in these patients, clinically, it appears important to obtain comprehensive neuropsychological evaluations on all patients with similar end-stage pulmonary diseases who are potential lung transplant candidates. Specifically, since such patients must be capable of complying with often complicated and changing medical treatments/medication regimens both pre- and post-transplantation, comprehensive neuropsychological evaluations may be beneficial in identifying individual neurocognitive strengths and weaknesses. In turn, individualized, remedial treatments/therapy strategies (e.g., education as regards memory enhancement techniques, placing information to be retained in a verbal or visual context depending on a particular patient’s most efficient learning strategies, providing concrete/simplified medical treatment directions) can be formulated for patients with identifiable neurocognitive difficulties to help them compensate for their deficiencies and to increase their compliance with medical treatments and medication regimens. Identification and treatment (e.g., via psychotherapy and/or psychotropic medication management) of patients with significant psychological distress is also important to enhance their overall senses of well-being and mental health and to maximize their levels of neurocognitive functioning.

Future research is required with similar patients with chronic pulmonary disease to examine the precise mechanisms that are involved in the neurocognitive impairments that were identified in this study. The potential changes in such patients’ neuropsychological functioning over time and the impact that various medical interventions/treatments and lung transplantation may have on their neurocognitive processes should also be studied.

References

  • Borak J, Sliwinski P, Tobiasz M, Gorecka D, Zielinski J. Psychological status of COPD patients before and after one year of long-term oxygen therapy. Monaldi Archives of Chest Diseases. 1996;51(1):7–11. [PubMed]
  • Buschke H, Fuld PA. Evaluating storage, retention, and retrieval in disordered memory and learning. Neurology. 1974;11:1019–1025. [PubMed]
  • Butcher JN, Graham JR, Williams CL, Kaemmer B. Minnesota Multiphasic Personality Inventory-Adolescent. Minneapolis, MN: University of Minnesota Press; 1992.
  • Crews WD, Jr, Jefferson AL, Bolduc T, Elliott JB, Ferro NM, Broshek DK, Barth JT, Robbins MK. Neuropsychological dysfunction in patients suffering from end-stage chronic obstructive pulmonary disease. Archives of Clinical Neuropsychology. 2001;16(7):643–652. [PMC free article] [PubMed]
  • Crews WD, Jr, Jefferson AL, Broshek DK, Barth JT, Robbins MK. Neuropsychological sequelae in a series of patients with end-stage cystic fibrosis: Lung transplant evaluation. Archives of Clinical Neuropsychology. 2000;15(1):59–70. [PubMed]
  • Graham JR. MMPI-2 assessing personality and psychopathology. 2nd ed. New York: Oxford University Press; 1993.
  • Grant I, Heaton RK, McSweeny AJ, Adams KM, Timms RM. Neuropsychologic findings in hypoxemic chronic obstructive pulmonary disease. Archives of Internal Medicine. 1982;142:1470–1476. [PubMed]
  • Grant I, Prigatano GP, Heaton RK, McSweeny AJ, Wright EC, Adams KM. Progressive neuropsychologic impairment and hypoxemia. Archives of General Psychiatry. 1987;44:999–1006. [PubMed]
  • Greene RL. MMPI-2/MMPI, an interpretation manual. Boston: Allyn and Bacon; 1991.
  • Hathaway SR, McKinley JC. Minnesota Multiphasic Personality Inventory-2 test booklet. Minneapolis, MN: University of Minnesota Press; 1989.
  • Heaton RK, Chelune GJ, Talley JL, Kay GG, Curtiss G. Wisconsin Card Sorting Test manual: revised and expanded. Odessa, FL: Psychological Assessment Resources; 1993.
  • Heaton RK, Grant I, Matthews CG. Comprehensive norms for an expanded Halstead–Reitan battery. Odessa, FL: Psychological Assessment Resources; 1991.
  • Jefferson AL, Crews WD, Jr, Broshek DK, Barth JT, Robbins MK. An examination of the neuropsychological sequelae associated with end-stage pulmonary disease secondary to alpha-1-antitrypsin deficiency. Advances in Medical Psychotherapy and Psychodiagnosis. in press.
  • Lezak MD. Neuropsychological assessment. 3rd ed. New York: Oxford University Press; 1995.
  • McSweeny AJ, Grant I, Heaton RK, Adams KM, Timms RM. Life quality of patients with chronic obstructive pulmonary disease. Archives of Internal Medicine. 1982;142:473–478. [PubMed]
  • Prigatano GP, Parsons O, Wright E, Levin DC, Hawryluk G. Neuropsychological test performance in mildly hypoxemic COPD patients. Journal of Consulting and Clinical Psychology. 1983;51:108–116. [PubMed]
  • Prigatano GP, Wright EC, Levin D. Quality of life and its predictors in patients with mild hypoxemia and chronic obstructive pulmonary disease. Archives of Internal Medicine. 1984;144:1613–1619. [PubMed]
  • Reitan RM. Manual of administration of neuropsychological test batteries for adults and children. Tucson, AZ: Reitan Neuropsychological Laboratories; 1979.
  • Reitan RM, Wolfson D. Halstead–Reitan neuropsychological test battery: Theory and clinical interpretation. 2nd ed. Tucson, AZ: Neuropsychology Press; 1993.
  • Rey A. L’;examen clinique en psychologie. Paris: Presses Universitaires de France; 1964.
  • Spreen O, Strauss E. A compendium of neuropsychological tests. 2nd ed. New York: Oxford University Press; 1998.
  • Stuss DT, Peterkin I, Guzman DA, Guzman C, Troyer AK. Chronic obstructive pulmonary disease: Effects of hypoxia on neurological and neuropsychological measures. Journal of Clinical and Experimental Neuropsychology. 1997;19(4):515–524. [PubMed]
  • Wechsler D. WAIS-R manual. New York: Psychological Corporation; 1981.
  • Wechsler D. WMS-R manual: Wechsler Memory Scale—Revised. San Antonio, TX: Psychological Corporation; 1987.
  • Wechsler D. Wechsler Intelligence Scale for Children. 3rd ed. San Antonio, TX: Psychological Corporation; 1991.
  • Wechsler D. WMS-III: Wechsler Memory Scale—III. San Antonio, TX: Psychological Corporation; 1997.
  • Winer BJ. Statistical principles in experimental design. 2nd ed. New York: McGraw-Hill; 1971.