PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Am J Alzheimers Dis Other Demen. Author manuscript; available in PMC 2010 April 1.
Published in final edited form as:
PMCID: PMC2836907
NIHMSID: NIHMS128224

The Northwestern Anagram Test: Measuring Sentence Production in Primary Progressive Aphasia

Abstract

Primary progressive aphasia (PPA) is a clinical dementia syndrome with early symptoms of language dysfunction. Post mortem findings are varied and include Alzheimer’s disease and frontotemporal lobar degeneration (FTLD), both tauopathies and TDP-43 proteinopathies. Clinical-pathologic correlations in PPA are complex but the presence in the clinical profile of agrammatism has a high association with tauopathy. Grammatical competence is difficult to assess in the clinical setting with available methods. This paper describes the Northwestern Anagram Test (NAT), a new clinical measure of sentence production. Sixteen patients with PPA and their controls assembled single printed words to create sentences describing pictures. NAT performance was significantly correlated with a measure of sentence production and with aphasia severity, but not with measures of naming, single word comprehension, object recognition or motor speech. The NAT can be used to assess syntax competence when patients cannot be tested with measures that requiring intact speech production.

Keywords: grammar, syntax, frontotemporal dementia, speech

INTRODUCTION

Primary progressive aphasia is a clinical dementia syndrome in which the earliest symptoms consist of a decline in language, with initial preservation of episodic memory and other cognitive and behavioral domains 1. The clinical syndrome of PPA has been associated with a variety of neuropathological diseases, including Alzheimer’s and the tauopathy and TDP-43 proteinopathy forms of frontotemporal lobar degeneration (FTLD) 27. As in stroke-related aphasia, the clinical aphasia profile can vary based on the level of functioning in language components, in particular single word comprehension, naming, grammatical processing and output fluency, and this has led to classification of clinical subtypes of PPA 8.

Assessment of language in PPA typically involves administration of a number of tests to examine deficit patterns within and across language domains, including lexical semantics and syntax. Lexical semantic deficits are tested by evaluating single word comprehension, which is relatively simple to accomplish in the clinical setting and there are several commercially available measures for this, including subtests of the Western Aphasia Battery-Revised (WAB-R) 9 and Boston Diagnostic Aphasia Examination (BDAE) 10 and the Peabody Picture Vocabulary Test-IVth Edition (PPVT-IV) 11. Naming, or lexical access, also can be tested using subtests of the WAB-R or BDAE as well as the Boston Naming Test (BNT) 12. Unlike the relative simplicity of evaluating lexical semantic deficits such as single word comprehension and naming, however, testing syntactic processing, that is, patients’ ability to comprehend and produce sentences, is much more challenging. Motor speech deficits and anomia often interfere with spontaneous or prompted sentence production and single word comprehension and working memory deficits can influence performance on sentence comprehension tests, even when syntactic impairments do not exist. Indeed, it has been shown that the presence of syntactic impairments, a symptom of agrammatism, may reliably predict tauopathy at post mortem autopsy 5. Thus, it would be advantageous to be able to easily detect syntactic deficits, especially when relevant pharmacologic treatments targeted at the disease pathophysiology become available.

There are several clinical methods for testing sentence comprehension and production. To test sentence comprehension, sentence-picture matching tasks often are used, which require that patients point to the one picture among two or more choices that corresponds to a sentence produced by the examiner. Another method that has been used in studies of grammatical comprehension in patients with FTLD by Grossman and colleagues 13, is to provide the patient with a sentence and then ask questions that probe understanding of grammatical relations between sentence constituents. For testing sentence production, picture description tasks are common. These involve providing the patient with a picture and requesting that a sentence be produced to describe it. This method has been used extensively in stroke-induced aphasia and also to characterize patients with PPA 8. Another method is the cloze procedure, whereby sentence production is prompted with the first few words of target sentences. Finally, sentence production ability can be evaluated using linguistic analysis of spontaneous speech. Beeke and Wilkinson 14 argue that tasks such as sentence-picture matching are artificial and do not capture the true level of processing that can be obtained with spontaneous speech samples. In fact, spontaneous speech samples provide an extraordinarily rich source of data from which to analyze syntactic ability 15, 16

All of these methods have shortcomings, which are particularly relevant when testing patients with progressive illness. With regard to spontaneous speech analysis, it is difficult to control the topic, the rate, and length of the interchange, resulting in less than optimal corpus size. Even if this obstacle is overcome with controlled stimuli, such as telling the “Cinderella Story” from a wordless picture book, often used to evaluate stroke-induced agrammatic aphasia 15, coding the speech sample is time-consuming and requires a methodology not readily available in most clinical settings. Constrained comprehension and production tasks are simpler to administer and the results are more straightforward to analyze. However, sentence comprehension tasks such as sentence-picture matching rely on relatively intact single word comprehension and working memory, and sentence production tasks require intelligible speech production. In patients with PPA, language declines across domains. Thus, sentence deficits may be difficult to test in later stages of the disease.

Even if word comprehension and production deficits in PPA did not need to be considered for testing sentence comprehension and production, there are few standardized/published tests available for this purpose. Of the few available, none examine important aspects of syntax, such as comprehending and producing canonical and non canonical sentence structures. Canonical sentences follow the basic form of any language --Subject-Verb-Object (SVO) in English – whereas, non canonical forms depart from this order and render sentences more complex and difficult to comprehend and produce. One recently developed test, the Northwestern Assessment of Verbs and Sentences (NAVS; Thompson, experimental version), includes subtests for evaluating both canonical and non canonical sentences, as illustrated in Table 1 below. The NAVS has been field-tested, resulting in a substantial amount of laboratory data on stroke-induced aphasic patients, PPA patients, and controls.

Table 1
Samples Of Canonical And Non Canonical Sentence Types On The Northwestern Assessment of Verbs And Sentences (NAVS)

The NAVS includes the Sentence Comprehension Test (SCT), which uses a sentence-picture matching paradigm and the Sentence Production Priming Test (SPPT), which primes target sentence structures by providing a sample (prime) sentence describing a picture and then asking the patient to produce a sentence with identical structure in response to a semantically reversed sentence. Thus, even though the NAVS controls for the sentence structures tested, it may be too difficult for some patients with PPA to perform.

The purpose of the present paper is to describe and present data derived from a new measure, the Northwestern Anagram Test (NAT), which we developed to test accuracy of word order (syntax) in sentence production in patients who present with speech production, word comprehension and/or word-finding difficulties or reduced working memory capacity. The anagram method requires the assembly of individual word cards presented in scrambled order into meaningful sentences. One published test for aphasia, the Verb and Sentence Test (VAST; 17) includes an anagram task, however, its major purpose is to evaluate verb and verb argument structure production deficits. Thus, the stimulus cards group words to form complete sentence constituents (e.g., complete noun phrases: The boy is chasing the dog.) In addition, the VAST examines only active and passive sentence structures. The NAT provides a separate stimulus card for each word of the target structures such that the patient has few clues as to permissible word combinations. In addition, like the NAVS, the NAT examines several canonical forms (actives, subject extracted wh-questions, and subject clefts) and non canonical forms (passives, object extracted wh-questions, and object clefts) (Table 1).

Here we report results obtained with the NAT from patients with PPA and their cognitively intact controls. NAT performance was compared with performance on the SPPT of the NAVS, and was compared with measures of aphasia severity, naming, single word comprehension, oral apraxia and object recognition to demonstrate its specificity for syntax. It was hypothesized that the NAT would correlate with measures of syntax but not with measures of naming, word comprehension or motor speech production. Furthermore, it was predicted that performance would be influenced by grammatical complexity. Thus, scores would be higher for canonical than non canonical structures and, within the non canonical category, sentences that involve more syntactic movement, or displacement of sentence constituents from their base (canonical) position to other positions in the sentence, would be harder than those in which there is less movement. For example, Object-Cleft and Object-Wh questions involve Wh-movement, whereas Passive structures involve NP-movement 18, 19. Thus, Wh-movement can be considered more complex than NP-movement because in the former movement occurs across clausal boundaries, whereas in the latter it does not. This principle, well documented in stroke-induced agrammatic aphasia 20, was also demonstrated in patients with PPA in the present study.

METHOD

Participants

Sixteen patients diagnosed with PPA, ranging in age from 48–75 years of age, and 16 age- and gender-matched healthy volunteers participated in the study. Patients with PPA were recruited from the Primary Progressive Aphasia Program at the Northwestern Cognitive Neurology and Alzheimer’s Disease Center (CNADC) and healthy control participants were recruited from the CNADC Clinical Core registry. The registry includes well characterized unimpaired older adults, all of whom were administered research neuropsychological and neurological examinations to confirm normal cognitive functioning. All PPA patients also were participants in the CNADC registry, which provided the clinical and neuropsychological data used to establish the diagnosis. Furthermore, the CNADC registry also invited participants to consider brain donation at the time of death so that future studies might address the relationship between clinical symptoms and neuropathology.

Table 2 presents demographics and clinical aphasia scores for each PPA patient and mean scores for the control sample. The mean age of PPA patients was 60 (± 6.4) and mean education level was 15.9 years (± 2.4). Half the sample was male. Patients and controls did not differ by age, education or gender distribution. The duration of reported symptoms in the PPA group ranged from 2–7 years (mean 4.0 ± 1.9). Informed consent was obtained from all participants and the study was approved by the Institutional Review Board of Northwestern University. None of the study participants have been reported elsewhere.

Table 2
Demographics And Clinical Language Measures

Procedures

Neuropsychological and Language Testing

All participants had been administered a battery of neuropsychological tests to assess attention, memory, visuospatial functions, face recognition, and executive functions. In addition, the Western Aphasia Battery 9, the Boston Naming Test (BNT, 12), the Peabody Picture Vocabulary Test (PPVT-IV,11 ), the three picture version of the Pyramids and Palm Trees (PPT) 21 and the NAVS (Thompson, experimental version) were administered to examine language abilities, and a test of oral apraxia was administered to evaluate speech function. As expected, the healthy volunteers performed at ceiling on all tests, whereas the PPA patients presented language deficits of various types and severity. For the purposes of the present report, the following language measures derived from this battery were of interest: the WAB Aphasia Quotient (AQ, for general severity), BNT total score (naming), PPVT-IV total score (single word comprehension,) NAVS SPPT, and the oral apraxia score. Participants' performance on these measures appears in Table 2.

For the patients, AQ’s ranged from 65.4 to 97.2 (mean 82.1 (± 9.0). Scores on the BNT ranged from 3–59 with a median of 49.5, indicating that half the subjects had naming scores within the average range for their age. Single word comprehension as measured by the mean score on the PPVT-IV ranged from 8 to 36 (perfect score) (mean 29.1 ± 9.8), with four subjects’ scores falling considerably below the mean. The oral apraxia scores also indicated a range of ability, from moderately impaired to normal. One subject (P14) was unable to perform the oral apraxia measures due to severe output deficits.

Northwestern Anagram Test Stimuli and Procedure

Picture stimuli for the anagram test were taken from the NAVS such that oral sentence production from the NAVS SPPT and sentence anagram construction could be compared. In addition, the same six sentence types included in the NAVS were used for the NAT. As noted above these consisted of three canonical structures [active (ACT), subject- extracted wh-questions (Subj-Wh), and subject clefts (Subj-Cl)] and three non canonical structures [passives, object-extracted wh- questions (Obj-Wh) and object clefts (Obj-Cl)]. Five exemplars of each sentence type were tested, for a total of 15 canonical and 15 non canonical items.

Subjects were presented with a target drawing depicting two actors and an action (see Figure 1). Printed words and arrows labeled each actor and the action in the picture. For each target sentence, the individual words constituting the correct sentence were printed on small cards. For each item, the examiner pointed to each actor in the picture, stating, for example, “This picture shows a man and a woman. The action is ‘kiss’.” The examiner then placed the test cards under the picture and said, “Make a sentence that matches this picture using these word cards.” For active and passive structures, the examiner provided the first two words; for questions, the first word was provided and for cleft constructions the first four words (e.g., “It is the boy….) were set out for the patient. It was necessary to provide the first words in order to obligate production of target sentence types. Without this method, subjects could produce simpler sentence structures. The examiner said, “Start with these words but use all the words provided.” Two sample items, one active construction and one Subject-Wh question, were given prior to testing and subjects were provided feedback and corrected if wrong. The 30 items then were administered, with 20 seconds to complete each. One point was given for each correct sentence. Sentence types were presented in random sequence but each subject received the same sequence.

Figure 1
Sample Item From The Northwestern Anagram Test (NAT)

Data Analysis

Response accuracy for each participant and group means for canonical and non canonical sentence production as well as total score were computed. These means then were compared using repeated measures analysis of variance, followed by post-hoc t-test. Data are reported as mean ± standard deviation (SD). NAT total, canonical, and non canonical scores also were correlated with the NAVS SPPT total, canonical, and non canonical scores to determine if these measures are similar in their ability to detect sentence production impairment. Finally, scores on the NAT were correlated with the WAB AQ, and with scores on the BNT, PPVT-IV, PPT and the oral apraxia test to evaluate if performance was related to aphasia severity, naming, single word comprehension, object recognition and motor speech deficits, respectively. All correlations were done using Spearman’s correlation co-efficient. No comparisons were made between PPA patients’ performance and that of the normal participants because the healthy volunteers unanimously obtained near perfect scores on all measures, as noted above.

Results

Performance on the NAT for control participants was near ceiling for both canonical and non canonical structures. Percent correct production across sentence types for the PPA participants and mean NAVS SPPT scores are shown in Table 3 and Figure 2. Statistical analyses of these data indicated that the average percent correct for canonical sentences was significantly greater than for non canonical sentences [89.2 ± 24.0 vs 67.2 ± 29.0 , p (2-tail) = .0015]. Repeated measures ANOVA comparing individual sentence types showed an overall significant effect (p=.0002). Table 4 shows the results of pairwise comparisons among the 6 sentence types. Active, Subj-Wh, and Subj-Cl sentences differed significantly from Obj-Wh and Obj-Cl sentences, but not from the other sentence types, with production of all canonical sentences superior to noncanoncial structures. Considering the non canonical forms, Passive structures were produced significantly better than both Obj-Wh and Obj-Clefts and there was no significant difference between production of Obj-Wh and Obj-Cleft structures, although Obj-Clefts were more difficult to produce than Obj-Wh structures.

Figure 2
NAT and NAVS Percent Correct By Sentence Type
TABLE 3
Northwestern Anagram Test Scores
TABLE 4
Significance Levels For Pairwise Comparisons Among NAT Sentence Types

The NAT canonical score was correlated with the NAVS SPPT canonical score but the relationship just missed significance (r=.51, p=.06), likely due to the relative lack of variability in scores on these easier sentence structures. The non canonical scores on these two measures, however, were significantly correlated (r=.76, p=.002). In addition the NAT total and non canonical scores were each significantly correlated with the WAB AQ (r=.57, p=.02 and r=.57, p=.02, respectively), but not with the BNT, PPVT-IV, or oral apraxia scores. NAT canonical scores were not significantly correlated with any of the clinical aphasia measures of word comprehension, naming and motor speech production.

We also analyzed a subset of NAT items, namely, Obj-Wh and Subj-Wh structures to determine whether scores for these structures would be representative of scores derived from the 30-item NAT. Even though the 30-item NAT did not require a great deal of time to administer (less than 30 minutes even for lower functioning patients), a shorter test might be more desirable in certain settings. The Obj-Cleft sentences were the most difficult for all subjects and, therefore, were not considered desirable as the sole measure of sentence production. Active structures were far too easy and non discriminating. We, therefore, selected the two intermediate level difficulty structures (Obj-Wh and Subj-Wh) from the canonical and non canonical subsets, respectively, and combined these two into a 10-item subset. The total on this subset showed the same correlation pattern as did the total and non canonical NAT scores. That is, there was a significant correlation between the 10-item NAT and the WAB (r=0.54, p=.03), but not with measures of naming, single word comprehension, object recognition or motor speech. In addition, the total score on the10-item subset correlated highly with the 30-item total, canonical and non canonical NAT scores (p<.0001, =.006, and <.0001, respectively).

Two Cases Illustrating Advantages of The NAT

Case P26: Motor speech impairment, normal sentence construction

P26 was a 48-year-old, right-handed, Caucasian man, with 16 years of education and a long-term job as a maintenance technician in a hospital. His symptoms began approximately 6.5 years before testing and consisted of the early appearance of word-finding difficulty progressing to deficits in speech production, spelling and writing. His Aphasia Quotient of 83.2 indicated a moderately severe aphasia, with significant reduction in speech fluency. In contrast, he obtained perfect or near-perfect scores on the PPVT-IV and the BNT, respectively. Due to severe speech limitations, he struggled for a very long time to complete the NAVS SPPT but obtained a score of 91% correct. In contrast, he was quickly able to demonstrate the preservation of grammatical processing on the NAT, with a total score of 100%.

Case P24: Single word comprehension impaired, relatively preserved sentence construction

P24 was a 53-year-old, right-handed woman with a 3.5-year history of PPA. Single word comprehension was severely impaired as was naming. On the NAVS SPPT, her total accuracy score was 53%, suggesting impairment of grammatical sentence production. However, performance on the NAT was 73%, suggesting more preserved appreciation for sentence structure than indicated by the sentence production task.

Discussion

Syntactic abilities are important to evaluate in patients with PPA because deficits in this domain may signal tauopathy as opposed to some other type of neurodegenerative disease 5. One aspect of syntactic ability, sentence production, is typically assessed using picture description, spontaneous language sample analysis or other methods, which require adequate word finding, motor speech, single word comprehension and working memory abilities. In this paper we examined sentence production in patients with a clinical diagnosis of PPA using a novel anagram assembly task, the Northwestern Anagram Test (NAT), which permitted assessment of sentence construction while minimizing the impact of these deficits on performance. Consistent with the different clinical profiles expressed in PPA, patients of varying severity and clinical profiles demonstrated a range of scores on this measure, with the lowest scores obtained on the non canonical sentence types. Performance on the NAT was significantly correlated with the NAVS SPPT and with aphasia severity as measured by the WAB, but not with scores on tests of naming, single word comprehension, object recognition or oral apraxia.

The validity of the NAT as a measure of syntactic production was demonstrated by its correlation with the NAVS SPPT, which assesses similar types of sentences. In some cases the NAT mean score was higher, in others, the SPPT score was higher, suggesting that each measure may have its individual benefits for different patients depending on their relative strengths and weaknesses. Scores on the NAT for sentences with canonical word order were significantly higher than scores for those with non canonical order. Furthermore, pairwise comparisons among sentence types showed statistically significantly differences between all canonical forms and two of the non canonical forms: object wh-questions and object-clefts. Clearly, both Object-Cleft and Object Wh-question structures were more difficult, with scores falling below 58% accuracy, whereas mean scores on the canonical structures exceeded 86% accuracy. These results are consistent with our predictions and with the well known fact that the more complex the sentence structure, in terms of standard word order, the more difficult the sentence is to process for aphasic patients 22, 23.

The validity of the NAT for assessing syntactic production also was demonstrated by performance patterns on the non canonical structures. As predicted, Object-Clefts and Object-Wh questions were significantly more difficult to produce than Passive structures. Another interesting finding, although not statistically significant, was that within Wh-movement structures Object-Clefts were more difficult to produce than Object-Wh questions. Once again, considering syntactic complexity, Object-Clefts are more complex than Object-Wh questions. Even though both rely on Wh-movement, Object-Clefts involve syntactic embedding, whereas Object-Wh questions do not. Notably, this same pattern of performance is pervasive in stroke-induced agrammatic aphasia; that is, syntactically more complex structures are more difficult to produce than syntactically simpler ones 20

A subset of ten items was identified that had the same correlation pattern as the total test and was made up of Object Wh- and Subject Wh-questions. Thus, similarly to the entire test, these items were correlated with aphasia severity but not with measures of naming, single word comprehension, object recognition or oral apraxia. This subset could serve to provide a suitable shortened version amenable to more rapid clinical assessment.

Although the NAT appears to fill the need for a short, clinically convenient and valid measure of syntactic processing, there are several potential caveats. First, the NAT is not a substitute for a thorough investigation of grammatical processing that would include more extensive investigation of sentence processing in different modalities, verb processing and morphology. Sentence processing speed may also play a role in comprehension and production and we did not investigate time as a variable in NAT performance. Although most subjects were able to complete their constructions within the 20-second time limit, the addition of a time variable might contribute to a scoring system that could mitigate ceiling effects.

The NAT provides a relatively brief, easily administered test of grammatical sentence production that can be used even with moderately impaired patients with PPA to assess an important aspect of grammar competence in language output. Patients who had significant motor speech deficits or word-finding difficulty were able to demonstrate sentence processing skills that could not easily be measured with tasks requiring verbal output. Patients with single word comprehension deficits were able to demonstrate intact sentence processing ability due to the availability of labels for the actors and actions in the stimulus pictures to which they could refer in the course of performance. The presence of all the words necessary for constructing the sentences also avoided potential interference from working memory deficits that can influence sentence comprehension and production tasks.

The subtyping of PPA by clinical profile has been encouraged by studies reporting distinctive neuroanatomical and neuropathological characteristics associated with different profiles 8. The NAT offers a convenient alternative test of syntactic ability that could be utilized to identify patients with agrammatism.

Acknowledgments

This project was supported supported by DC008552 from the National Institute on Deafness and Other Communication Disorders and AG13854 (Alzheimer Disease Core Center) from the National Institute on Aging. The authors wish to thank Kyla Garibaldi, B.S., for her help with testing and Sarah Weitner, M.A. for her assistance with data processing.

Funding Sources: Grants DC008552 from the National Institute on Deafness and Other Communication Disorders and AG13854 (Alzheimer Disease Core Center) from the Nationa Institute on Aging.

Footnotes

Disclosures: The authors have reported no conflicts of interest

REFERENCES

1. Mesulam MM. Primary progressive aphasia--a language-based dementia. N Engl J Med. 2003 Oct 16;349(16):1535–1542. [PubMed]
2. Turner RS, Kenyon LC, Trojanowski JQ, Gonatas N, Grossman M. Clinical, neuroimaging, and pathologic features of progressive nonfluent aphasia. Ann Neurol. 1996 Feb;39(2):166–173. [PubMed]
3. Rossor MN, Revesz T, Lantos PL, Warrington EK. Semantic dementia with ubiquitin-positive tau-negative inclusion bodies. Brain. 2000;123(Pt 2):267–276. [PubMed]
4. Mesulam MM, Weintraub S. Spectrum of primary progressive aphasia. Baillieres Clin Neurol. 1992;1(3):583–609. [PubMed]
5. Mesulam M, Wicklund A, Johnson N, et al. Alzheimer and frontotemporal pathology in subsets of primary progressive aphasia. Ann Neurol. 2008 Jun;63(6):709–719. [PMC free article] [PubMed]
6. Knibb JA, Xuereb JH, Patterson K, Hodges JR. Clinical and pathological characterization of progressive aphasia. Ann Neurol. 2006 Jan;59(1):156–165. [PubMed]
7. Kertesz A, Hudson L, Mackenzie IR, Munoz DG. The pathology and nosology of primary progressive aphasia. Neurology. 1994 Nov;44(11):2065–2072. [PubMed]
8. Gorno-Tempini ML, Dronkers NF, Rankin KP, et al. Cognition and anatomy in three variants of primary progressive aphasia. Annals of Neurology. 2004;55:335–346. [PMC free article] [PubMed]
9. Kertesz A. Western Aphasia Battery- Revised (WAB-R) Austin, Texas: 2006. Pro-Ed.
10. Goodglass H, Kaplan E, Barresi B. Boston Diagnostic Aphasia Examination Third Edition (BDAE-3) Lutz, Florida: 2001. Par.
11. Dunn LM, Dunn LM. Peabody Picture Vocabulary Test Fourth Edition (PPVT-IV) Bloomington, MN: NCS Pearson; 2007.
12. Kaplan E, Goodglass H, Weintraub S. The Boston Naming Test. Philadelphia: Lea and Febiger; 1983.
13. Grossman M, Rhee J, Moore P. Sentence processing in frontotemporal dementia. Cortex. 2005 Dec;41(6):764–777. [PubMed]
14. Beeke S, Wilkinson R, Maxim J. Exploring aphasic grammar. 2: Do language testing and conversation tell a similar story? Clin Linguist Phon. 2003 Mar;17(2):109–134. [PubMed]
15. Thompson CK, Shapiro LP. Training sentence production in agrammatism: implications for normal and disordered language. Brain & Language. 1995;50(2):201–224. [PubMed]
16. Saffran E, Berndt RS, Schwartz MF. The quantitative analysis of agrammatic production: procedure and data. Brain Lang. 1989;37:440–479. [PubMed]
17. Bastiaanse R, Edwards S, Rispens J. The verb and sentence test (VAST) Thurston, Suffolk: Thames Valley Test Company; 2002.
18. Marantz A. The minimalist program. In: Webelhuth G, editor. Government and binding theory and the minimalist program. London: Basil Blackwell; 1995.
19. Chomsky N. Barriers (Linguistic Inquiry Monograph 13) Cambridge, MA: MIT Press; 1986.
20. Thompson CK, Shapiro LP. Complexity in treatment of syntactic deficits. American Journal of Speech-Language Pathology. 2007 Feb;16(1):30–42. [PMC free article] [PubMed]
21. Howard D, Patterson K. Pyramids and palm trees: a test of semantic access from pictures and words. Bury, St. Edmunds: Thames Valley Test Company; 1992.
22. Thompson CK, Shapiro LP, Kiran S, Sobecks J. The role of syntactic complexity in treatment of sentence deficits in agrammatic aphasia: the complexity account of treatment efficacy (CATE) J Speech Lang Hear Res. 2003 Jun;46(3):591–607. [PMC free article] [PubMed]
23. Saffran EM, Schwartz MF, Marin OS. The word order problem in agrammatism. II. Production. Brain & Language. 1980 Jul;10(2):263–280. [PubMed]