We found that prospective estimates of the lifetime prevalence of DSM-defined disorders markedly exceed retrospective estimates. Our comparison of prospective versus retrospective data held constant the 15-year age window from 18 to 32 years, and the historical period when data were collected, from 1990 to 2005. Lifetime prevalence was almost identical in the NZMHS and the NCS-R, ruling out the possibility that cultural or ethnic differences between New Zealand and the USA could account for our findings.
Three findings suggest that lifetime prevalence of disorder is higher than previously estimated by retrospective surveys. First, the percentage of people who experienced lifetime disorder to age 32 was approximately doubled in prospective data as compared to retrospective data. Second, prospective assessment resulted in a mean of only 38% of lifetime cases having disorder during the past year whereas retrospective measurement of lifetime disorder resulted in higher means of 57% (NZMHS, NCS-R) and 65% (NCS) of lifetime cases having disorder during the past year. Third, prospective measurement yielded lifetime estimates that suggest the experience of certain DSM-defined disorders by age 32 may be very common indeed: anxiety disorder (49.5%), depression (41.4%), alcohol dependence (31.8%) and cannabis dependence (18.0%).
We initially compared past-year prevalence rates in the Dunedin Study versus past-year prevalence rates in the NZMHS, NCS-R and NCS (). This comparison was required because only if past-year prevalences in the Dunedin cohort seemed reasonably valid could we later infer that the lifetime prevalences derived as a count of past-year cases were likewise reasonable. There were methodological differences among the studies but despite these differences, the past-year prevalence of disorder in the Dunedin Study was similar to the past-year prevalence in the NZMHS, NCS-R and NCS, or somewhat higher for expected reasons (such as the Dunedin Study’s more complete sample). Dunedin lifetime prevalence rates from age 18 to 32 reflect a cumulative count of these past-year cases.
The Dunedin Study’s cumulative prevalence of individuals experiencing disorder between ages 18 and 32 years was approximately double the counterpart prevalence in the NZMHS, NCS-R and NCS (). For example, a count of individuals ever diagnosed at any Dunedin Study assessment revealed that 41.4% of the cohort experienced at least one episode of depression between ages 18 and 32 versus 18.5% in the NZMHS, 19.0% in the NCS-R and 16.9% in the NCS. This discrepancy does not arise from cultural differences between the USA and New Zealand because the NCS-R and NZMHS lifetime depression rates match. However, even if the Dunedin Study’s past-year depression data are doubted, the identical prospective/retrospective discrepancy was observed for anxiety and substance disorders. Past-year rates of anxiety disorders were almost the same in all four studies, but the prospective lifetime rates were double the rates from the three retrospective surveys. Furthermore, past-year rates of substance disorders were similar in Dunedin and NCS, but Dunedin prospective lifetime rates were double NCS retrospective rates.
If the past-year diagnoses added up to yield the Dunedin cohort’s lifetime prevalence are acceptably valid, then the surviving explanation for the discrepancy must implicate the fundamental difference between prospective versus
retrospective measurement: recall failure. Research into depression supports the notion that recall failure is substantial. Studies show that half of hospitalized depression cases ceased to be lifetime cases when interviewed with the CIDI 25 years later (Andrews et al. 1999
), 10% of depression cases diagnosed at baseline ceased to be lifetime cases when reinterviewed only 3 years later (Newman & Bland, 1998
), and half of longitudinal cohort members who previously reported depression did not recall their episodes by age 21 years (Wells & Horwood, 2004
). Another indicator of recall failure is that, among primary-care patients interviewed up to age 65, most recalled their first depression episode as occurring within 5 years of the interview, which was deemed implausible given depression’s peak age of onset in young adulthood (Simon et al. 1995
). One analysis indicated that plausible rates of recall failure (concealing 2–4% of depression cases per year) accumulated across the lifetime could account for retrospective surveys’ low prevalence (Patten, 2003
). An analysis that modeled Dutch and Australian national surveys to correct for recall failure estimated the lifetime prevalence of depression to be 30% in men and 40% in women (Kruijshaar et al. 2005
). Our findings suggest that this amount of recall failure applies beyond depression, to other disorders.
Who are all these people who experience disorder and then forget it ? It is possible that people who under- report have only mild disorder, but this is unlikely to be the full explanation because marked underreporting occurs among individuals hospitalized for depression (Andrews et al. 1999
). A check of the data for Dunedin cohort members with lifetime disorder revealed that many had not experienced disorder that was chronic or recurrent, at least not up to age 32. Of lifetime cases, 53% of those with anxiety, 60% of those with depression, 61% of those with alcohol dependence and 57% of those with cannabis dependence had been diagnosed with the disorder at only one of our past-year assessments. (Of lifetime cases, the percentages of cases diagnosed twice and diagnosed three or more times were respectively : 47% and 22% for anxiety disorder, 40% and 12% for depression, 39% and 12% for alcohol, and 42% and 18% for cannabis.) That half of lifetime cases were diagnosed only once in our longitudinal study suggests a hypothesis: retrospective surveys may undercount primarily individuals who have relatively short-term disorder or single episodes. Testing this hypothesis requires undertaking retrospective interviews in a prospectively studied cohort, to reveal which prospectively diagnosed cases go undetected retrospectively. Lacking retrospective lifetime interviews in the Dunedin Study, we could not carry out this test. However, our comparison of prospective versus
retrospective past-year-to-lifetime ratios is relevant (). In prospective studies, many more respondents have lifetime disorder than have past-year disorder, yielding a low past-year-to-lifetime ratio. This finding is expected. By contrast, in many retrospective surveys almost as many respondents have past-year disorder as have lifetime disorder, yielding a higher past-year-to-lifetime ratio. It is implausible that most respondents who report that they ever in their lives had an episode also happen to have an episode during the year they are interviewed for a survey (Kessler et al. 2002
). This implausible result from retrospective surveys could be explained if respondents who have long-standing, chronic or recurrent disorder are particularly likely to remember and report symptoms from the long-distant past, whereas respondents who experienced short-term, single episodes of disorder are likely to forget them, regardless of severity (Simon & VonKorff, 1995
). Supporting evidence comes from a two-wave study of depression that revealed that short illness duration is associated with unreliable reporting of lifetime depression (Foley et al. 1998
The data we used to estimate prospective prevalence come from one cohort in New Zealand. However, similarly high cumulative prevalence rates have been reported by researchers who have followed adolescent cohorts to young adulthood while conducting repeated diagnostic assessments (using different standardized interview instruments) in North Carolina (Costello et al. 2003
), New York (Jaffee et al. 2005
) and Oregon (Lewinsohn et al. 1993
), and in a 25-year longitudinal study of Australian teachers (Wilhelm et al. 2006
). That lifetime prevalence accumulates with repeated longitudinal measurement was confirmed by follow-ups in the ECA study (Regier et al. 1998
) and the NCS (Kessler et al. 2007
). For example, when the NCS-1 sample was followed up, NCS-1 lifetime depression prevalence was 21%, but this rose to 29% by adding NCS-2 (Kessler et al. 2007
Our study has three additional design limitations. However, all three indicate that the true lifetime prevalence of mental disorder may in fact be higher than we have been able to estimate in the Dunedin Study. First, NZMHS, NCS-R and NCS 18–32-year-old respondents could retrospectively report disorder they recalled as having occurred before age 18, whereas our Dunedin Study cumulative lifetime count began only at age 18. (The three surveys did not have respondents under age 18 to make ’s comparisons of past-year prevalence. Therefore, although the Dunedin Study has juvenile diagnoses we could not include them in this article.) As an example, adding depression diagnoses made before age 18 increases the Dunedin lifetime prevalence of depression from 41% to 44%. This limitation has the net effects of lowering the Dunedin Study’s estimates of lifetime prevalence and narrowing the prospective versus retrospective discrepancy we reported here.
A second limitation is that gaps between the Dunedin Study’s four 12-month assessment windows did not allow us to count individuals who experienced an episode of disorder only between windows. We previously reported that our ‘net’ of 1-year DIS diagnoses at ages 18, 21, 26 and 32 has captured all but eight of the cohort members who reported treatment for mental-health or substance-use problems between assessment windows (Moffitt et al. 2007
). Nevertheless, the number of cohort members we failed to count here because their only episodes of disorder occurred between study windows and went untreated is unknown. This limitation has the net effects of lowering our estimate of lifetime prevalence and narrowing the prospective versus
A third reason why Dunedin’s lifetime rates are underestimates is that Dunedin data are right-hand censored at age 32. Retrospective surveys suggest that many new cases should be expected after age 32 (Kessler et al. 2003a
). On the one hand, the number of new-onset cases after age 32 has probably been over-estimated because survey respondents often recall their onset age as older than it was, and forget episodes from early life (Simon et al. 1995
). On the other hand, new cases will be diagnosed as the Dunedin cohort ages. Our estimate to age 32 is an under-estimate of lifetime prevalence for the full life course.
We compared retrospective versus
prospective methods of ascertaining lifetime prevalence while holding constant the use of the DSM definitional approach to diagnosis in both types of studies. Therefore, this article is uninformative (and agnostic) about the validity of diagnoses of depression, anxiety and substance dependence as defined by DSM-IV. That is a separate debate (Horwitz & Wakefield, 2007
). Our rather more modest aim was to point out that objections voiced to surveys’ higher than expected lifetime prevalence of disorder are objections to prevalence that is only half what it could be in reality, because a very great deal of disorder has been lost to recall failure. Limitations of our research are such that we cannot here provide an estimate of the true prevalence of lifetime psychiatric disorders, but the findings can be taken as evidence that existing and oft-cited retrospective prevalence rates undercount not trivially, but substantially. This substantial undercounting is consequential because it can generate misleading findings in etiological research (Kendler et al. 1993
; Foley et al. 1998
) and misleading estimates of economic disease burden (Tang & Lopez, 1997). It is time for critical thinking about retrospective data.
It is not a new idea that retrospective surveys underdetect mental disorder (Kramer et al. 1980
). However, despite repeated demonstrations that this underdetection is real, resources are still invested in collecting retrospective data and journals continue to publish them as epidemiological information. In addition, peer reviewers still recommend rejection of papers from longitudinal studies on the basis that their cumulative number of prospectively diagnosed cases is far too high, as compared to survey prevalence rates. Survey researchers have taken care to explain that their surveys’ prevalence estimates are lower
than they could be (Kessler et al. 1994
) but paradoxically, much criticism continues to stem from the widespread belief that surveys’ prevalence rates are higher
than they should be. If the much-higher-than-expected lifetime prevalence now emerging from prospective studies is correct, then this widespread belief is a myth. If lifetime prevalence rates are as high as those we report here (or higher), debates may shift. It may be time to stop asking how surveys can achieve acceptably low rates of disorder. Instead, researchers might begin to ask why so many people experience a DSM-defined disorder at least once during their lifetimes, and what this prevalence means for etiological theory, the construct validity of the DSM approach to defining disorder, service-delivery policy, the economic burden of disease, and public perceptions of the stigma of mental disorder.