Side effects are common with both classes of antipsychotics and include; extra-pyramidal symptoms [3-7] and somnolence [4-6]. These effects may be less frequent [9] and less severe [8] with risperidone compared to haloperidol, but only at lower doses of the medicines [9]. Experimental evidence regarding the more serious adverse events of antipsychotics is limited to atypical antipsychotics and little is known about the safety of conventional antipsychotics in the elderly. Furthermore, most randomised controlled trials have limited follow-up of up to 12weeks [1] and the safety of both antipsychotic classes with long-term treatment remains unclear.

Post marketing observational studies of antipsychotics have identified many safety issues associated with antipsychotics, however, study results vary and conclusions are inconsistent.

Previous reviews of observational studies of antispychotics have focused on the clinical safety of conventional and atypical antipsychotics [10-13]. The aim of this review was to synthesise the current evidence from observational studies, regarding the serious adverse events of antipsychotics in elderly patients and to determine the impact of the observational study design utilised and the technique employed to control for confounding on study results.

We searched the PUBMED database and the Cochrane controlled trials register for all English-language articles published up to December 2010. We also conducted a manual search of bibliographies for other relevant articles. We included double-blind randomised controlled trials (RCTs), meta-analyses and published observational studies that evaluated adverse events of either conventional or atypical antipsychotic medications in elderly populations. All studies were included if they reported at least one of the adverse events of interest; death, cerebrovascular events, hip fracture or pneumonia. Studies specifically investigating the use of antipsychotics in schizophrenic patients were not included. Individual RCTs contributing to meta-analysis were not included in this review to avoid double counting of studies. In PUBMED we combined the results of 2 search domains: Dementia (MESH terms Dementia OR Dementia, Vascular NOT Schizophrenia), and drug therapy (Antipsychotic Agents) with each of the following searches for the outcomes of interest; death (Death OR Death, Sudden, Cardiac OR Death, Sudden OR Mortality), cerebrovascular events (Stroke), hip fracture (Hip Fractures) and pneumonia (Pneumonia OR Pneumonia, Bacterial OR Pneumonia, Aspiration). Author 1 (NP) was responsible for the retrieval and review of studies to be included in the final review. Author 2 (ER) provided independent assessment of studies when required.

We identified 106 articles, 7 of which were duplicates. Of the 99 unique records, 33 articles were excluded based upon review of the title and abstract. We reviewed 66 full text articles, of which 22 were excluded (Figure​(Figure1).1). Articles were excluded if they were editorials, letters, did not contain original data or outcome data of interest or were commentaries on already included published data.

We included observational studies that employed either a case–control design, a cohort design or a self-controlled case-series design. Studies that also used an instrumental variable analysis in addition to the standard cohort analysis were also included. The use of instrumental variables has been suggested as a possible alternative to conventional analyses when there is concern about the effect of unmeasured confounding [14,15]. Instrumental variable (IV) analysis attempts to mimic the process of randomisation in an RCT by exploiting the existence of another variable (the instrument) which can be measured in the available data, which is highly correlated with the probability of exposure but unrelated to the outcome of interest except through its association with treatment [16]. The instrument is similar to random arm assignment in that it should distribute both measured and unmeasured patient characteristics evenly between exposure groups resulting in an estimate less affected by confounding. The self-controlled case-series design compares the risk of adverse events in periods of exposure compared to non-exposure within the same person. This design attempts to limit the effects of major unmeasured confounders as the within-person study design controls implicitly for confounders that do not vary over time [17].

The final identified studies were grouped by outcome type and categorised according to the primary study medication comparison. Studies were rated according to a hierarchy of evidence of study designs [18] with meta analyses considered as the highest level of evidence, observational studies were considered as lower quality. Data were extracted including the study population, study size, study design, propensity for bias, procedures employed to minimise bias, a rating of study quality, duration of follow-up, outcome rate in the reference group, effect estimate (either rate ratio, odds ratio, hazard ratio or risk difference depending on the study design utilised) and appropriate 95% confidence intervals. The propensity for bias was assigned as ‘Low’ for RCTs and ‘Moderate’ for observational studies. Study quality was assigned as ‘High’ for RCTs. Observational studies were assigned a ‘Moderate’ quality rating if specific methods were employed to minimise bias otherwise studies were assigned a ‘Low’ quality if they made only minimal efforts to control for potential bias. Study designs such as the self-controlled case-series or instrumental variable analysis are not formally recognised in the hierarchy of evidence, however, we have considered these studies as they attempt to account for the common problem of unmeasured confounding in observational studies. The probable place of instrumental variable analysis in the hierarchy is either equivalent to or better than cohort studies. The place of the self-controlled case-series is as yet unclear.

RCT evidence for atypical antipsychotics showed an absolute increase of 1 extra death per 100 people treated. Collectively, observational evidence showed that conventional antipsychotics were associated with a greater risk of death than the atypical antipsychotics, however, the estimates of absolute risk differed between studies. Risk estimates from covariate adjusted cohort studies ranged from 2 to 3 extra deaths per 100 patients treated with conventional compared to atypical antipsychotics over 6months while instrumental variable analysis estimates ranged from 4 and 7 extra deaths per 100 patients over 6months. These discrepancies suggest that unmeasured confounding may have contributed to an underestimate of risk in the traditional cohort studies. Many of the cohort studies also employed propensity score methods to adjust for confounding either by use of propensity score matching [22,26] or numerical adjustment by propensity score quintiles [28,29,31]. In these latter studies propensity score adjustment estimates were consistent with multivariate adjusted results while instrumental variable analyses were marginally different suggesting some residual unmeasured confounding in the propensity score and multivariate adjusted analyses. Additionally studies have undertaken sensitivity analyses to rule out potential bias from unmeasured confounding which revealed that only very strong unmeasured confounders would explain the observed increased mortality association with conventional antipsychotic use, if in fact, the observed difference was not true [22,28].

RCT evidence of the risk of cerebrovascular events was limited to the atypical antipsychotics and demonstrated an increased risk of all cerebrovascular events[1] but not serious strokes requiring hospitalization [4,33]. Cerebrovascular events were the most studied and reported adverse event in observational studies of antipsychotics compared to non-use, however, the definition of this outcome was not consistent between studies. In general, cohort studies reported negative associations when investigating cerebrovascular hospitalisation events and positive associations when investigating outcomes defined as a diagnosis of all cerebrovascular events from general practitioners’ medical records which supports the findings from available RCTs. In contrast, case–control studies failed to find statistically significant results for either outcome definition. Case–control studies often employ techniques to minimise possible bias, such as matching or numerical adjustment for potential confounders, however, studies of this type may still be subject to unmeasured confounding [57]. Self-controlled case-series studies found similar results to RCT evidence when similar definition of cerebrovascular events were used which suggests that this technique may be a reliable design for the investigation of adverse events not previously detected in RCTs. One of the advantages of the self-controlled case-series design is that it controls implicitly for patient-specific confounders that do not vary over time. This means that it is not necessary to adjust for variables such as sex, frailty or other risk factors that are constant over time. However, a limitation of this approach is that it is unable to adjust for changes in prescribing due to rapid changes in underlying disease severity [17]. Observational cohort studies comparing the antipsychotic classes have used consistent definitions of outcome, specifically, hospitalisation for stroke, however, results vary according to the methods employed to adjust for differences between treatment groups. Of the 6 studies that used numerical covariate adjustment, three found no difference in risk [42-44] between the classes, one found a reduced risk [39], while two found an increased risk [33,45] with conventional compared to atypical antispychotics. One study [45], used an instrumental variable method to adjust for unmeasured confounding. This study found a 10% increased risk of stroke with conventional compared to atypical antipsychotics but only after 60days treatment, however, the instrumental variable estimates were not presented and the prevalence of cerebrovascular disease at baseline in the studied population, described in a separate paper, was high [29].

While no RCT data for the risk of hospitalisation for hip fracture could be located, a Cochrane review [1] found that risperidone may be associated with an increased risk of falls in the elderly which suggests that an increased risk of hip fracture may also be likely. Observational studies reported an increased risk of hip fracture with both classes compared to non-use and this risk may increase with increasing duration of therapy. A study using a self-controlled case-series design [50], found an increased risk of hospitalization for hip fracture for both conventional and atypical antipsychotics. It is unclear whether the risk of hip fracture differs between the classes. Only one cohort study [46] investigated the comparative risk of hip fracture between the classes finding an increased risk with conventional antipsychotics.

Meta-analyses [19-50,52-58] of randomized controlled trials found that one of the major causes of death associated with atypical antipsychotics was pneumonia. Few observational studies, however, have investigated the risk of pneumonia associated with antipsychotics in elderly patients, and most have used a case–control design. An increased risk of pneumonia was found with both classes [52,55] compared to non-use or previous use [53]. One case–control study identified that the risk appeared to be highest in the first week of treatment but returned to baseline after 90days [52]. A study using a self-controlled case-series design [50] also found and increased risk of hospitalization for pneumonia for both conventional and atypical antipsychotics. The risk of pneumonia was similar between the classes [46,54]. One study, which used an instrumental variable analysis, found no difference in the risk of pneumonia between conventional and atypical antipsychotics [45].

As in any systematic review, publication bias is a potential limitation of this study. It is possible that only positive observational study findings may have been published. In particular, the majority of published studies investigating hip fracture and pneumonia associated with antipsychotics reported significantly increased risks with treatment compared to non-use. Additionally, our search criteria specified that only those study designs that resulted in standard effect estimates were included for comparison purposes. We therefore excluded case reports and studies that used designs such as the self-controlled cohort analysis [59]. We only included outcomes that could be consistently defined to limit heterogeneity across studies. The endpoints for this study, cerebrovascular events, hip fracture and pneumonia, were chosen because we believed that patients were likely to present to hospital for these conditions, and therefore were outcomes that would most likely be available in observational studies using administrative claims databases. The application of methods such as the self controlled case-series and instrumental variable analyses have evolved mainly to address unmeasured confounding due to the lack of clinical information in claims databases. Antipsychotics have also been associated with other adverse events such as deep vein thrombosis, diabetes onset and heat stroke which were not included in this review.

The aim of this review was to explore the effects of study design on the results of observational studies of antipsychotics. To do this we simplified our comparisons to with-in class comparisons and we have not considered differences according to individual products, dose or effect modifiers. The majority of published observational studies have provided comparisons between classes, however, where available we have presented the results by individual products and by dose. Besides study design, many other factors may have contributed to the differences observed among observational studies. These include differences in populations studied, in terms of drug utilization patterns and health care settings, differences in the definition of outcome and variations in the number and type of confounders that may have been included in the analysis. We have attempted to limit the impact of these effects by considering the effects of antipsychotics in the elderly only and by limiting our events to serious adverse events potentially requiring hospitalization. In the case of cerebrovascular events we considered cerebrovascular events separately to hospitalizations for stroke as these were the two events most often considered in observational studies.