|Home | About | Journals | Submit | Contact Us | Français|
To the Editors:—I thank Drs. Jena, Sun and Goldman for applying an interesting falsification method to investigate the observed association between PPI use and incident community-acquired pneumonia.1 The authors advocate their method as “an intuitive approach to assessing the likelihood of confounding in observational studies” and as perhaps useful “for validating the causal plausibility of observational studies more generally.” I wish to recast the authors’ analysis in a form better aligned with modern concepts of causal inference, clarifying the underlying intuition and suggesting a suitable role for their method in the process of model-based data analysis.
Jena and colleagues begin with a linear probability model in which, despite multiple adjustments, an association persists between PPI fills and CAP diagnosis. Concerned that residual confounding makes their model incorrectly specified to identify the causal effect of PPI on CAP, the authors undertake an interesting demonstration. From their model, they obtain a set of falsification models by substituting “multiple, prevalent diseases” in place of the original regressand, CAP. Importantly, these are diseases for which there exists a strong prior expectation of no effect. Thus, when the falsification models discover implausible PPI effects, they reveal themselves to be misspecified. The authors’ leap, however, is to infer that the original model is also likely misspecified. Formulated simply, the authors’ intuition might be: a model correctly specified to identify PPI effects on CAP should also correctly identify (as zero) PPI effects on similar conditions for which no effect exists.
While by no means obviously correct, a heuristic of this kind may well prove fruitful if it spurs deeper, more explicit thinking about causal relations2 as they inform the meaning of similar. I venture to assert that any regressand validly substitutable under this principle must also be substitutable in a directed acyclic graph3 of relevant causal relations. On this account, the authors’ choices of several substituted conditions may be questioned. Osteoarthritis, as a degenerative disease, must be causally dissimilar from an episodic illness such as CAP. Chest pain, unlike CAP, may itself exhibit causal effects on PPI fills, through both empiric prescribing and elective refills by patients.
Falsification methods of this kind appear to speak more directly to model misspecification than to the validity of interpreting observed associations as causal effects. They are therefore best viewed as accessory to the model-building phase of data analysis and not as a concluding or validating step of inquiry.