We attempted to replicate the CCF results but were unable to do so, even after several efforts to acquire information about specific details of CCF’s modeling choices from CCF. We first relied on their manuscript and supplement, but it gives an incomplete description of the model; for example, it was unclear how income data were imputed. We then contacted CCF to ask for the code they used to generate their results. They were unwilling to share this code.
We therefore wrote new code to analyze the AddHealth data to replicate their results; we sent it to them and asked them to comment. A brief response from CCF stated that a “difference in our approaches is that we ‘lock in’ Wave 1 friends—that is, we do not allow individuals to switch friends over time.” This is an important and noteworthy omission from their description in the paper since Add Health (like the FHS-Net data) contains dynamic information about friendships at later waves, yet they chose instead to ignore this information and rely on a static representation of that data. This will cause them to assume some individuals continue to be friends when, in fact, they are not. This assumption by CCF stacks the deck against finding an effect, since it essentially adds “random” non-friend relationships (i.e., people who are no longer friends) to the pool of friends.
Since the CCF results cannot be replicated, and since they are in any case inconsistent with the conclusions of TNP and HK, we present our own results here based on the Add Health data, in an effort to figure out what is going on. CCF use only the first friend named by each subject, but when we interact the friend list order with alter’s contemporaneous obesity, we cannot reject the hypothesis that order is irrelevant to the strength of the effect. We therefore use all observed friendships to maximize efficiency. CCF impute missing income and education data, but it is unclear how they do this. We use expectation maximization with importance sampling to impute all missing data (King et al. 2001
). This procedure is widely used and well known to reduce bias relative to alternatives such as listwise deletion (Rubin 2004
, King et al. 2001
). Like CCF, we also include variables for age, gender, race, ethnicity, and a fixed effect for the wave of the observation. And, as already noted, we use dynamic friendship data available from Waves 1, 2, and 3 to update friendships at each wave, rather than making the assumption made by CCF that friendships are static and all people who became friends in junior high school retain all of their relationships into adulthood.
presents the results. In model 1, notice that the coefficient of 0.033 on alter’s contemporaneous obesity is significant (p=0.02). In model 2, we add school trends as suggested by CCF. Not only does this addition have no effect on the induction effect, it also fails to achieve significance (CCF do not report the coefficient or standard error for their school trend effect, so we cannot compare results). However, what is clear is that the addition of school trends does not matter in a model that uses all available data.
Replication of Add Health Results Show School Trends Do Not Matter and Show Evidence of Directional Effects
CCF appear dismissive of evidence that the induction effect in our Framingham data is directional. That is, if Mark names John as a friend, we expect John to have an effect on Mark. However, if John does not reciprocate by naming Mark as a friend, then John may not be affected by Mark’s opinions or health behaviors. We hypothesize that influence flows from the named friend to the person who named him, but not necessarily vice versa, and in Framingham we find exactly that. The named friend has a significant effect on the namer but the namer has no significant effect on the named. New work in the econometrics of networks confirms that exploiting directionality in networks is a useful identification strategy (Bramoulle, Djebbari, and Fortin 2007). Fortunately, Add Health also allows us to test the directional hypothesis. In model 3 of , we show that there is no evidence that named friends are influenced by namers (p=0.90), confirming our results in the FHS-Net and providing additional evidence in favor of the causal interpretation regarding social influence in weight behaviors.
It is interesting to note that CCF did not report this result, since it speaks directly to their main point. If contextual effects are spuriously driving the relationship between ego and alter, then there is no reason to expect a directional result. The context should cause the named friend and the namer to move up and down simultaneously; hence, if we find a significant effect in one direction, we should also find it in the other: the named friend should appear to have an influence on the namer. Since we do not find such a significant effect, we believe the evidence in Add Health is suggestive of a causal effect, just as it is in Framingham. Obesity appears to spread from person to person.
CCF assert that our model specification in Framingham does not capture any contextual effects that vary across geographic space. It is important to note that friends in Add Health are all physically proximate (they are in the same school), whereas they are not in Framingham. If our estimates are biased because they capture community-level correlation, one implication is that the increased distance between friends will reduce the effect size (since distant social contacts are not contemporaneously affected by community-level variables). We specifically find that the relationship does not decay with physical distance, even up to hundreds of miles away (Fig. 3 in CF). We strongly emphasize this point in our article, and it was widely reported in the popular press, so it is difficult to understand how CCF could have missed this. The implication of this observation is that any contextual effects that are geographic in nature probably do not have an effect on the association between ego’s and alter’s obesity. In addition, as reported in CF, we also analyzed whether there was a relationship in weight behaviors between individuals and their immediate neighbors living at adjoining housing units. If contextual effects were driving the association in weight between social contacts, we would expect neighbors to appear to have an effect on each other, but we found that they do not.
Finally, in this regard, it is also worth noting that in Framingham we include dummy variables as controls for each exam, which effectively controls for average weight change in the whole population (time-specific effects). This might not capture regional variation within time blocks, but we also control for ego-specific factors that would account for some regional variation, including age, gender, and education.