The measurement of patient behaviors in bariatric surgery research has relied largely on retrospective self-report measures such as questionnaires and clinical interviews, which depend on patients’ ability to recall behaviors, experiences, and/or environmental conditions over periods ranging from days to years. While it is assumed that patients are able to give an accurate report, this assumption has been rarely tested. A growing body of evidence suggests that individuals are highly accurate at recalling information about unique events (e.g., “the day I had surgery”), but highly inaccurate at recalling information about individual experiences that are “frequent, mundane, and irregular” (e.g., episodes of eating, physical activity, chronic pain, use of medication).(12)
Given that behavioral researchers are frequently interested in the latter, and rarely interested in the former, these inaccuracies are, or should be, cause for concern.
The inaccuracies of retrospective self-reports are attributable to a combination of simple forgetting, a variety of mental “shortcuts” that individuals use to reconstruct their memory when forgetting has occurred, and a desire to please their health care providers by reporting “good behavior.”(12-14)
For example, bariatric surgery patients may be asked about the frequency, size, and composition of their meals during follow-up visits with their surgeon and surgical support team (e.g., nurses, nutritionists). However, most individuals cannot remember the details of every meal and snack eaten within the last few days or weeks. Notably, when asked to recall their pattern of eating, patients will rarely give the accurate answer (i.e., “I can’t recall exactly.”). Instead, they will make assumptions about what their behavior must have been
in order to produce an answer.(12, 15)
The assumptions that are used to reconstruct memory are susceptible to bias. For example, Consistency Bias
may lead a patient is likely to assume that their past behavior is essentially the same as their current behavior, and so they will give a report about their behavior during the last month that actually only reflects their behavior during the most recent few days. Other examples of bias that negatively affect retrospective self report measures include Mood Congruent Memory Bias
(i.e., patients are more likely to recall positive information when happy and negative information when unhappy), and the Peak Effect
(i.e., patients are more likely to remember and generalize events of high intensity). For a more extensive discussion of biases that influence retrospective recall, see Schwarz, 2007 or Gorin & Stone, 2001.(13, 15)
Ecological validity is another cause for concern associated with retrospective self-report measures.(13-14, 16)
Behavioral information that is collected in an artificial environment such as a research laboratory or physician’s office may not generalize to an individual’s natural environment. For example, as described in detail below, there is a substantial discrepancy between self-reported levels of PA obtained from bariatric surgery patients in a research setting, and levels of PA in the same individuals measured objectively via accelerometry in the natural environment.(17)
Problems with generalizability are a well-known threat to the external validity of research findings, but like the other types of bias described above, this threat is often ignored by researchers and clinicians.
The problems associated with using self-report measures to assess health behaviors have been demonstrated empirically in a variety of studies. For example, studies using doubly-labeled water have shown that dietary recall measures are susceptible to systematic underreporting that is influenced by individual difference characteristics (e.g., body size, body dissatisfaction, dietary restraint, ethnicity).(18-20)
Pencil-and-paper food diaries are subject to many of the same problems, partially because individuals tend not to complete them in real-time, as instructed.(21)
Levels of adherence to self-care behaviors such as blood-glucose monitoring(22)
and use of prescription medication(23)
also differs substantially when measured objectively versus by self-report. Discrepancies between objective and self-report measures have also been demonstrated in bariatric surgery patients. For example, we have shown that accelerometer-determined changes in moderate-to-vigorous physical activity (MVPA) from pre- to 6-months postoperatively are much smaller than self-reported changes.(17)
Biased information collected via retrospective self-report measures may lead researchers to invalid conclusions that could influence postoperative outcomes. For example, a current controversy in the field of bariatric surgery concerns the role of binge eating in surgical weight loss outcomes. Early studies suggested that reports of preoperative binge eating may be associated with poor weight loss outcomes (24)
, while a more recent study indicates that patients with and without preoperative binge eating achieve similar weight losses at least within the first year after surgery.(25)
The mixed conclusions of these studies may be due to inconsistent and/or inadequate assessment of binge eating behavior in this population. Thus, having valid and reliable data on patient behaviors is essential to resolving such controversies and developing appropriate behavioral guidelines for pre- and postoperative care. Recent advances in technology show great promise for addressing the limitations inherent in retrospectively measuring patients’ physical activity and eating behaviors, and assisting with patient care, as described in the following sections.