PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Pharmaceut Med. Author manuscript; available in PMC 2009 December 22.
Published in final edited form as:
Pharmaceut Med. 2008 January 1; 22(2): 69–74.
PMCID: PMC2796846
NIHMSID: NIHMS132840

Electronic Diaries: Appraisal and Current Status

Abstract

The recent explosion of technology has moved the field of patient reported outcomes (PROs) into a new era. Use of paper-and-pencil questionnaires administered before and after treatment has been eclipsed by highly sophisticated random prompts for symptom ratings at multiple points throughout the day, a method known as ecological momentary assessment (EMA). During the last 25 years, research has demonstrated that retrospective ratings are subject to a variety of cognitive heuristics that can distort the report. Initially, this was addressed by adopting paper diary protocols involving multiple ratings in a day or across a week. Technology was also advancing, and some researchers began to utilize electronic platforms for EMA assessment. A good deal of research has been conducted comparing paper and electronic formats. Issues of compliance have been particularly problematic for paper diaries. Electronic technologies can be expensive and require expertise in programming and data management. Not all research questions will require intensive momentary assessment, and end-of-day ratings may be adequate for many applications. What is required of the investigator is familiarity with the strengths and weaknesses of the methods and platforms available as well as a reasoned decision to elect a particular methodology for the study question at hand.

I. Historical roots of diaries

The historical roots of diaries, as a form of data collection go back to at least the early part of the 20th century. In order to better understand the etiology and course of symptoms, patients were asked to keep an on-going symptom record that could be reviewed by the physician.[1] However, it was not until later in the century that the method gained scientific attention. Csikszentmihalyi, while at the University of Chicago, developed the “Experience Sampling Method” during his seminal work on the “flow” of personal experience.[2] Rather than limiting the characterization of experience to a single rating or set of ratings provided retrospectively, Csikszentmihalyi asserted that the unfolding of experience on a moment-to-moment basis contained unique information not available from traditional self-report methods. At the same time, the blossoming of the Skinnerian behavioral movement in psychology in the 1970's viewed human behavior as the result of learned experiences happening at the moment in the person's environment. Consequently, behavioral observation of momentary events and reactions by an individual were viewed as theoretically and methodologically superior to traditional questionnaire assessment.[3] Shortly after that, psychologists in Europe and the United States began to apply this new approach in a variety of studies, such as schizophrenia,[4] the relationship between stress and onset of physical illness symptoms,[5] and daily experiences and mood.[6] The term, “Ecological Momentary Assessment (EMA),” became widely adopted in the United States to refer to methods involving multiple random assessments within a day.[7] Initially, diary studies gathered observations using paper-and-pencil diaries. However, once hand-held computers became widely available and relatively inexpensive, electronic diaries began to be used.

II. Traditional self-report questionnaires versus momentary assessment

From a practical and economic point of view, traditional baseline and post-treatment assessment of symptomatology with a questionnaire is by far the easiest. Many self-report instruments have a long history of use in clinical trials and good reliability and validity data associated with them. They are inexpensive for the researcher to use and involve minimal burden on the patient. So, why has there been a movement to more burdensome diaries? The most important reason stems from accumulating evidence of bias in retrospective questionnaire ratings. Most assessment instruments have recall periods that range from days to weeks or months. Thus, patients are being asked to generate a summary rating of patient reported outcomes (PROs), such as pain, fatigue, sleepiness, or nausea over some period of time. This recall task assumes that the patient has access to all of the relevant symptom experiences over the recall period and is able to aggregate them to generate an accurate representation of their symptom experience. Emerging research suggests that both of these assumptions are faulty.[8, 9] Moreover, there is evidence that as patients attempt to remember and report their symptoms, cognitive heuristics become operative and result in distorted reports.[10] Diaries that ask the patient to simply report on his experience at the moment theoretically avoid these threats to the validity of the assessment.[11]

While diaries can avoid recall bias, the high frequency of some momentary assessment protocols has raised concern about reactive effects. Does the process of repetitive monitoring and reporting alter the phenomenon being assessed? A few studies have examined this question and have failed to uncover evidence of reactivity.[12, 13] However, there may well be instances in which reactivity could take place. Likewise, the issue of stereotypic responding as the protocol extends over long time periods needs to be investigated.

III. Cognitive Heuristics

When patients are recalling experiences retrospectively, research has shown that several cognitive heuristics can come into play even over short recall periods (see Fredrickson for review[14]). Two heuristics are known as the “peak” and “end” effects. Redelmeier and Kahneman conducted a study with patients who remained awake during a colonoscopy or lithotripsy.[15] Patients rated their pain throughout the procedure, and at the end of the procedure they provided an overall rating of their pain. Analyses indicated that the recalled pain was not a simple arithmetic average of the pain, but instead was more heavily weighted by the peak pain and the pain during the last three minutes of the procedure. A second study by these investigators randomly assigned colonoscopy patients to a usual colonoscopy procedure or to one in which at the end of the procedure, the colonoscope was left in the rectum without moving for several minutes – thus producing no pain.[16] Despite the fact that the second condition was longer, patients rated the colonoscopy as less aversive compared with the shorter, usual procedure. Moreover, those patients indicated that they were significantly more willing to undergo another colonoscopy at a future date. Thus, the “end” pain had a significant impact on patients' ratings. We examined these phenomena in naturalistic, prospective studies of patients with chronic pain. In the first study of rheumatoid arthritis patients who completed up to 7 pain ratings a day for 7 days, the combined variable of peak and end pain from momentary ratings corresponded significantly greater to the 7-day recall at the end of the week compared with the mean of all momentary reports.[17] In a second study, using data collected over two weeks by patients with chronic pain, we also found evidence that one week recall of pain was over-weighted by the peak of pain during that week. We also observed that patients with more variability in their momentary pain gave higher recall pain ratings, even though their mean levels were not different from those with less variability.[18] Other researchers studying other populations have observed similar outcomes.[19, 20]

These findings demonstrate that retrospective ratings are the product of multiple selected aspects of the experience being reported. Robinson and Clore have written eloquently about the types of memory accessible across different time frames.[21] When patients cannot access in memory some or all of the specific experiences being assessed, there is evidence that they rely upon general beliefs or knowledge about their experiences. Even though beliefs at times may be generally good estimates of experience during a targeted interval, some situations may evoke distortion. Of relevance to clinical trials is the finding that patients who want to believe that a treatment was helpful may distort their memory of baseline status to yield an exaggerated view of improvement. Two studies have demonstrated that at post-treatment ratings of pre-treatment status can be recalled as worse than they actually were, thus increasing the apparent effectiveness of the treatment.[22, 23]

IV. Paper or electronic diaries?

Currently, there is controversy surrounding the choice of paper or electronic diaries. Paper diaries preceded the technological advancements that led to the availability of small, hand-held computers to use as electronic diaries. The advantages of paper diaries include inexpensive assembly and little training required for patients in their use. A major disadvantage that has become apparent from studies in recent years is compliance with the timing of completion of paper diaries. Two studies have found significant non-compliance with paper diaries over a three-week recording period in samples of patients with chronic pain.[24, 25] In the first study with a sampling density of three fixed-time assessments per day, compliance with paper was directly compared with electronic diaries in patients with chronic pain randomized to one of the two data collection methods. The paper diaries were surreptitiously imbedded with a microchip that could determine when the diary was opened to make an entry. Although patients indicated on the paper diary cards that they completed the entry at the scheduled assessment 90% of the time, in fact, the actual compliance was only 11% (allowing a ± 15 minute response window). When the compliance window was expanded to ± 45 minutes, the verified compliance was 20%. Patients recorded entries dated for more than 90% of the days, yet the electronic chip determined that the binder was not even opened on 32% of the study days, indicating that patients filled out the ratings for those days on another day. Moreover, our data found evidence of both backfilling as well as forward filling of symptom ratings. These outcomes were in contrast to 94% verified compliance in the patients randomized to the electronic diary condition.[25] Our second study from the same pool of patients attempted to improve paper diary compliance by conducting the same protocol and adding a watch with pre-set alarms to remind the patient to complete the diary at each of the three assessment times during the day. As in the first study, the self-reported compliance was over 85%, however, the verified compliance was 29% for the ± 15 minute response window and 39% for the ± 45 minute response window. Thus, signaling did provide some improvement in compliance. Nevertheless, less than half of the patients opened the paper diary at least once every day of the 21 days of the study. Furthermore, verified compliance dropped significantly after the first week of symptom ratings.[24]

To date, these are the only studies that have empirically examined compliance with paper diaries. The results have been sobering and have stimulated a spirited dialogue.[26] Other researchers have presented data to suggest that compliance with paper diaries might be higher, but the evidence is indirect and unverified.[27] A noteworthy commentary by Tennen and colleagues suggested that acceptable compliance with paper diaries is probably possible in some samples and under some circumstances.[28] The research participant's motivation for being in the study, the perceived importance of the study, the collaboration established between researcher and participant, the characteristics of preparation and training to provide the ratings, and the length and burden of the questions are likely important determinants of how successful participants will be in adhering to the assessment protocol. Nevertheless, without having in place some procedures for verifying when questions were answered, the researcher will have to accept on faith participants' self-report of compliance.

A recent systematic review of nine health-related studies comparing data collected by paper and electronically examined the feasibility, compliance, data accuracy, and respondent preference for the two methods.[29] Five of the nine studies reported occasional technical difficulties with the electronic diary, most often battery/power problems as well as software or hardware malfunctions. Two studies monitored the required hours for data entry and found that the electronic diary took only a fraction of the time compared with paper (e.g., 4 hours versus 96 hours[30]). However, appropriately the review's authors commented that the costs of electronic hardware as well as the time for software programming and uploading and downloading of data on the electronic units can be substantial. There was a clear pattern across most studies indicating less missing data and fewer entry errors in the electronic version compared with paper. Four of the nine studies reported evidence that patients were falsifying paper entries, whereas this was precluded in the electronic version. Finally, patients rated both methods easy to use, but there was a clear trend toward preference of the electronic method.

There are also some instances in which use of electronic data capture may be preferable to paper based on the respondent's view of confidentiality of the ratings. Turner and colleagues found that adolescents reported more sexual behavior in a computer assessment than in a paper version.[31]

V. Beyond personal data assistants

Of course, as the technology advances, so does the capacity of electronic diaries to conduct highly sophisticated assessment protocols involving branching of queries based upon an initial response (e.g., presence of headache), context-driven assessments (work versus home), and biological state (glucose level, blood pressure, or alcohol intoxication). The platform for electronic data capture is also expanding. Cell phones and smartphones (e.g., BlackBerry) are being used to signal and record patient reports as are interactive voice recorded computer/telephone systems,[32-34] including direct voice capture of responses.[35] Finally, with rapidly increasing access to the internet across the socioeconomic continuum, many patients can complete diary ratings on websites provided to them.

VI. Are multiple daily assessments collected electronically always necessary?

The primary rationale underlying momentary assessment is to collect a patient report that is not distorted by recall bias or memory deficit. As mentioned above, some questions require precise collection of ratings in order to investigate the relationship between phenomena that change across hours. Such studies require a high resolution of the PROs being investigated. Furthermore, they require random sampling of experiences in order to prevent reporting of selected experiences due to their salience, meaningfulness, or distress to the respondent. Non-random, selected reporting could easily undermine the representativeness of the data being collected. With higher levels of sampling density of ratings, the possibility of examining prospective relationships between variables is created. This can be done using time series or cross-lagged panel analyses of experiences at Time 1 predicting experiences at Time 2. Examples of such studies include determining the effect of exercise on pain,[36] the associated characteristics and course of migraine headaches,[37] the relationship between activity level and onset of pain,[38] emotional reactivity and depressed mood,[39] the relationship between mood and pain,[40] and the antecedents of cigarette smoking.[41] These studies have yielded data that have confirmed existing conceptualizations (e.g. the course and associated symptoms of migraine headaches[37] and also have challenged them generating new insights into the variables being studied (e.g., antecedents of smoking a cigarette.[41]

There are other applications that may tolerate a less intensive sampling density of assessments, such as asthma attacks. When infrequent events are being assessed or when lagged relationships are not the focus, end-of-day reports may suffice for a variety of PROs.[42, 43] These may be collected via paper questionnaire and mailed, and they may yield adequate compliance when next-day postmarks are monitored. Although this method does not guarantee that the questions were answered on the correct day, it does substantially reduce hoarding and backfilling problems. Alternatively, end-of-day electronic assessments can be used with automatic time and date-stamping of the entry. Our laboratory has just completed a study examining the relationship between six daily momentary ratings of pain and of fatigue and end-of-day ratings (both collected electronically) across 28 days for patients with chronic rheumatological conditions, and we found that an end-of-day rating provides a good representation of these symptoms for the day. Finally, there may be instances when end-of-day reporting is preferable to momentary assessment, as in cases where episodes of infrequent events are targeted and could be missed by momentary ratings or the burden of frequent assessment would not be justified.

Apart from literacy problems and substantial cognitive or physical deficits, diaries can be completed by most individuals. Diary data have been conducted successfully in children[44] and in older, chronically ill adults.[12, 40] Even patients with serious illness, such as cancer patients undergoing chemotherapy, are able to provide diary assessments.[45] Concerns about older, computer-naive adults being unable to learn the diary task are largely unfounded. The evidence suggests that they are able to comfortably engage in diary methodologies using paper and electronic means.

The particular questions being addressed by a study and the natural course of the symptoms or behavior being monitored should be the determinants of the assessment protocol. Interested readers can find guidelines for the design of momentary assessment protocols in several review articles.[42, 46-48]

VII. Analysis of diary data

Another important consideration for studies using diaries that generate multiple within-person measurements is the issue of data analysis. Although in some cases it may be appropriate to aggregate across ratings to create a single, mean rating for the reporting period, in many cases the complexity of the data issues will dictate a more sophisticated approach using multi-level modeling. Several reviews of these methods are available.[42, 49, 50] Given the multiple ratings generated by diaries, the reliability of the construct being measured and the variability across time can be examined in ways not available in traditional single-point assessment methods.

VIII. Conclusion

The methodological literature exploring the characteristics of different methods of collecting PROs has increased in recent years, primarily due to the widely expanded options afforded by new technology. Much of the debate has been cast as a horse race across methods that leads to conclusions framed in terms of “winners” and “losers.” The question is not, which is the superior method? Rather the researcher needs to become acquainted with the factors that are at play in the methods that are available. Different methods bring different strengths and weaknesses including patient burden, researcher burden, cost, and compliance. Moreover, the phenomena and inter-relationships being examined will necessitate some method characteristics, while allowing leeway in others. Highly sensitive information may benefit from the impersonal and protected nature of electronic assessment compared with paper diaries or personal interviews.[51] PROs of chronic, stable conditions may be well-served by paper or electronic end-of-day assessments. What has become clear is that simply asking patients to provide PROs in a particular manner does not guarantee that it will be accomplished according to protocol. The extent to which these deviations may impact the study question must be evaluated in each case. It is also important to bear in mind that employing electronic methods does not guarantee the validity of the measurement. A sampling density that is insufficient may miss important events. And, electronic methods are still self-report and may be poor indicators of some phenomena compared with physiological monitoring (e.g., hot flashes[52]). In all cases, the selection of an assessment methodology should be thoughtful and informed, and the rationale for the chosen method should be outlined in study reports.

Acknowledgements

This writing of this paper was supported in part by an investigator-initiated grant from the National Institutes of Health (1 U01-AR052170-01; Arthur A. Stone, principal investigator). The NIH has neither been involved in the design or interpretation of our research, nor in the preparation or review of this manuscript.

Footnotes

Conflict of interest disclosure: the author has a financial interest in invivo data, inc, Pittsburgh, PA, a company providing electronic data services to the pharmaceutical community. There were no other contributors to this paper.

References

1. Favill D, Rennick CF. A case of family periodic paralysis. Archives of Neurology and Psychiatry. 1924;11:674–9.
2. Csikszentmihalyi M, Larsen R. Validity and reliability of the Experience-Sampling Method. Journal of Nervous and Mental Disease. 1987;175:526–36. [PubMed]
3. Barker R. Habitats, environments and human behavior: Studies in the ecological psychology and ecobehavioral science of the Midwest Psychological Field Station: 1947-1972. Jossey-Bass; San Francisco: 1978.
4. Delespaul P, De Vries MW. The daily life of ambulatory chronic mental patients. Journal of Nervous & Mental Disease. 1987;175(9):537–44. [PubMed]
5. Stone AA, Reed BR, Neale JM. Changes in daily event frequency precede episodes of physical symptoms. Journal of Human Stress. 1987 Summer;13(2):70–4. [PubMed]
6. Stone AA, Neale JM. Effects of severe daily events on mood. Journal of Personality and Social Psychology. 1984 Jan;46(1):137–44. [PubMed]
7. Stone A, Shiffman S. Ecological momentary assessment (EMA) in behavioral medicine. Annals of Behavioral Medicine. 1994;16:199–202.
8. Bradburn N, Rips L, Shevell S. Answering autobiographical questions: The impact of memory and inference on surveys. Science. 1987;236:157–61. [PubMed]
9. Schwarz N. Retrospective and concurrent self-reports: The rationale for real-time data capture. In: Stone AA, Shiffman S, Atienza AA, Nebeling L, editors. The science of real-time data capture. Oxford University Press; New York: 2007.
10. Gorin AA, Stone AA. Recall biases and cognitive errors in retrospective self-reports: A call for momentary assessments. In: Baum A, Revenson T, Singer J, editors. Handbook of Health Psychology. Erlbaum; Mahwah, NJ: 2001. pp. 405–13.
11. Robinson M, Clore G. Episodic and semantic knowledge in emotional self-report: Evidence for two judgment processes. Journal of Personality and Social Psychology. 2002;83:198–215. [PubMed]
12. Stone AA, Broderick JE, Schwartz JE, Shiffman S, Litcher-Kelly L, Calvanese P. Intensive momentary reporting of pain with an electronic diary: reactivity, compliance, and patient satisfaction. Pain. 2003 Jul;104(12):343–51. [PubMed]
13. Hufford MR, Shields AL, Shiffman S, Paty J, Balabanis M. Reactivity to ecological momentary assessment: an example using undergraduate problem drinkers. Psychology of Addictive Behaviors. 2002 Sep;16(3):205–11. [PubMed]
14. Fredrickson B. Extracting meaning form past affective experiences: The importance of peaks, ends, and specific emotions. Cognition and Emotion. 2000;14(4):577–606.
15. Redelmeier DA, Kahneman D. Patients' memories of painful medical treatments: real-time and retrospective evaluations of two minimally invasive procedures. Pain. 1996;66(1):3–8. [PubMed]
16. Redelmeier D, Katz J, Kahneman D. Memories of colonoscopy: a randomized trial. Pain. 2003;104:187–94. [PubMed]
17. Stone A, Broderick J, Kaell A, DelesPaul P, Porter L. Does the peak-end phenomenon observed in laboratory pain studies apply to real-world pain in rheumatoid arthritics? Journal of Pain. 2000;1(3):212–7. [PubMed]
18. Stone AA, Schwartz JE, Broderick JE, Shiffman S. Variability of momentary pain predicts recall of weekly pain: A consequence of the peak (or salience) memory heuristic. Personality and Social Psychology Bulletin. 2005;31(10):1340–6. [PubMed]
19. Eich E, Reeves JL, Jaeger B, Graff-Radford SB. Memory for pain: relation between past and present pain intensity. Pain. 1985;23(4):375–80. [PubMed]
20. Collins L, Graham J, Hansen W, Johnson C. Agreement between retrospective accounts of substance use and earlier reported substance use. Applied Psychological Measurement. 1985;9:301–9.
21. Robinson M, Clore G. Belief and feeling: Evidence for an accessibility model of emotional self-report. Psychological Bulletin. 2002;128(6):934–60. [PubMed]
22. Linton SJ, Melin L. The accuracy of remembering chronic pain. Pain. 1982;13:281–5. [PubMed]
23. Ross M. The relation of implicit theories to the construction of personal histories. Psychological Review. 1989;96:341–57.
24. Broderick J, Schwartz J, Shiffman S, Hufford M, Stone A. Signaling does not adequately improve diary compliance. Annals of Behavioral Medicine. 2003;26(2):139–48. [PubMed]
25. Stone A, Shiffman S, Schwartz J, Broderick J, Hufford M. Patient compliance with paper and electronic diaries. Controlled Clinical Trials. 2003;24(2):182–99. [PubMed]
26. Broderick JE, Stone AA. Paper and electronic diaries: Too early for conclusions on compliance rates and their effects--Comment on Green, Rafaeli, Bolger, Shrout, and Reis (2006) Psychological Methods. 2006 Mar;11(1):106–11. discussion 23-5. [PubMed]
27. Green AS, Rafaeli E, Bolger N, Shrout PE, Reis HT. Paper or plastic? Data equivalence in paper and electronic diaries. Psychological Methods. 2006;11(1):87–105. [PubMed]
28. Tennen H, Affleck G, Coyne JC, Larsen RJ, Delongis A. Paper and plastic in daily diary research: Comment on Green, Rafaeli, Bolger, Shrout, and Reis (2006) Psychological Methods. 2006 Mar;11(1):112–8. discussion 23-25. [PubMed]
29. Dale O, Hagen KB. Despite technical problems personal digital assistants outperform pen and paper when collecting patient diary data. Journal of Clinical Epidemiology. 2007 Jan;60(1):8–17. [PubMed]
30. Nyholm D, Kowalski J, Aquilonius SM. Wireless real-time electronic data capture for self-assessment of motor function and quality of life in Parkinson's disease. Movement Disorders. 2004 Apr;19(4):446–51. [PubMed]
31. Turner CF, Ku L, Rogers SM, Lindberg LD, Pleck JH, Sonenstein FL. Adolescent sexual behavior, drug use, and violence: increased reporting with computer survey technology. Science. 1998 May 8;280(5365):867–73. [PubMed]
32. Anhoj J, Moldrup C. Feasibility of collecting diary data from asthma patients through mobile phones and SMS (short message service): response rate analysis and focus group evaluation from a pilot study. Journal of Medical Internet Researcj. 2004 Dec 2;6(4):e42. [PMC free article] [PubMed]
33. Weiler K, Christ AM, Woodworth GG, Weiler RL, Weiler JM. Quality of patient-reported outcome data captured using paper and interactive voice response diaries in an allergic rhinitis study: is electronic data capture really better? Annals of Allergy Asthma Immunology. 2004 Mar;92(3):335–9. [PubMed]
34. Lauritsen K, Degl' Innocenti A, Hendel L, Praest J, Lytje MF, Clemmensen-Rotne K, et al. Symptom recording in a randomised clinical trial: paper diaries vs. electronic or telephone data capture. Controlled Clinical Trials. 2004 Dec;25(6):585–97. [PubMed]
35. Levin E, Levin A. Evaluation of spoken dialogue technology for real-time health data collection. Journal of Medical Internet Research. 2006;8(4):e30. [PMC free article] [PubMed]
36. Focht BC, Ewing V, Gauvin L, Rejeski WJ. The unique and transient impact of acute exercise on pain perception in older, overweight, or obese adults with knee osteoarthritis. Annals of Behavioral Medicine. 2002 Summer;24(3):201–10. [PubMed]
37. Honkoop PC, Sorbi MJ, Godaert GL, Spierings EL. High-density assessment of the IHS classification criteria for migraine without aura: a prospective study. Cephalalgia. 1999 May;19(4):201–6. [PubMed]
38. Liszka-Hackzell JJ, Martin DP. An analysis of the relationship between activity and pain in chronic and acute low back pain. Anesthesia and Analgesia. 2004 Aug;99(2):477–81. [PubMed]
39. Cohen LH, Gunthert KC, Butler AC, O'Neill SC, Tolpin LH. Daily affective reactivity as a prospective predictor of depressive symptoms. Journal of Personality. 2005 Dec;73(6):1687–713. [PubMed]
40. Tennen H, Affleck G, Zautra A. Depression history and coping with chronic pain: a daily process analysis. Health Psychology. 2006 May;25(3):370–9. [PubMed]
41. Shiffman S, Gwaltney CJ, Balabanis MH, Liu KS, Paty JA, Kassel JD, et al. Immediate antecedents of cigarette smoking: an analysis from ecological momentary assessment. Journal of Abnormal Psychology. 2002 Nov;111(4):531–45. [PubMed]
42. Bolger N, Davis A, Rafaeli E. Diary methods: capturing life as it is lived. Annual Review of Psychology. 2003;54:579–616. [PubMed]
43. Feinn R, Tennen H, Cramer J, Kranzler HR. Measurement and prediction of medication compliance in problem drinkers. Alcoholism, clinical and experimental research. 2003 Aug;27(8):1286–92. [PubMed]
44. Palermo TM, Valenzuela D, Stork PP. A randomized trial of electronic versus paper pain diaries in children: impact on compliance, accuracy, and acceptability. Pain. 2004 Feb;107(3):213–9. [PubMed]
45. Booth CM, Clemons M, Dranitsaris G, Joy A, Young S, Callaghan W, et al. Chemotherapy-induced nausea and vomiting in breast cancer patients: a prospective observational study. Journal of Supportive Oncology. 2007 Sep;5(8):374–80. [PubMed]
46. Hufford M. Special methodological challenges and opportunities in ecological momentary assessment. In: Stone AA, Shiffman S, Autienza A, Nebeling L, editors. The science of real-time data capture: Self-reports in health research. Oxford University Press; New York: 2007.
47. Palmblad M, Tiplady B. Electronic diaries and questionnaires: designing user interfaces that are easy for all patients to use. Quality of Life Research. 2004 Sep;13(7):1199–207. [PubMed]
48. Stone AA, Shiffman S. Capturing momentary, self-report data: a proposal for reporting guidelines. Annals of Behavioral Medicine. 2002 Summer;24(3):236–43. [PubMed]
49. Affleck G, Zautra A, Tennen H, Armeli S. Multilevel daily process designs for consulting and clinical psychology: A preface for the perplexed. Journal of Consulting & Clinical Psychology. 1999;67(5):746–54. [PubMed]
50. Schwartz JE, Stone AA. The analysis of real-time momentary data: A practical guide. In: Stone AA, Shiffman S, Atienza AA, Nebeling L, editors. The science of real-time data capture. Oxford University Press; New York: 2007.
51. Shrier LA, Shih MC, Beardslee WR. Affect and sexual behavior in adolescents: a review of the literature and comparison of momentary sampling with diary and retrospective self-report methods of measurement. Pediatrics. 2005 May;115(5):e573–81. [PMC free article] [PubMed]
52. Carpenter JS, Monahan PO, Azzouz F. Accuracy of subjective hot flush reports compared with continuous sternal skin conductance monitoring. Obstetrics and Gynecology. 2004 Dec;104(6):1322–6. [PubMed]