Our results support the feasibility and validity of using ESMc for assessment of momentary psychotic symptoms, mood, and experiences among hospitalized individuals with schizophrenia. Our findings are particularly significant given the fact our subjects had no previous experience using PDAs. The ESMc methodology was equally acceptable to the patients with schizophrenia and healthy controls, with similar ratings of comfort carrying the PDAs, operating them, degree of interference with daily activities, as well as overall levels of stress and difficulties participating in the study. Equally important, both groups had similar response rates to the beeps.
The subjective nature of ESM data (e.g., thoughts, moods, mental states) makes the use of customary reliability and validity techniques problematic. The assessment of reliability is complicated by the fact that experiences assessed by ESMc, such as “I feel lonely”, may not necessarily have behavioral expressions, thus making them difficult to verify. As a result, the ascertainment of reliability can be obtained only by evaluation of the validity (Delespaul, 1995
). Once validity is demonstrated, reliability can be assumed. Delespaul (1995)
suggested a number of methods to validate raw ESM data including the use of face validity, comparison of aggregated data between distinct groups, correlations between similar and dissimilar items, as well as determining associations with available behavioral/external referents.
In accord with these recommendations, we found ESMc to be valid. The items used in our assessment were designed to maximize face validity by using everyday vocabulary. A comparison of ratings of symptoms and mood between the subjects in the schizophrenia and healthy control groups indicate differences in the expected direction. Individuals with schizophrenia reported significantly higher ratings of auditory and visual hallucinations, suspicious thinking, sense of unreality, lack of thought control, fear of losing control, difficulty expressing thoughts, as well as depression/sadness, lack of cheerfulness, and loneliness. Similarly, the negative correlations between dissimilar items, such as feeling cheerful and feeling stressed, irritated, and sad/depressed support the discriminant validity of ESMc. The trend in the relationship between suspiciousness ratings on SAPS and ESMc give additional support for the validity of ESMc ratings among hospitalized patients with schizophrenia. However, the number of participants is small and should be considered preliminary.
The finding that there were no significant differences in ratings of stress between the schizophrenia and healthy control groups is noteworthy. Stress is common among individuals with schizophrenia (Kimhy et al., 2004
; Steer et al., 2003
) and has long been assumed to be associated with psychosis. Our findings suggest that schizophrenia patients are not necessarily experiencing more stress, but rather their sensitivity to stress leads to exacerbation of psychotic symptoms.
Overall, we believe ESMc offers a number of advantages in research methodology. The most significant contribution of ESMc to research is in providing precise time-stamps for each response. By synchronizing the times of PDAs and ambulatory physiological measures, this feature makes it feasible to investigate interactions between psychological and neurobiological mechanisms. Thus, it offers exciting new ways to understand human behavior and functioning. In particular, this methodology has the potential to offer new insights into the mechanisms involved in psychopathology by studying the dynamic processes that govern the interactions between subjective experiences, neurobiology, and the environment. ESMc also limits the cost and labor associated with transcription and cleaning of data – (Nyholm et al., 2004
) reported on a study of 20 subjects in which the total time required for data entry from paper-diaries was 96 person–hours compared to less than 4 person–hours for PDA diaries. However, the authors did not report the amount of time spent programming the PDAs. It is also important to note that data entered on a PDA by a subject may not be necessarily less susceptible to data-entry errors compared to data entered by a trained data entry person. Furthermore, there is no way for researchers to verify the correctness of data entered by subjects using ESMc after the data was collected. Finally, ESMc offers advantages with regard to study design and data management. In particular, the ability to limit the response time available to participants as well as branching are important features.
Potential limitations of this study should be acknowledged. The sample size in the present study was small and the assessment period was shorter than what is typical in ESM schizophrenia studies. Thus, the results should be interpreted with caution. A number of factors may have contributed to the higher response rate with ESMc compared to ESMp. Studies of individuals with schizophrenia using ESMp typically assess subjects 10 times per day from 7:30 am to 10:30 pm over 6 days (60 beeps). In contrast, the assessment period in the present study lasted only one day (10 beeps) from 10:00 am to 10:00 pm. Delespaul (personal communication) found that the average response rate among 50 individuals with schizophrenia studied over six days using ESMp was 72% during the first day of assessment, with the response rate gradually declining to 61% at day four and stabilizing afterwards. However, the majority of the missed beeps occurred in the early morning and late night periods that were not sampled in the present study. Thus, the relatively higher response rate using ESMc may reflect the initial higher response rate associated with the novelty of participating in an ESM research protocol and/or the designation of an assessment period that is characterized by a higher response rate. Similarly, participants in our study were presented with 27–32 ESM questions per beep (depending on branching) compared to approximately 50 in typical ESMp studies with individuals with schizophrenia (Delespaul, 1995
; Myin-Germeys et al., 2000
), which may have influence the response rate. Additionally, the current study was carried out in an inpatient unit, whereas most ESMp studies were carried out among outpatient individuals with schizophrenia. Such environment may potentially increase the risk of PDAs having malfunctions or being damaged, lost, or stolen. However, Kwapil (personal communication) reported only two PDAs being broken, none lost/stolen, and limited technical malfunctions in an ESMc study among approximately 400 college students.
An important question is the feasibility to using ESMc with patients with severe psychotic symptoms. While this methodology will clearly not be suitable for sampling with some schizophrenia patients, the hospitalized schizophrenia patients who participated in this pilot study were quite symptomatic, with mean SAPS ratings of psychotic symptoms ranging from mild to moderate (see ), with many displaying marked and severe psychotic symptoms. Consistent with this data, Delespaul (1995)
and Myin-Germeys et al. (2001)
has demonstrated extensively that ESM assessments are not restricted to small sub-samples of relatively asymptomatic schizophrenia patients.
In summary, assessment of momentary psychotic symptoms, mood, and experiences using ESMc is feasible among hospitalized schizophrenia patients. The methodology was acceptable to patients and caused minimal disruption in their daily activities and routines. The data also provide preliminary support for the validity of using ESMc in this population. Future studies should aim to replicate our findings with larger samples. Similarly, the feasibility of sampling experiences of over longer periods of time should be firmly established. Finally, future studies should also establish the feasibility of using ESMc with schizophrenia patients with more severe symptoms. Overall, ESMc is novel and exciting methodology that allows studying the dynamic processes that govern the interactions between subjective experiences, neurobiology, and the environment. Such knowledge may improve our understanding of mechanisms of development and resolution of psychopathology.