EMAc permits the monitoring of behaviors, moods, and cognitions in natural contexts and therefore provides important information that is largely inaccessible to standard laboratory self-report and interview measures.3,6,16
This powerful, ecologically valid measurement technology has rarely been used in investigations of severe psychiatric disorders. The purpose of the present study was to examine the feasibility and validity of EMAc in outpatients with schizophrenia. With regard to overall compliance, 2 people (less than 5%) declined participation, and only 13% of those who agreed to participate were noncompliant with the EMAc procedures. It is notable that 87% completed an average of over two thirds of the 28 programmed electronic questionnaires. These rates indicate that people with schizophrenia are willing to participate in EMAc protocols and complete enough assessments to examine both within- and across-day variation, while the compliance rate found in this study was somewhat lower than previous EMAc studies of nonpsychiatric (90%–96%)29–31
and higher functioning psychiatric samples (86%–92%).32,33
The number of completed questionnaires remained informative and adequate for data analysis. Moreover, no salient fatigue effects were observed over to course of the study, and feedback from the participants themselves indicated that EMAc was highly acceptable. Taken together, the compliance and acceptability findings suggest that EMAc methods are highly feasible for severe mental illnesses, such as schizophrenia. Several safeguards were nonetheless used in the present study to encourage compliance, such as telephone contacts during the data collection phase and monetary incentives for completing EMAc assessments. The number, type, or format of questions posed to participants may also need to be adapted depending on the clinical characteristics of the samples studied. The use of such procedures, however, should not substantially reduce the feasibility of this novel and powerful approach to assessment in real-world contexts.
The validity of EMAc data was examined in several ways. The relationships among EMAc variables were consistent with expectations (eg, increased stress perception was associated with decreased feeling happy). The presence of such expected patterns in EMAc responses was possible only if participants consistently rated the 2 variables in the opposite direction on Likert scale, a finding that confirms deliberate responding by the participants. Finally, significant moderate to strong associations were found when several laboratory self-report and interview instruments were used to predict variance in corresponding constructs measured by EMAc. In this way, EMAc appeared to measure the target construct (eg, functional behaviors, mood states, and psychotic symptoms) while still providing unique information concerning relationships between daily life contexts, behaviors, and experiences. Applications of data generated from EMAc are widespread, and relatively simple research protocols may provide rich information relative to daily life experiences. Concerning the present data, eg, variables assessed at any given assessment (t
) may be used to prospectively predict the onset or changes in psychotic symptoms in subsequent assessments (t
+ 1). Conversely, the presence of psychotic symptoms can be used to predict behavioral, social, or emotional consequences that occur over subsequent hours. The acquisition of data demonstrating real-time symptom expression provides a novel outcome for controlled clinical trials, and information concerning discrete functional behaviors provides the ecological validity that is currently lacking in this domain. From a clinical point of view, EMAc may also permit treatment advances through verification of medication compliance and the completion of desired exercises or the detection of early warning signs of relapse. The findings were consistent with the positive feasibility findings in the only other study that used EMAc in people with schizophrenia,17
but the present study also demonstrated that EMAc can be used with outpatients over a week of monitoring. The capacity of people with schizophrenia to independently use EMAc in nonsupervised conditions is of key importance for promoting research on functional assessment and the daily life expression of this disorder. These findings should, nevertheless, be interpreted relative to the characteristics of the participants and methodology used in this study. The feasibility of these methods may be decreased in patients with greater cognitive deficits, eg, as was demonstrated in the present sample. In examining the predictive validity of laboratory measures, only major or frequent outcomes and psychological experiences in daily life were examined, such as self-reported living skills, psychotic symptoms, mood, and perceived stress. Less frequent but important outcome variables, such as suicidal ideation or specific clinical events, may require larger sample sizes and possibly different assessment methods. It should also be considered that each EMAc electronic interview was designed to be brief and easy to complete. The content of all clinical laboratory questionnaires (eg, all ILSS items) used in the concurrent validity analysis could not be exactly duplicated in daily assessments due to the necessary brevity of EMAc. More detailed lengthy protocols may, therefore, induce greater fatigue effects or lower compliance rates with different samples. The high compliance rates found were also for a convenience sample that was already participating in a psychotherapy treatment outcome project with the researchers. Consequently, refusal and noncompliance rates may be higher for a more general community sample. Finally, with regard to sample characteristics, the participants were not preselected for having experience with electronic devices of any kind, so it is reasonable to assume that the findings should be generalizable to cohorts of the same or younger age. It is possible, however, that older samples may require different methods. For example, very old individuals (mean age ≥ 80 years) may be unable to use PDAs but can readily use cell phones if such applications involve direct communication with an investigator, rather than use of electronically coded response options.34
It is also important to note that this investigation provided 4 assessments per day and utilized fixed assessments for each participant (randomized across individuals) as part of a research program with specific scientific goals. Increasing the number of assessments per day may be more adapted for other objectives, such as time budget assessments of behavior and experiences in daily life. Concerning random vs fixed assessments, either may be justified concerning the goals of a given study. Should fixed assessments be used, however, investigators may wish to examine if the frequency of variables differs as a function of day of study and adjust statistical models for time-dependent effects that may reflect a participant's increasing ability to anticipate signals over the course of the investigation. Such potential reactive effects, as well as response rates in general, are also more readily assessed through computerized techniques than paper-based measures.15,35
Future investigations may benefit from considering these issues, while testing other EMAc assessments and techniques for consumer self-reporting, including those using cellular technology.