Semi-structured interview scales for psychosis are the gold standard approach to assessing psychotic and other symptoms. However, such assessments have limitations such as recall bias, averaging, insensitivity to change and variable interrater reliability. Ambulant, real-time self-report assessment devices may hold advantages over interview measures, but it needs to be shown that the data thus collected are valid, and the collection method is acceptable, feasible and safe. We report on a monitoring system for the assessment of psychosis using smartphone technology. The primary aims were to: i) assess validity through correlations of item responses with those on widely accepted interview assessments of psychosis, and ii) examine compliance to the procedure in individuals with psychosis of varying severity.