We applied machine learning to signals from mobile phones to classify the activities of people with PD. Instead of handpicking the most relevant features and comparing them, we used a large feature set and had the relevant features selected by the machine learning algorithms. Because many algorithms depend on the training data, these methods were not expected to test well for populations with unique movement patterns. This was done using mobile phones carried naturally in pants pockets. Though this natural way of carrying was expected to lower accuracy values, it is more indicative of expected accuracy if this research is to be applied to studies with patient groups.
The most accurate methods of data collection for activity recognition rely on multiple sensors. Often this involves accelerometers, as they are small, relatively inexpensive, and register both movement and orientation to gravity. Some systems have integrated temperature, compass, light, and sound sensors on the waist (Choudhury et al., 2008
) or a similar collection of multimodal sensors on the wrist (Maurer et al., 2006
; Gyorbiro et al., 2009
). For accelerometer-only arrays, multiple sensors may be placed throughout the body – anywhere from three to five locations (Bao and Intille, 2004
; Tapia et al., 2007
; Krishnan and Panchanathan, 2008
) or more. Mannini and Sabatini (2010
) provide a review of these approaches.
There are simpler alternatives to using multiple sensors, improving the convenience, cost, and compliance rates. The most common approach is to use a single, waist-mounted accelerometer. This approach has been analyzed on very specific sets of instructed activities with over 98% accuracy (Mathie et al., 2004a
; Mathie et al., 2004b
; Ravi et al., 2005
; Lee et al., 2009
). High accuracy ratings were possible in part due to the fixed location of the accelerometers on the body, the use of within-subject vs. across-subject cross validation, and the artificial nature of instructed movements. Signals from walking, sitting, and standing are necessarily more repeatable when in a consistent lab setting following instruction. For comparison, when subjects simply wore such a device for 24
h, with more natural activities, accuracy was closer to 80% (Long et al., 2009
). Single, waist-worn accelerometers have been well-studied in the domain of activity recognition, but may need consistent placement for high accuracy.
Unlike dedicated accelerometers, some people already consistently carry mobile phones, making them a convenient platform for recording movements. Most smartphones have built-in accelerometers and are often worn on the person, similar in principle to previous work on accelerometry. Mobile phones have built-in communication protocols that allow simple data logging to the device and wireless transmission. This permits real-time response, or in an experimental setting, compliance verification. Because mobile phones are widely adopted, compliance without verification is already high, as people are used to carrying them. Due to these advantages, mobile phones have the promise to provide a convenient, inexpensive, and objective means to detect the activities of people.
Mobile phones have been used to classify activities of healthy subjects (Bieber et al., 2009
; Brezmes et al., 2009
; Gyorbiro et al., 2009
; Ryder et al., 2009
; Wang et al., 2009
; Yang, 2009
; Kwapisz et al., 2011
). Common activities include walking, jogging/running, standing, sitting, and using stairs. The choice of activities influences accuracy rates, and also because most rates in these previous studies are not subject-wise cross-validated, applicability across subjects is more difficult to interpret. In Kwapisz et al. (2011
), healthy subjects were instructed to carry the phone in their left pocket and perform a specific set of activities; all activities except stair climbing were classified with at least 90% accuracy. Other studies found similarly high accuracy but with different classification techniques (Brezmes et al., 2009
; Ryder et al., 2009
; Yang, 2009
). In Wang et al. (2009
), classes were divided as still, walking, running, or in a vehicle, which simplified classification which was done using microphones and GPS as well as accelerometer readings. In Yang (2009
), a preprocessing technique was used which converted the axes from phone-specific to phone-independent coordinates based on orientation of gravity, providing 88–90% accuracy. While our results on healthy subjects are in line with previous studies, the central contribution of our paper is the careful analysis of precision of activity recognition in the context of PD.
We chose to analyze the PD population for various reasons. Millions of people throughout the world are suffering from diseases that affect mobility. Many diseases, such as stroke, heart disease, or depression affect large populations but have a wide variety of causes, types, and symptoms. PD, on the other hand, is characterized by a number of common characteristics, which makes analysis easier across subjects (Gelb et al., 1999
). Common symptoms such as tremor are visible in movements and lend themselves well to analyses using accelerometers in mobile phones (Joundi et al., 2011
; Surangsrirat and Thanawattano, 2012
). The PD population is also an important subgroup to consider as it also effects a relatively large population – approximately four million people globally (Dorsey et al., 2007
There is another study that automatically classified and characterized postures and activities for a population of PD patients (Salarian et al., 2007
). However, their results used within-subject cross validation and thus cannot speak to the across-subject generalization issue we are discussing here. Moreover, they used a set of accelerometers and gyroscopes instead of mobile phones. Our paper demonstrates the ability to use mobile phone recordings of acceleration to enable quality activity recognition with PD patients.
There are a few limitations to the interpretation of our results to address. For our healthy subjects, we used a population of both younger and older subjects, instead of age-matched controls. Some of the difference between the groups can be age-related, however we believe this effect was minor compared to the effect of PD on patient movements. Also, the PD group was relatively small (eight subjects) and heterogeneous (Hoehn and Yahr stage 1–3), however even from this heterogeneous group we note a significant improvement by using PD training data. Lastly, because we had both the researcher and subject observing, we relied on the recording procedure for accurate activity labeling. Subjects did not always perform the instructed actions in a typical fashion (e.g., moving feet while standing, stopping briefly while walking, etc). Instead of removing possible inconsistencies by hand, and thus affecting the validity of this approach, we retained all samples in the data set. Despite these limitations, the main conclusions of this study are supported.
There were two main goals for this study. First, we demonstrate how machine learning can be used to infer the activities of PD populations; the focus is not on particular, hand-picked features of movement, but on automated methods of weighing and combining those features. The second major goal was to highlight and quantify the effect of applying classifiers designed for healthy subjects on a PD patient population. A demonstrable drop in classification accuracy from 92.2 to 60.3% makes this point clear; it is important to use tools and analyses designed for specific patient populations. Although this study is not thorough enough to validate this classification method for clinical practice, it does demonstrate a strong benefit of machine learning, and a caution for clinicians who may want to use any activity recognition methods designed for healthy subjects.
The ultimate objective of therapies is to improve patient quality of life and activity tracking is an additional way of quantifying this. Such quantitative evaluation techniques could help clinicians test and optimize aspects of many therapies for motor disorders. By only downloading an application, mobile phones can record a person’s movements, greatly simplifying the study design and improving compliance. This information can be of personal or community medical use, improving evaluation of patient outcomes in therapeutic interventions. It is clear that populations with motor impairments require special consideration in approaches that analyze movement patterns. Mobile phones provide a means of tracking movements in an objective, convenient, and inexpensive way. The extent to which leveraging these qualities can improve and enable new therapeutic approaches is an area of further research.