Numerous studies have examined how Internet searches can "predict the present", meaning that search volume correlates with contemporaneous events [18
]. Specifically in the case of influenza, search volume was shown to estimate flu activity, which was not officially reported until two weeks later, and despite unknown flu status of the searchers. Building on this concept, a medium such as AMT allows for obtaining more detailed information beyond disease prevalence in real-time while harnessing the vast pervasiveness and convenience of the Internet. For instance, this study shows that AMT can be a way to garner public health information and at resolutions in time, space and demography that are unavailable in other forms of surveillance.
This work demonstrates the first use of harnessing micro-monetary incentives and online-reporting for public health surveillance. Traditionally AMT is used to recruit individuals to perform tasks difficult for artificial intelligence, such as in image and natural language processing. Previous studies investigating the efficacy of monetary incentives for performing tasks showed AMT provides a flexible and robust venue optimized for payment type [7
]. The motivation for Turkers in India is more often monetary than for non-Indian Turkers (27% of Indians report requiring AMT income to make ends meet). Although financial motivation could lead to arbitrary responses, here we demonstrate that the data collected reflects other surveillance methods' findings for the outbreak period. Several studies have used AMT without a gold standard verification and have shown how to shape surveys, e.g. by including validation tests, to help ensure the quality of AMT responses [21
]. Verifiable questions signal to users that their answers will be scrutinized, potentially both reducing invalid responses and increasing time-on-task. Experience from this study also suggests that public health surveillance via online self-reporting should also incorporate a structured set of verifiable questions to enable substantiation, particularly when other traditional surveillance methods may be deficient.
This study capitalizes on AMT's demographics; India is the second largest user base [22
]. Additionally, Internet use and access, although increasing, has higher reach in urban areas. Further, AMT's user-base in India has an average age of 26-28 years, and Indian Turkers are substantially more likely to be male than US Turkers (two-thirds of Indian Turkers are male) [22
The very small amounts of payment administered through AMT also have been shown to be sufficient to garner public health information from this population, thus demonstrating AMT as a way to perform a field study at a very reduced cost. The optimal amount of monetary incentives used to solicit public health information should be studied further, however, this kind of payment offers a drastically reduced cost for administering a field study over traditional methods [23
]. In addition, this medium can easily and quickly reach remote subjects who may be underserved by traditional health infrastructure, where the majority of malaria deaths occur in India [5
]. AMT is a particularly useful platform because it maintains anonymity of users, thereby assuring study subjects that sensitive personal health data will be kept private and secure.
Self-reporting is worth exploring due to likely differences in content and timing of self-reported versus physician-reported information [24
]. Through self-reporting users gain more involvement with their own health, which can be important in fostering preventative health behaviours. Furthermore, self-report is facilitated by the rapid spread of consumer technology like mobile phones and eliminates delays by bypassing the chain-of-command relay structure of traditional public health surveillance [6
Methods like AMT offer an epidemiologic tool with greatly reduced cost compared to traditional field surveys. Shown here, AMT can give unprecedented access to finely-resolved real-time public health information (daily, weekly) that would otherwise be unavailable and have vital implications for prevention and control measures. Taking advantage of a tool such as AMT for public health reporting on a particular environment and with a specific disease focus (here, malaria in India), can be useful as a complementary tool to existing and traditional public health infrastructure by providing focused outbreak investigation from particular groups. For malaria surveillance in particular, AMT could be used to investigate drug therapy adherence, which is a large issue in malaria relapse.
There is no available gold-standard with comparable temporal or spatial resolution which to confirm accuracy of the proportion of malaria infections, as garnered through AMT. HealthMap reports were one available source with similar resolution in time (daily). The outbreak peak, measured through volume of positive responses in AMT for 2010, was delayed compared to the volume of HealthMap reports. This could be due to the fact that there is more news reporting earlier in an epidemic period. In addition, by the time an outbreak peaks, awareness of the outbreak may then subsequently increase self-reporting response rate from the public.
In examination of the proportion of positive malaria diagnoses through AMT in 2010, the percentage of reported positive malaria diagnoses was markedly higher than the most relevant data (during the outbreak, from June 1-20, 8.4% of people from Mumbai examined tested positive, vs. 36.9% of AMT responses) [25
]. The percentage of positive diagnoses from AMT in 2011 was significantly lower than in 2010. This corresponds to the trends conveyed by governmental organizations [14
]; the number of cases dropped by 80.4% for the year until early August and the slide positivity rate, the measure of malaria incidence, dropped by 18.9%. Officially reported numbers of course only represent burden of the population using health care facilities. The lower proportion of the reports relaying the Plasmodium
type in 2011 via AMT could be due to a slight change in wording of the survey from 2010-2011.
Media reports may underestimate disease prevalence, as some cases are not reported to a physician. Furthermore, some cases seen by physicians are not reported to regional offices [28
]. Conversely, reports from AMT may also be biased due to a likely greater proportion of reports to physicians by the Turker population's demographics (age, education level, geographic concentration in urban areas and technology usage). The AMT responses may also be skewed by Turkers who have recently heard about malaria in the media and are more interested in a malaria-related HIT, or who might falsely believe that the researchers desire and better reward positive diagnosis reports.
In comparing spatial and demographics, no finely resolved official age prevalence information exists to compare our finding of the age-specific prevalence.