In a household survey, using previously untrained CHW's as data collectors, we utilized mobile phones to enter and upload data at the point of collection. The software application and web-based interface enhanced real-time supervision of data collectors. We were able to implement this survey with low direct costs of materials. While we are not able to compare costs to a paper based approach in this study, it is likely that a cost of $0.30 per survey compares favorably. Were we using a paper based approach this figure would include paper costs, printing costs as well as data entry costs. The issue of the cost effectiveness of the system needs to be explored in further research that employs a comparison group. Overall, our findings demonstrate that mobile phone based data collection is feasible at scale.
Real-time supervision of CHW performance was a significant advance over previous implementation work. It has been argued that pen and paper are prone to fabrication [21
]. Our web-based interface permitted a previously impractical degree of detailed, hour-by-hour supervision, which markedly improved our ability to detect one type of data fabrication. Other types of data fabrication could still go undetected; for example, a data collector could key in random answers on a timed basis, and our supervisors would be unable to detect this activity. Such fabrication could only be detected and eliminated with the use of a more expensive mobile phone with GPS capabilities that would enable supervisors to track CHW movements. The web-based interface had other advantages as well. The automated graphs and real-time information allowed supervisors to focus their time on other aspects of quality control and solving logistical difficulties in the field.
With regards to the direct costs of materials, the automated uploading of completed surveys obviated the step of having to transfer data in the field from a PDA to a laptop. This is a significant cost saving, as laptops are often the most expensive up front cost in studies using PDAs for data collection [2
]. In addition, the web-based interface permitted us to monitoring the costs of uploading survey data in real time.
Automated data upload from the mobile phone to the server significantly reduces data loss due to PDA damage, theft or loss because of the time elapsed between data collection and data upload. Prior studies have reported technical problems with data upload and download using PDA-based survey systems [24
], as well as difficulties in remote sites with data upload due to electrical interference with telephone lines and switchboard difficulties [1
]. In addition, there have been considerable problems noted with the instability of the "active synchronization" process when transferring data from PDA to computer [25
]. In our study, there was no data loss. While this does not guarantee that data loss cannot occur with this system, we believe the chances of data loss were significantly minimized.
Our mobile phone based survey apparatus may be particularly suited for conducting survey research in rural areas. In surveys where multiple research sites may be remote and dispersed, and where vehicles have to be used to travel from site to site to download data onto laptops, the mobile phone based data collection system may be a significantly cheaper option. Importantly, survey storage (up to 50 completed surveys can be stored on an entry-level mobile phone) and delayed upload permits surveys to be conducted in areas where there is no mobile phone coverage.
Mobile phone based surveys have other advantages and disadvantages that may be particularly important to researchers depending on their needs and the study setting. The automatic uploading of surveys and encrypted access to the web-based interface contributes to improved data security and respondent confidentiality. In diverse populations where multiple languages are spoken, multilingual paper assessments are cumbersome and costly [26
]. Mobile phones (like PDAs) easily integrate multiple language assessments with a simple drop down menu of language options. With regards to potential disadvantages, one of the drawbacks of using a paperless system is that there is no paper questionnaire to review in the event that problems are detected in the field. Also, although the automated data upload reduces the potential for data loss, mobile phones are valued items in resource-limited settings [27
]. Thus mobile phones can be stolen, or may cause research staff to be targeted when conducting household surveys in high-crime areas. We chose to use entry level mobile phones for this very reason, but we did also permit the use of a paper and pencil version of the survey in isolated instances (for instance, if the CHW's felt threatened in a particular household or area). It is tempting with the introduction of a new technology to see it as a panacea for wide ranging methodological challenges that are common to most research. Using mobile phones for data collection will for instance not be able to address issues such as household access, selecting an appropriate sample that permit generalization of findings, or the complexities of clustered sampling to reflect population statistics. On the other hand, the immediate real time access to data significantly improves data quality, while the complex skip patterns of the mobile phone programme provides comprehensive validity and readability checks within the instrument.