In fact, as evidenced by a considerable body of literature, the use of portable handheld computer technology in the field of health care and clinical research is on the rise [6]. However, most of the literature is descriptive, focusing on the technology, the methods and/or the experience. Descriptive studies that often lack a control group are limited with regard to comparisons of the effectiveness of different methods. A review of randomized controlled trials comparing the effectiveness of hand held computers with paper methods from 2006 summarizes that handheld computers appear superior in timeliness of receipt and data handling, but that the studies reviewed used different study designs and error definitions for assessment of data accuracy and reported inconsistent results [7].

This paper compares the performance of four electronic data capture methods (see supplementary material, File S1, for machine specifications) with the performance of conventional paper-based data collection in a Gambian medical research field station setting:

There are a plethora of devices suitable for EDC; this study uses three technology types, netbook, tablet PC and PDA , which are commonly used for EDC. Scanning devices such as smartpen or optical character recognition technologies [8], [9], were not considered in this study. The objectives of the study were to combine a formal comparison with a demonstration that EDC is viable for clinical trials routinely undertaken at The MRC Gambia. Tools to improve EDC data accuracy, automated range and consistency checks or skip patterns, were not evaluated. Data entry during telephone interviews, in which CRFs were administered via mobile phone, was included due to the high and increasing number of mobile subscribers (In 2009, 800,000 out of a population of 1.7 million) in The Gambia.

Within the MRC Gambia research structure, field workers and nurses are responsible for administering CRFs in the community and would be the staff to pioneer the use of EDC tools in the future. Consequently the five interviewers were randomly selected from a pool of 12 available field workers and nurses. The fifteen interviewees were voluntarily and randomly recruited from 400 MRC staff based at the main Fajara site.

The CRF for this study was a facsimile of typical CRFs from the range of clinical and research studies undertaken at the MRC. It was developed in the style of the World Health Organization's individual questionnaire, which was used for the World Health Survey in 2002 [11].To make the interviews as authentic as possible, genuine questions flowed through identifier, demographic and adult/child clinical data sections, but with an emphasis on the question fields, free text and date fields, which are typically associated with the highest error rates. The number of children per interviewee in the scripted responses was restricted to four to ensure the comparability between interviews regarding the number of fields and duration of the interview.

For clinical trials conducted at the MRC, OpenClinica® [12], an open source software package exclusively designed for EDC and compliant with Good Clinical Practice (GCP) requirements, is the software of choice. OpenClinica® is a web-based application ideally suited to wireless client-server implementation and is available for free or as paid edition, which includes product support.

MRC field workers have little or no professional experience with handheld devices and a wide range of informal computer experience, largely dependent on their age. They are not routinely trained in electronic data entry techniques or the use of data management tools such as OpenClinica®. Consequently the five randomly selected field workers (plus one reserve) were given a three day training course typically offered to data entry personnel at the MRC in the Gambia. The major areas covered included an introduction to using the OpenClinica® software and the web based application for EDC, familiarization with the electronic devices and interview practice.

In an effort to create realistic field conditions to test the performance of the screens and ruggedness of the EDC machines, the interviews were conducted outside in a tree shaded area on the MRC Fajara field station. Telephone interviews were conducted indoors with the interviewer sitting next to a conference call facility. The interviewee was called on her/his mobile phone and data were simultaneously entered into a PostgreSQL database using OpenClinica® as front end on a standard laptop. The control method was the conventional paper-based CRF with adjudicated double entry into OpenClinica®, by two out of a possible 25 randomly selected MRC data entry staff.

The interviewers were asked to record the start and end time of the interview process, excluding the time needed for starting the program and entering their credentials.

OpenClinica® was not used in PDA as no compatible version for PDA was released at the time of the study. Therefore, ASP.NET technology for mobile devices was used with SQL Server at the backend to develop an application program to capture data using PDA device. Nevertheless, the underlying application architecture was client-server based for both OpenClinica® (Version 3.0.2) and ASP.NET with wireless connectivity to communicate with the server. Wireless technology to rural areas, where many field studies take place, is in early stages of development within the Gambia [13], [14], [15], [16], [17], therefore this approach was not used for the study. The netbook and the tablet PC each had sufficient memory to store the requisite data on the device. With the PDA, conventional data capture, and telephone interview modalities, data were remotely entered into a database on SQL server 2000. To prevent data loss the databases were backed up daily. The five resulting databases were compared to the gold standard database of the randomly generated answers using a generic web-based double entry application for SQL server databases [18]. Any missing values or inconsistencies with exception of the use of capital and lower case letters counted as an error.

Assuming salary costs of US $ X/hour for a data entry clerk, US $ 1.33*X/hour for a field worker, US $ 1.67*X/hour for a data supervisor and US $ 2.33*X/hour for a data manager, salary costs per correctly entered field were calculated with the following formulas:

For the standard method (involvement of a field worker, a data entry clerk and a supervisor):For EDC methods (involvement of a field worker and a data manager):

All study volunteers provided informed, written consent to participate in this study.

Each questionnaire comprised of 115 questions. Due to problems with the partial date function in OpenClinica® (a known bug in OpenClinica® version 3.0.2), which led to inconsistent swaps of the day and month fields within a date during the data extraction process, only 111 fields per CRF were analyzed; 25 (22.5%) text, 10 (9.0%) date, 6 (5.4%) numerical, 37 (33.3%) single select and 33 (29.7%) skip questions. 3 (9.1%) of the skip questions were text fields , 9 (27.3%) numerical, and 21 (63.6%) single select fields.

73 (97.3%) of the 75 interviews were conducted according to the time schedule. Two interviews using the tablet PC could not be conducted due to problems with the software during the first study week. One record in the PDA database was partially deleted immediately after the information had been collected. The time record for this interview could not be retrieved. In total 8,103 (73*111) data fields were analysed to calculate error rates. The information, which was collected with the PDA, but was erroneously deleted, was counted as missing values.

(For the access database holding the generated reference and the raw data from this study see supplementary material, File S2).

So far the literature is mainly descriptive when comparing EDC to the conventional paper method. Analytic studies usually compare the transcription (CRF-to-database) error rates for two different devices such as for example a laptop and a handheld device.

This study confirms that after a short period of EDC introduction, electronic data capture can be more time effective than the standard, paper-based data capturing process followed by double entry and verification, although the duration for the interviews using the standard paper questionnaire were shorter compared to the interviews, in which the data was collected electronically.

The second advantage, which has been described in the literature, that electronic data capture is more accurate than the paper-based method could not be confirmed in this pilot study. Even though the interviewers familiarised themselves with the new data capturing methods very quickly and the overall error rate decreased considerably over time for all EDC. For two EDC methods (tablet PC and netbook) error rates approached those of the standard method, but were never lower. In addition, since the main objective of this pilot study was to test if EDC was feasible and to explore which EDC method(s) proved to be the most suitable for the next step of field studies, range and consistency checks to optimize data quality were not implemented.

It should be noted that text fields were overrepresented in the applied questionnaire due to sample size considerations. The 2010 Demographic and Health Survey (DHS) model household questionnaire [19], or the 2010 Multiple Indicator Cluster Survey (MICS) household questionnaire [20] comprised roughly 5 or 11% ‘text field’ questions, respectively, depending on household size. The most prevalent type of question in the DHS or MICS surveys was the single select type. Errors in text fields were based on exact comparisons, other than case, which might have resulted in an overestimation of the error rate for character based data, since some text field errors might have been of minor importance. For example, the misspelling of names is usually inconsequential due to unique identifiers. On the other hand this pilot study was not sufficiently powered to detect small differences in data accuracy within scales which might be important for large field studies. Direct comparison of the performance of the PDA with the remaining EDC devices is limited since a different, web based application was used for the PDA. High error rates for this device in the first week could be explained by the high proportion of missing values due to the partial loss of data. In OpenClinica® the required fields feature was implemented but not in the web based application, which might account for higher error rates for the PDA in general. The relatively poor performance of the telephone interviews is most likely due to miscommunication. Administering the questions via telephone might also have increased the interview duration since questions/answers had to be repeated more often than in face to face interviews. Interviewers improved rapidly with respect to i) the time needed to conduct the interview ii) to record the information electronically and iii) data accuracy. Since the study was designed as a pre -study for the duration of three weeks with a brief training phase, we can not judge if the interviewers had reached their highest level of performance, or if further significant improvement could be achieved. A field study would require a more intensive/longer training where a pilot phase is standard procedure. Another important consideration is whether field workers would be able to maintain their performance in the field. Comparing data accuracy for the EDC and conventional data capture process should certainly be part of follow up studies in the field, which are appropriately designed to address this question with respect to samples size and study duration.

A commonly-raised question about the feasibility of EDC in a setting like The Gambia is acceptance among field workers and whether they are capable of handling the electronic devices and software for electronic data capture. With the random selection of 6 (5 interviewers & 1 reserve) of the 12 workers nominated by the four MRC programs (Bacterial Diseases, Viral Diseases, Malaria and Nutrition), we sought a group of interviewers representative in their age, gender, work experience and performance, computer legacy, and attitudes towards EDC. Although fieldworkers were ultimately selected randomly, each programme purposely nominated three candidates and selection bias can not be ruled out completely.

At the beginning of the training session for this study, 5 out of 6 interviewers stated that EDC would be appreciated by the MRC field workers and that, since field workers have established a very good relationship with their study participants in the past, the participants would likely embrace EDC. The field workers/nurses who participated in this study were keen to learn new techniques and develop themselves further.

The majority of the interviewers would prefer the ‘bulkier’ devices, netbook and laptop, which were used for electronic data capture during the telephone interviews, to the smaller devices tablet PC and PDA for field work. That older people find it more satisfactory to use the ‘bulkier’ devices with a keyboard rather than the smaller pen-based devices has already been described in the literature [21]. The display of the QWERTY keyboard on the screen of a palm-size computer results in each key being very small. This slows performance [22] and may also give rise to legibility problems with difficulty in discriminating between some letters such as ‘u’ and ‘v’. The limited screen space also results in a lack of gaps between adjacent keys. This necessitates fine motor control by those making entries. The space available on the screen is very small so that the keyboard often requires the use of special function keys to access certain numeric and punctuation characters. It has long been known that moded styles of interaction can be confusing for users [23], [24]. All interviewers in our study were already familiar with the standard keyboard and had at least two years MRC work experience; each also accessed their MRC e-mail account at least once a week. The familiarity with the keyboard enabled the interviewers to enter data within a shorter time period after training compared to devices which required use of a pen. Another reason given for preferring the bulkier devices was that the larger screen allowed working with larger font sizes. Some of the field workers, especially the older ones, had problems with their eyesight. During the study interviewers would only use their glasses when the font size became so small that they could not cope otherwise. In the Gambian setting reluctance to the use of glasses is common. While two fieldworkers had problems reading text on the PDA and the tablet PC, prompting one of them to use his glasses, no such problems occurred with the netbook or the laptop. Nevertheless, in this study the use of the pen/touch screen improved rapidly for all fieldworkers , so that in the final week the duration of the interviews using OpenClinica® as the front end, which did not depend on the network connection, was in a similar range, regardless if a keyboard or a pen/touch screen was used.

Studies of performance with normal size keyboards have shown that although older people are slower in key tapping and in selecting the key to tap [25], especially if they are unskilled typists [26], there is no loss of accuracy. Given sufficient experience with the task there is evidence that older people may be able to use compensatory strategies to maintain their performance [27]. This study lends support to this statement, since accuracy was not associated with age. Speed was highly associated with age, however. Younger interviewers conducted the interviews in less time than their older colleagues. As one would expect, interviewers with more experience in fieldwork were superior with respect to accuracy than those with less fieldwork experience.

Despite the availability of diverse data management software applications, it was beyond the scope of the study to test each implementation for compatibility with specific EDC tools. The data management unit at the MRC routinely supports 10 simultaneous clinical (phase I or II) and research trials; to be of practical relevance, EDC tools must be capable of supporting the commonly used software platforms. The open source, clinical trials software for EDC and clinical data management OpenClinica® was used for three of the four EDC methods, since this clinical informatics system has been designed to support GCP standards. Unfortunately an OpenClinica® version for PDAs was not available and a web based system was established. The PDA is at a disadvantage in this regard as coding a bespoke solution or integrating an open source/commercial application is time consuming and potentially expensive if commercial software is used.

At the MRC in the Gambia various types of field based epidemiological studies are conducted apart from clinical trials. For those studies alternative data capturing methods such as improved web based systems or telephone interviews in combination with EDC, or standard data collection on paper followed by optical character recognition (OCR) might be more appropriate or more cost effective.

Obvious rules, which apply for telephone interviews in general and can be found in many standard epidemiological text books, will be valid for telephone interviews in which data is collected electronically as well. For short follow up studies, in which a physical examination of the interviewee is not needed, telephone interviews might be a quick and cheap alternative to sending a field worker to the field, especially since the Gambia enjoys excellent mobile phone coverage.

A South African group recently described a web based system using standard mobile phones for electronic data capture [28]. The questionnaire was developed in a word processor and sent to a standard mobile phone, which was used to collect the data during a survey. The data was stored as a text file and whenever the network connection was available, the encrypted information was sent via SMS to the host computer. The automated data upload from the mobile phone to the server reduces data loss due to damage , theft or loss of the device. Mobile phones have the advantage, that they are available, cheap, relatively robust and less attractive targets for theft compared to the more expensive devices tablet PC and netbooks. The downside of this method is, that techincal problems with data upload and download [29], electrical interference with telephone lines [30] and considerable problems with the synchronization process [31] have been encountered in the past.

In May 2010, a new web services framework became available in OpenClinica®, which supports mobile devices as a means of data collection for clinical trials [32].

For surveys, with a high percentage of tick questions can be developed, the OCR might be the appropriate choice.

EDC gives scope for applications in combination with additional functions such as GPS for identification and retracing of households, probability sampling for household surveys, which results in statistically valid samples, rapid aggregation, analyses, and availabitlity of preliminary results within days of completing the field work [33], or barcode reading to identify study subjects or samples.

EDC promises to expedite the availability of accurate data for the research program. However, there are considerable risks in achieving this goal. Pens might be lost, devices might be defective, broken or stolen. Especially in a setting like the Gambia devices are exposed to dust and humidity, and are transported on motorbikes during the day to day routine. Malfunction or loss of the device might imply that the data won't be collected according to schedule, or that data which have been already collected might be lost or have to be collected again. This will rapidly increase the time spent on gathering such information and error rates are likely to increase. For this situations a rescue plan should be established, which might involve provision of a replacement device, which will be readily available within a short period of time. Longer study duration and replacement of devices are associated with increased costs.

File S1

File S2