Even though national regulations often require medical providers to notify public health authorities of relevant infectious diseases, this process is incomplete and delayed even in developed countries where considerable amounts of resources are assigned to disease surveillance every year [4-6]. The situation is often more difficult in developing settings due to lack of trained personnel and lack of resources to ensure proper functioning of the system.

Many technological solutions have been developed to improve the performance of disease surveillance systems [7,8]. The implementation of electronic-based platforms, for instance, improves timeliness and facilitates access to epidemiological data allowing more rapid analysis and response [9-11]. In addition, enhancement of laboratory capacities and automated results reporting contributes to better data quality [12]. Comparisons between automated electronic laboratory reporting and traditional paper-based reporting have shown that automatizing the reporting process improves completeness of data and timeliness of reporting in disease surveillance [13,14]. However, new strategies must be further investigated, especially those aimed at reporting personnel, as they remain the most important component of disease surveillance systems, especially in resource-limited settings.

There are several attributes of surveillance systems that the CDC recommends be regularly assessed. Among these, data quality and timeliness are essential characteristics influenced by several factors that depend on reporting personnel [15,16]. For example, time constraints and confidentiality concerns worsen data quality and timeliness [17,18], whereas strengthening ties with reporting health care personnel improves notification rates by encouraging interactions between medical care and preventive health sectors [7,19].

Alerta is an electronic surveillance system for infectious diseases that has been successfully implemented in the Peruvian Navy and currently covers 97.5% of the Navy population [11]. Alerta was created in order to improve the detection, prevention and control of disease outbreaks in the Peruvian military settings. The Alerta system was conceived as a collaborative effort between the US Naval Medical Research Center Detachment (NMRCD) and the Peruvian Navy, with technological support of Voxiva. Since its creation in 2002, disease trends have been established for each of the 86 reporting units, allowing the detection of many outbreaks. Acute respiratory illnesses, diarrheal diseases and pneumonias are reported twice a week, whereas 38 other notifiable infectious diseases including malaria, dengue, leptospirosis, viral hepatitis, tuberculosis and others are reported immediately after their detection via phone, internet or radio relay. The data are then available in an internet-based platform with secure access in real-time.

Reporting personnel responsible for surveillance in each unit participated in a 3.5-hour retraining course one week before the initiation of the study. The course program consisted of lectures about fundamentals of surveillance (1 hour) and case definitions of the most common notifiable diseases to Alerta (1 hour). In addition, hands-on sessions were conducted to practice how to notify via phone and internet (1.5 hours). Participants were provided with the Alerta system user's manual containing case definitions and description of the notification process. Reporting personnel were required to complete a self-administered questionnaire to assess factors related to deficient data quality and timeliness.

Units were then randomly assigned to one of three groups: phone, visit or control group. Reporting personnel in the phone group were contacted by telephone 3 hours before the biweekly deadline if they hadn't sent their reports at that point. A Peruvian Navy nurse from the Alerta Central Hub (ACH) was responsible for the monitoring process in all the units. He contacted the reporting personnel through their land-lines or mobile phones when appropriate. Phone conversations were based on an established template that included a reminder of the time deadline for the report and an opportunity for answering any questions: "Good Morning (title and last name). This is (navy nurse title and last name) from the Alerta Central Hub. You haven't sent the report of epidemiologic week (number) yet. Please, send your report today before 12:00 p.m. Do you have any questions/concerns regarding the notification? (Answer any questions). Thank you. Have a nice day". Reporting personnel in the visit group received a 30-minute supervision visit on week 4 and week 8 of intervention. The visit consisted of: (1) discussing the unit's notification performance during the last month; (2) retraining if needed in specific topics of the notification process; and (3) encouraging reporting personnel to report properly and on time. A team consisting of a physician and a navy nurse from the ACH performed the intervention visits. Units in the control group were not contacted by phone or visited during the study period. To avoid introduction of bias, reporting personnel were not told about the objective of the study or their group assignment. Phone calls and visits were considered part of the regular supervision activities of Alerta central hub personnel.

Timeliness was assessed through the report on time rate (ROTR), defined as the number of reports sent on time divided by the total number of reports, multiplied by 100. Data quality was assessed through errors per total number of reports (EPTR), defined as the number of errors detected in reports divided by the total number of reports, multiplied by 100. Trained personnel at the central hub reviewed the reports daily for data quality and error detection. Reporting personnel were contacted when reports seemed to contain erroneous data to confirm the accuracy of the information. Common errors included incorrect epidemiological weeks, duplication in reports, errors in case definition and erroneous number of cases.

Timeliness and data quality were assessed for acute respiratory illnesses, diarrheal diseases and pneumonias. Given that notification of these diseases was mandatory twice a week, ROTR and EPTR were also calculated twice per week, with a total of 24 measures of EPTR and ROTR in the 12-week pre-intervention period and another 24 measures of EPTR and ROTR during the 12-week intervention period for each reporting unit. Timeliness and data quality for other notifiable diseases such as malaria, dengue, viral hepatitis, tuberculosis and other infections that are reported immediately after their detection were not assessed given their expected very low incidence during the study period.

All reporting personnel responsible for coordinating surveillance in the selected units participated in the short retraining course and completed the questionnaire about factors related to deficient data quality and timeliness (Table ​(Table2).2). Absence due to military missions or assignments, limited access to reporting tools and forgetfulness in reporting were the most frequent factors related to delayed reports. Lack of training and generation of errors by the system itself were the most important factors related to reporting errors, according to the personnel.

When we compared the pre-intervention and during-intervention timeliness of reports in each group, the phone group showed a significant increase in ROTR (p < 0.001). This improvement was also maintained after stratifying groups according to type of unit (clinic or ship) (Table ​(Table3).3). Neither the visit group nor the control group showed a significant improvement in timeliness (p = 0.798; p = 0.847 respectively). Generalized linear models were used to compare the effects in ROTR between the intervention groups. To demonstrate that different effects were not due to the differences in baseline ROTR levels, an interaction between time (pre-intervention & during intervention) and group (phone, visit and control) was introduced into the model. The analysis showed that the interaction between time and phone group had a significant effect in ROTR among clinics (β = 0.2153; p = 0.005) and ships (β = 0.3067; p = 0.002).

To explore in detail the effect on timeliness after each visit in the visit group, a temporal analysis was performed looking for any gradual changes in ROTR between periods 0-to-4 week, 5-to-8 week and 9-to-12 week. No significant differences were found between these periods (p = 0.673; p = 0.632).

Regarding the data quality assessment, the visit group showed a decrease in the EPTR (p = 0.017). However, after stratifying groups by type of site, the decrease in EPTR persisted in clinics (p = 0.007), but was not evident in ships (p = 0.455) (Table ​(Table4).4). Neither the phone group nor control group showed a significant decrease in error rates (p = 0.241, p = 0.309 respectively). Again, using generalized linear models, we introduced an interaction between time and group to evaluate the effect in EPTR. The analysis showed that the interaction between time and visit group had a significant effect in EPTR among clinics (β = -0.0418; p = 0.049), but not among ships (β = 0.0041; p = 0.820).

There are factors intrinsically related to military settings that might have contributed negatively to timely surveillance in this sample. For example, frequent deployments away from home and restriction of communications during missions could generate unintentional epidemiological silence. In addition, limited access to reporting tools such as computers with internet access also affects timeliness in electronic-based surveillance systems, especially in developing countries where availability of technology is not widespread [20].

Despite the fact that no outbreaks were detected during the study, it logically follows that enhancing timeliness in the phone group would have led to more rapid outbreak detection. Surveillance systems that diminish notification delays improve their capacity for detecting outbreaks and allow the triggering of more appropriate responses [9,16,21]. Military and other confined populations are of special concern because they are often exposed to conditions favoring the transmission and perpetuation of pathogens [22]. Moreover, the need for early disease detection systems in these settings is increasing as the potential for pandemics persists and studies have shown that military populations played an important role in the origin and explosive expansion of pandemic agents in the past – in the US, the influenza pandemic of 1918–19 began among military trainees [23,24].

The effect of supervision visits on data quality differed in clinics when compared to ships. Previous studies have shown that regular training decreases reporting errors [25,26]; however, in our study, there was no beneficial effect of briefly training personnel during supervision visits performed on the ships. It is possible that because ships had less experience (time) using the Alerta system, visits did not improve data quality on the ships as they did in the clinics. More intensive monitoring and training may be needed in these newly incorporated sites. The lack of improvement in data quality on the ships might also be partially explained by inherent adverse conditions that exist on Navy ships, such as frequent personnel turnover, frequent accidents and frequent deployments with limited access to assistance.

Although electronic-based notification often enhances data quality [9-11], Alerta reporting personnel perceived that the system's platform itself generated errors. However, the data did not corroborate this perception. Perceived barriers for the implementation of biotechnologies have been described in developing countries [20], including the difficulty in replacing traditional pen and paper reporting systems with technological solutions that are not commonly used. This negative perception of technology is very important to address in order to assure successful adoption of technology among users and thus favor the optimal implementation of an electronic-based system in developing countries [27].

This study had some limitations. We performed 170 calls in the phone group during the study period, but 16.5% of the calls were not successful (reporting personnel did not answer or were absent from the unit despite at least three attempts to contact them before time deadline). Although there was a small sample size, we did a total of 24 measures of EPTR and ROTR before and during the interventions for each site. The use of generalized linear models (GLM) for binomial data allowed us to calculate the impact of the interventions considering these pre and post measures in each reporting unit maintaining a clustered analysis. Finally, specific characteristics in this sample might affect the reproducibility of our results in other settings. For example, health care providers in the Alerta system are also charged with disease notification, which differs from the majority of other surveillance systems in the region where administrative personnel provide notification. In addition, constant deployments and frequent missions might negatively affect timeliness and data quality of reports in this sample, especially among ships when they are underway.

Financial support: This work was supported by DoD-GEIS Unit 847705 82000 25 GB B0016. Disclaimer: The views expressed in this article are those of the author and do not necessarily reflect the official policy or position of the Department of the Navy, Department of Defense, nor the U.S. Government.