We evaluate costs and cost-effectiveness under three different implementation scenarios: (1) as implemented in the trial under study conditions in the study districts; (2) if the basic intervention was implemented by the Ministry of Health in the same study locations under routine, non-study conditions where specific study related costs are removed; and (3) if the intervention was expanded to a national scale in Kenya and implemented entirely by the Ministry of Health.
All costs are financial costs reported in 2010 U.S. Dollar values (USD). Costs are evaluated from the perspective of the program implementer. For Scenario 1, costs are actual financial costs for implementing the study by the research programme. Based on the experience gained in implementing the study and improvements in access to low-cost bulk text-messaging systems that are now available in Kenya, costs for Scenarios 2 and 3 are based on estimates of resources needed and associated costs for the Ministry of Health. We evaluate the sensitivity of results for each scenario to increases in implementation costs or reductions in intervention effectiveness.
Scenario 1 – The text-messaging intervention under the trial conditions in study areas
The cluster randomized controlled trial was undertaken between March 2009 and May 2010 at 107 rural government health facilities in two malaria endemic areas in Kenya (Greater Kisii/Gucha and Greater Kwale). The trial is registered with Current Controlled Trials, ISRCTN72328636 
. The detailed description of study areas and intervention characteristics was provided elsewhere 
. The intervention was a one-way communication of text-message reminders on paediatric malaria case-management accompanied by “motivating” quotes. The messages were sent to personal mobile phones of 119 health workers performing outpatient consultations at 54 intervention facilities in study areas.
The intervention development process and subsequent implementation included the following activities. First, the content, order, frequency and duration of the text-messages were developed over 5 days in partnership between researchers and policy makers of Kenyan Ministry of Health's Division of Malaria Control. The key messages addressed recommendations from the national malaria guidelines and training manuals valid at the time of the study, which recommended presumptive treatment of childhood fevers with artemether-lumefantrine (AL) and related AL dosing, dispensing and counseling tasks 
. During the same activity, understanding of text-messages was pretested during two rounds of individual interviews with 20 health workers from health facilities in neighboring study districts. In total, 10 different malaria text-messages were selected as part of this process. During the implementation of the study, messages were sent for 5 working days a week (two messages daily at 9am and 2pm). These messages were repeated every week during the 6-month intervention period.
Second, a computerized distribution system was developed on a desktop server interfaced with the network of a local mobile service provider through a global system for mobile communication modem. The system ensured automated delivery of text-messages according to a pre-determined list of phone numbers, timing of message transmission, and content of the text-message. The system was developed by an information technology specialist, who also provided maintenance support during the implementation period. The performance of the distribution system was tested in two rounds: first on 40 mobile phone numbers during 6 weeks of testing, and then on 120 recipients during one week. Third, the mobile phone numbers of all health workers at study facilities who were recipients of the intervention during the trial were collected by study teams during a health facility survey.
Finally, the intervention was implemented between May 4 and October 30, 2009, when 33,361 text-messages were sent to 119 health workers on 150 phone numbers (31 health workers had more than one number). During this period, the delivery of text messages was monitored by a research assistant using the computerized system developed by the information technology specialist.
Scenario 2 - The text-messaging intervention if implemented under routine conditions in study areas
Under this scenario, all components of the intervention development and delivery would have remained the same as under the trial conditions with one exception, the collection of health workers' mobile phone numbers. As part of the trial, collecting these numbers was integrated into the health facility survey used to establish baseline case-management practices in study districts. If the Ministry of Health implemented this intervention, district supervisors would be required to update and verify already existing lists of health workers' phone numbers by district. The district supervisors would then provide the list to the intervention implementers at the national level. Considering this scenario is relevant because the relatively complex and expensive health facility surveys completed during the trial accounted for almost 30% of total costs of the Scenario 1. Such costs would fall substantially if the intervention was implemented by the Ministry of Health under routine, non-study conditions.
Scenario 3 - Scaling up to the national level
If the Ministry of Health (MoH) integrated this intervention into national policy, and implemented the intervention at a national scale, two specific modifications to the basic intervention package would be required. First, Kenya has revised the national malaria outpatient treatment guidelines since 2010 and now recommends universal parasitological testing with microscopy or rapid diagnostic tests and adherence to diagnostic test results. Therefore, a process of revision of text-messages and its field pretesting would be required to incorporate changes in national guidelines. Second, scaling the intervention to 149 districts supporting approximately 20,000 health workers at 5,367 health facilities countrywide would require an automated distribution system able to support relatively high volume of messages sent within short time periods (twice daily, five times a week, for six months). We have considered here the standard way bulk SMS services are delivered in Kenya by network-authorized service providers already used to undertaking large-scale text-messaging campaigns, such as service advertisements or employee notifications. Finally, we have also incorporated costs for the development and maintenance of a web-based system for monitoring the delivery of the intervention for the MoH.
For each of three scenarios we calculated costs of the intervention as the direct financial costs (USD 2010) for implementing the scenario. For Scenario 1, the full costs of implementing the study come directly from the financial records for the study. For Scenario 2, program implementation costs are adjusted to exclude costs from Scenario 1 that were for research purposes but would not be included if the intervention was implemented by the government as routine practice. Personnel costs are also based on standard government employee salary and benefits information for the staff implementing the intervention. For Scenario 3, we extrapolated resource needs from the study sites to the national level based on the number of districts in the country, estimates of the number of health workers receiving the messages, and commercial rates for bulk messaging.
For effectiveness information for Scenarios 1 and 2, we use the results of the cluster randomized controlled trial to estimate the additional number of febrile children correctly managed according to national guidelines due to the intervention. The intention-to-treat analysis showed that correct AL management, defined as a child treated according to national malaria treatment guidelines, improved immediately after the intervention and this improvement was maintained 6 months later with nearly equal effect sizes (23.7 and 24.5 percentage points respectively).
Thus, the additional number of children correctly managed by the intervention was estimated by multiplying annual number of sick children at the intervention facilities by the proportion of febrile children at the same facilities. The annual number of children was extracted from the outpatient registers at intervention facilities while the proportion of febrile children at the same facilities was imputed based on data collected during the health facility surveys used to evaluate the intervention. In total, 153,379 children in the study sites required correct management for malaria during one year. As a base case, we assume that 25% more children with fever were correctly managed according to treatment guidelines in the intervention sites as compared to the control sites based on the intention-to-treat analysis during the six-month trial and during the six-months following the end of the trial 
. Based on these figures, 38,345 additional children were correctly managed in the study sites due to the intervention (38,345
For Scenario 3, the annual number of febrile children requiring correct management for malaria was obtained from the latest national estimates of febrile children presenting to public health facilities in Kenya 
. On the national scale, the annual number of febrile children presenting to public health facilities in 2007 was estimated to be about 11.8 million. Applying 25 percentage points effect size of the intervention at the national level, an annual number of febrile children correctly managed due to intervention would be about 3 million children.
Given the costs and effectiveness of the intervention estimated and modeled for each scenario, the average cost per additional child correctly managed is calculated as:
where c is the cost per child correctly managed for each scenario, TC is the total costs of implementing the intervention for each scenario, N is the total number of children during a year needing correct management, and E0
is the effect size as a proportion (E0
0.25 is the base case assumption).
Given the recent policy shift in Kenya from presumptive treatment for malaria to more complex management based on diagnostic tests and then treating according to test results, we consider the sensitivity of the results to higher program costs (a higher TC for each scenario) and a lower effect size. Conceptually, an increase in costs of the program is equivalent to a proportional reduction in the effect size. In the sensitivity analyses reported, we considered changes in effect size from 0.25 to 0.20, 0.15, 0.10, and 0.05, which are equivalent to increases in total costs of 25%, 67%, 150%, and 400% if the effect size remains 0.25.
Written informed consent was obtained from all health workers and caregivers of sick children during the trial and the study protocol was approved by the University of Oxford (OXTREC No 3808) and Kenya Medical Research Institute (SSC No 1329).