Setting, study population, and routine data collection
The study was conducted in the Bandim Health Project’s study area, which covers six districts with about 90
000 inhabitants or around 30% of the population of the capital. The project has implemented a routine data collection system in this area. A census has been carried out regularly, and all houses have painted numbers and have been mapped. All residents have an identification number in the census files, and information on place of living can be retrieved from these files along with socioeconomic and demographic information. All houses are visited every month to register new pregnancies and births. All children are given a UNICEF road to health card at birth, which is used to trace children when they attend health centres or the hospital. All children are visited at home every three months until the age of 3 years and information collected on risk factors associated with child survival, including breastfeeding status, infections, admissions to hospital, vaccination status, and living with the mother. There are three health centres in the area.
—Data on vaccination status are gathered in two complementary ways. Firstly, all vaccinations given at the three health centres in the study area are registered on the day of vaccination. This covers more than 80% of the vaccinations received in the study area but does not account for those obtained outside the study areas, such as when the mother is travelling or if she lives close to another health centre outside the study area. Secondly, at the regular three monthly home visits, the field assistant brings a list on which previously registered vaccinations are already printed. The vaccination card is inspected and the field assistant adds newly received vaccinations to the list. Such data are obtained only from children whose vaccination card can be seen at the home visit and, importantly, only from surviving children.27
Survival status—Observations on deaths are obtained from the paediatric ward and through the three monthly home visits. Mothers from Bissau often travel to visit relatives living in other parts of the country. Most women earn their living in the cashew harvest, going to the rural areas from April to June, and about 20% of the mothers and their children are travelling at any given time. It is always possible, however, to obtain information on the survival status of a child; if a child dies in the rural areas, the family in the city will be informed immediately.
We hoped to reduce a suspected negative effect of booster DTP vaccination at 18 months of age3 12 13 14 15 28
by revaccinating with BCG after DTP booster vaccination. Our main objective was to examine whether BCG revaccination would reduce child mortality by 30%.
When the trial started in July 2002 the vaccination programme in Guinea-Bissau comprised BCG and oral polio vaccine at birth; DTP and oral polio vaccine at age 6, 10, and 14 weeks; measles vaccine at 9 months; and booster doses of DTP and oral polio vaccine at 18 months. To prevent interference with the booster doses of DTP and oral polio vaccine, BCG revaccination was implemented at 19 months. We recruited children thought not to have tuberculosis and followed them for mortality and admissions to hospital until the age of 5 years. The analysis controlled for age and sex. We separately analysed children who were exposed to tuberculosis because they lived in the same house as a person with diagnosed tuberculosis.
In May 2006 the Ministry of Health organised a general measles vaccination campaign for all children under 15, when the mean age of children in the enrolled cohort was 5.4. We did not know about this campaign when we planned the BCG study. We have previously shown that measles vaccine has strong non-specific effects on child survival.1 3
As little follow-up was left and additional measles vaccination would make it impossible to measure any difference between BCG vaccinated children and controls, we decided to end follow-up in May 2006.
Children enrolled in the study were randomised to receive BCG vaccine (Statens Serum Institut, Copenhagen, Denmark) or no vaccine. The BCG vaccine was given in the standard intradermal dose of 0.1 ml, as recommended by WHO for this age group. In all other respects the two groups were treated equally; they received the same clinical examination and treatment before enrolment, the same home visits to assess adverse events and tuberculin skin test reactions, and the same access to consultations and essential drugs.
Our primary outcome was survival between 19 months and 5 years. Secondary outcomes included admission to hospital, BCG scarring, tuberculin reactivity, and malaria morbidity. As reported elsewhere we followed a subgroup of the study cohort for malaria infection.29
In another subgroup, we collected blood samples to measure changes in cytokine profile after BCG revaccination. We report the results for overall mortality, admission to hospital, and adverse events.
Based on data from 1990-8, we expected an annual mortality of 4.0% between the ages of 18 months and 4 years. With a significance level of 5%, a power of 80%, and 1:1 allocation of children to revaccination and control groups, we needed 3750 child years in each group with 150 deaths in the control group to document a 30% reduction in mortality. Assuming recruitment of 3000 children over a two year period with an average follow-up of three years, and no more than a 20% loss to follow-up, we expected to have at least 7500 child years of follow-up in the trial.
At 18-19 months of age, all children registered as living in the study area were assessed for BCG scar and tuberculin reaction. Tuberculin reactivity was measured with the Mantoux method, with an intradermal injection of 2 tuberculin units (RT23, Statens Serum Institute) and subsequent reading within 48-72 hours of bulae formation with the ball point technique. Children with a tuberculin reaction of 15 mm or more were referred to an experienced clinician to be examined for possible tuberculosis infection. If the child was considered to need tuberculosis prophylaxis or tuberculosis treatment according to a diagnostic scoring system this was supplied as currently recommended by the local tuberculosis programme.30
A field worker contacted the mothers/guardians of the children during the morning. They explained the study and filled in a questionnaire on background factors and vaccination status. They documented major risk factors for childhood mortality, including exposure to tuberculosis, recent infections, access to malaria drugs, ethnic group, mother’s education, and recent drug consumption. The mother was asked to bring the child to the health centre in the afternoon. When the study started in July 2002 four districts and two health centres participated. From the end of April 2003, two more districts and the third health centre in the area were included.
Inclusion criteria were residence in the study area, a Mantoux test reaction of less than 15 mm, and being sufficiently healthy to be vaccinated according to the clinician. Exclusion criteria were a reaction of 15 mm or more and not being sufficiently healthy to be vaccinated according to the clinician. Children did not have to have documentation of previous BCG vaccination or previous booster DTP vaccination to be included. We assumed that essentially all children had received BCG during the first year of life, and we were testing the effect of a general introduction of BCG revaccination in the community.
In the afternoon, the mothers/guardians of children presenting at the local health centre received an oral and a written explanation of the study from a physician. The physician performed a medical examination. Clinical examination and treatment was independent of consent and randomisation group. The clinical examination included anthropometrics and assessment of vaccination status from the vaccination card. Children who were missing doses of oral polio, DTP, or measles vaccines were advised to complete vaccinations. Children who were ill were not randomised. They received treatment according to local standards and were told to return when fully recovered.
The data manager, who was not involved in the recruitment of children, prepared bags of envelopes containing printed allocation numbers indicating to which of the two groups the child should be allocated. Allocation numbers could not be seen by the physician informing and obtaining consent from the mother/guardian.
If consent was obtained and if the clinician considered the child fit for participation, the mother or guardian was asked to pick an allocation number defining the randomisation group. This procedure has been used in several other trials in the study area to emphasise to the mother that she is participating in a randomised trial. Randomisation was even between the two groups, with block randomisation with 40 envelopes per bag.
There was no placebo for BCG and no “control” vaccine was given.
At the beginning of the study, we followed a group of 400 revaccinated and 400 control children weekly during the first month after enrolment to examine morbidity and to monitor possible adverse effects of the vaccination. Vaccination site, tuberculin reactivity, morbidity, and hospital visits were assessed after two and six months. We classified pustules ≥10 mm in diameter as a large local reaction to BCG injection compared with the normal local reaction to BCG (pustule, spontaneous drainage, and scarring).31 32
We did not expect that Koch-like reactions would be common in this age group. We did plan, however, that if too many adverse reactions were seen among children with a positive tuberculin test result (1-14 mm), we would include only children with negative results in the continuation of the trial.
Apart from the three monthly routine surveillance visits to all children in the study area, all children enrolled in the trial were visited two and six months after inclusion to assess tuberculin reaction and BCG scarring. The children were followed for admissions to hospital and mortality to age 5 years through the demographic and hospital surveillance systems. A trained local physician implemented a standardised verbal autopsy when one of the study children had died. We intended to censor deaths from injury in the survival analyses as these are unrelated to any immune stimulatory effect of BCG. None of the participating children reportedly died from injuries.
Admissions to hospital—Two assistants from the Bandim Health Project work at the national hospital and register all children admitted to the only paediatric ward in the country. Using the vaccination cards they attempt to identify all children from the study area.
—The project has maintained a tuberculosis surveillance system in the study area since 1997, and it has therefore been possible to conduct separate analyses for children who were or were not exposed to tuberculosis at home.30 33 34
—The project documented individual participation in all vitamin A supplementation and vaccination campaigns in the study area during the conduct of the present trial. The annual campaigns conducted in 2002,35
2004, and 2005 provided oral polio vaccine and vitamin A supplementation. These campaigns were based on small mobile teams visiting all houses in the study area. Each team was accompanied by a project field worker, who noted participation on a registration list for all children in the subdistrict. In 2003 there was no oral polio vaccine campaign but a campaign was organised in November 2003 with vitamin A supplementation and missing vaccines being distributed from 15 fixed posts in the area.36
At each post, project field workers documented participation. Participation in this campaign was considerably lower than in the house to house campaigns. In May 2006, a campaign with measles vaccine administered to all children aged 6 months to 14 years was conducted in Guinea-Bissau. This campaign with fixed posts also included distribution of vitamin A supplementation to all children aged 6 months to 4 years and mebendazol to children aged 1-4 years. Project field workers at all fixed posts documented participation. The vaccination status and survival of all children less than 5 years was assessed in connection with these campaigns.
Conduct of the trial
As will be apparent from the accompanying commentaries and editorial, the present publication is controversial. We had no funding for a data monitoring and safety board and there were no predefined stopping rules. The senior author (PA), who is not a clinician and not involved in clinical care related to any child in the trial, monitored mortality registered by the routine registration system quarterly as a check of safety. As previous experience has shown large and unexpected effects of vaccinations on mortality3 15 37
we thought it essential to identify unexpected safety issues when a potential new vaccination strategy is tested in this environment with high childhood mortality.
When we checked data in the beginning of April 2004, we identified a sudden increase in mortality starting at the end of 2003. Eighteen children in the BCG vaccination and four in the control group had died between November 2003 and March 2004, whereas there had been nine and 14 deaths, respectively, in the preceding 14 months since the start of the trial. We decided to stop the trial temporarily to prevent a possible but unknown risk to more children; at that time only 130 of the planned 3000 children had not been enrolled. The increase in mortality apparently started in November, when several campaigns with vitamin A supplementation, missing vaccinations, and iron treatment had been implemented, and at the same time a major measles epidemic started in the area. In an attempt to explain the unexpected findings we used mid-November 2003 as the starting point of the analyses. Use of any month back to June 2003 as cut point between the two periods, however, produced a significant inversion in mortality trends.
When we discovered the increase in mortality we carried out a follow-up assessment of survival of all children in the study. Free access to clinical care and essential drugs, which initially had been planned for the first year after enrolment, was extended for the full duration of the study. With longer follow-up there was no difference between the groups. At the same time we noted that overall mortality in the trial was considerably lower (2.5% a year) than originally anticipated (4% a year); no further corrective measures were therefore introduced. As we could not explain these findings and because the addition of another 130 children would matter little to the outcome of the trial, we decided not to restart enrolment.
Because of the unexpected cluster of deaths, we initiated a series of explorative analyses to clarify possible environmental causes. The timing of the trial coincided with major health interventions, campaigns, and epidemics (see legend to fig 1). The explorative analyses were limited by the routine data available about these events; the detection of epidemics other than measles was restricted by the limited diagnostic data collected at the paediatric ward. The paediatric ward was totally overbooked by measles cases in precisely the months of the cluster.
It should be noted that the statistical analysis of the cluster was conducted with a method that does not depend on setting specific dates for defining the cluster (see below). The explorative analysis of possible causes, however, had to be restricted to the period in which the potential cause was operational. When the cluster was detected it looked like something had happened in November 2003; a vitamin A supplementation and missing vaccinations campaign had occurred in the third week of November,36
the measles epidemic had gained strength around this time, and iron supplements had been distributed in mid-November. We therefore focused our explorative analyses of increased mortality on the period mid-November 2003 to March 2004. Hence, July 2002 to mid-November 2003 was the period before the increase and April 2004 to May 2006 the period after the increase.
We planned to measure the effect of BCG revaccination at age 19 months, after DTP vaccination, which was usually administered at 18 months. After the protocol was originally written, we found that the sequence of vaccinations—for example, DTP vaccine given after measles vaccine—has a major impact on the mortality effect of a trial.3 12 16 38
We therefore also conducted explorative analyses of the sequence of vaccination and the effect of other vaccines administered after enrolment in the trial.
As defined by the protocol, we compared admissions to hospital and mortality rates in Cox proportional hazards analyses, controlling for age as underlying time and sex, to estimate incidence rate ratios for admission to hospital and hazard ratios for mortality for the two arms of the study. Results were presented as hazard ratios with 95% confidence intervals. The assumption of proportionality was tested in several models with age, time since enrolment, and calendar time as underlying scale. The Kaplan-Meier method was used to depict crude cumulative mortality curves.
Children were followed to death, migration, age 5 years, or the general measles vaccination campaign in May 2006, whichever came first. The analysis of the increase in mortality was censored 1 April 2004—that is, at the time we noted an unusual distribution of deaths. We used the non-parametric total-time-on-test plot and total-time-on-test statistics for censored data39
to test the hypothesis of an increase in mortality. The total-time-on-test determines whether the mortality intensity is constant versus the alternative (increasing or decreasing). The test does not need to split the follow-up into separate periods and thus avoids the problem of defining cut points for testing an increased or decreased mortality rate. For the test plot, boundaries corresponding to a Kolmogorov-Smirnov-type test can be drawn and if the total-time-on-test curve crosses any of the boundaries the hypothesis of a constant mortality rate can be rejected.39
For the total-time-on-test we used calendar time as the time scale with time unit equal to days and interpreted a significant result as an increase or decrease of mortality during the study period. A separate test was calculated for each randomisation group.