Realist review to understand the efficacy of school feeding programmes

doi:10.1136/bmj.39359.525174.AD

BMJ. 2007 October 27; 335(7625): 858–861.

Published online 2007 October 22. doi: 10.1136/bmj.39359.525174.AD

PMCID: PMC2043412

Realist review to understand the efficacy of school feeding programmes

Trisha Greenhalgh, professor of primary health care,¹ Elizabeth Kristjansson, associate professor,² and Vivian Robinson, doctoral candidate³

¹Department of Primary Care and Population Sciences, University College London, London N19 5LW

²School of Psychology and Institute of Population Health, University of Ottawa, Canada K1N 6N5

³Institute of Population Health, University of Ottawa

Correspondence to: T Greenhalgh ; Email: p.greenhalgh/at/pcps.ucl.ac.uk

Accepted September 19, 2007.

This article has been cited by other articles in PMC.

A recent Cochrane review found that school feeding programmes significantly improve the growth and cognitive performance of disadvantaged children. Trisha Greenhalgh,Elizabeth Kristjansson, and Vivian Robinson look more closely at the highly heterogeneous trials to see what works, for whom, and in what circumstances

Our Cochrane review of school feeding programmes in disadvantaged children included trials from five continents and spanned eight decades.¹ Although we found that the programmes have significant positive effects on growth and cognitive performance, the trials had many different designs and were implemented in varying social contexts and educational systems; by staff with different backgrounds, skills, and cultural beliefs; and with huge variation in the prevailing social, economic, and political context. Simply knowing that feeding programmes work is not enough for policymakers to decide on the type of intervention that should be implemented. We therefore looked at the trials more closely to determine the aspects that determine success and failure in various situations.

Review methods

We analysed the 18 studies (reported in 29 articles) included in our Cochrane review² ³ ⁴ ⁵ ⁶ ⁷ ⁸ ⁹ ¹⁰ ¹¹ ¹² ¹³ ¹⁴ ¹⁵ ¹⁶ ¹⁷ ¹⁸ ¹⁹ ²⁰ ²¹ ²² ²³ ²⁴ ²⁵ ²⁶ ²⁷ ²⁸ ²⁹ ³⁰ using the methods of a realist review. Realist review exposes and articulates the mechanisms by which the primary studies assumed the interventions to work (either explicitly or implicitly); gathers evidence from primary sources about the process of implementing the intervention; and evaluates that evidence so as to judge the integrity with which each theory was actually tested and (where relevant) adjudicate between different theories.³¹ ³²

We read, re-read, and discussed the papers and constructed a matrix on an Excel spreadsheet to collate information for each trial on:

Study design, sample size, and outcome data
Nature of the experimental and (where present) control interventions, including intensity and timing
Process detail, especially comments on the fidelity of the intervention, changes made by staff on the ground, and reasons for those changes
Aspects of the study's history and context, especially those highlighted as important by the study's authors
Any theories or mechanisms postulated (or assumed) by the study's authors to explain the success or failure of the programme.

We considered relevant data first on a trial by trial basis in terms of the interaction between context, mechanism, and outcome, and then across the different trials to detect patterns and idiosyncrasies. We discussed preliminary conclusions and synthesised key findings using a narrative and interpretive approach.³³ We identified four broad areas relevant to this analysis: the historical context of school feeding programmes (see bmj.com), theories to explain the success of particular programmes (box 1), theories to explain their failure or qualify a partial success (box 2); and measurement issues (see bmj.com).

Box 1 Process factors that seem to enhance efficacy of school feeding programmes

Strong process evidence across many trials

Target group has clear nutritional deficiency (usually, inadequate energy intake) and trial is oriented to correcting this rather than to short term hunger relief
Well organised schools that form part of an efficient distribution chain for the supplement
Intervention developed with local teams rather than designed by distant experts
Supplement is piloted to exclude intolerance and confirm palatability and acceptability
Measures are in place to ensure that the food supplement is consumed (eg close supervision of eating)
In disaffected young people, attention is paid to social aspects of the meal

Limited process evidence from one or few trials

Use of local ingredients and cooking methods
In extreme poverty, intervention is designed so that attending school is more economically viable than keeping children at home
Intervention seeks to induce a change in home diet by educating or inspiring children

Possible factors that might be tested in future studies

Better nutrition and health literacy in this generation reduces intergenerational cycle of poverty

Box 2 Process factors that seem to reduce the efficacy of school feeding programmes

Participants not aware of, signed up to, or trained to take account of the research dimension of the trial
Study design involves role conflict or ethical difficulties for staff (eg requirement to serve nutritious meal to some but not all undernourished children)
Insufficient measures in place to reduce confounding (eg controlling for benevolent attention)
Adverse prevailing policy climate (eg policy conflicts with trial protocol or prompts rebranding of mainstream activity to gain research funding)
Measurement issues (see bmj.com)

Theories of why school feeding programmes work

Long term correction of nutritional deficiencies

Nine trials in our sample were based on a theory that school feeding corrects overt nutritional deficiencies, which in turn improves brain growth and performance.⁵ ⁶ ⁷ ⁸ ¹¹ ¹⁶ ¹⁷ ²⁴ ²⁵ Such trials assumed that food supplements should be rich in energy, protein, and vitamins and continued for a substantial period before their effect can be shown.

Most trials in low and middle income countries that set out to correct nutritional deficiency had positive results, although in two measurement of weight was distorted by oedema associated with kwashiorkor.¹³ ²⁰ Trials in high income countries had mixed results. Long term nutritional supplementation generally affected growth (and sometimes performance) when the children were genuinely undernourished, but not when they were not. For example, two trials of school milk supplements in Britain in the 1920s (a time of economic recession, high unemployment, and food shortage) showed a significant positive effect on children's growth⁵ ⁷; but a trial in the 1970s showed no significant benefit, according to our statistical analysis, with the same supplement.¹ ²

Short term hunger relief

Two trials in low and middle income¹⁵ ²⁰ and three in high income³ ⁴ ¹⁰ countries were built around the theory that school feeding leads to short term rises in blood (and hence brain) glucose levels, which counteracts the negative effect of hunger on concentration, memory, motivation, and other psychological prerequisites for learning.³⁴ Overall, the effect of interventions built on a hunger relief theory was not constant across different areas of performance (verbal, non-verbal, mathematical) or across studies.

Children feel valued and looked after

Powell criticised studies that failed to control for the effect of benevolent attention and recommended that, at the very least, the control group should receive a low energy drink or piece of fruit along with teacher or researcher attention.²³

Bro and colleagues did two studies of “at risk” teenagers (school drop-outs, drug users, teenage parents, or from families with other social problems) who, though not malnourished, rarely ate before school. They showed that a generous breakfast cooked in a practical class before the lesson began improved attention to set tasks.³ ⁴ Qualitative process data suggested that a meal at school can be a social event that engages, motivates, and stimulates the students.

Reduced absenteeism

Of the studies that measured attendance objectively, most of those in low and middle income countries showed significantly higher attendance levels in supplemented groups,¹² ²⁰ ²⁴ whereas studies in high income countries had non-significant effects on attendance.⁶ ⁸ ¹¹

Improved school diet inspires improved home diet

In one study, when children were given breakfast at school, their families subsequently bought more milk, meat, fish, and high vitamin C foods, whereas the families of a control group did not change their buying habits.¹¹ This study was done in Canada at a time of rapid social change and rising affluence; two other studies in low and middle income countries (where parents presumably had less choice in what they bought) found no changes in home eating patterns.¹⁶ ²⁴

Improved literacy reduces intergenerational cycle of poverty

Several authors speculated about a longer term impact of school feeding—namely, that it would lead to higher literacy rates, which would offer the chance to break the cycle of poverty, giving the next generation of children better opportunities for good nutrition and health.¹⁴ ¹⁶ Such an effect is difficult to measure because of the long time frame involved, but it should be borne in mind in future research.

Theories why school feeding programmes do not work

The commonest reason for failure was that the programme was built around a misguided theory (such as correcting a nutritional deficiency that did not exist) but other reasons may also apply.

Food offered is not consumed, or provides too little of the missing nutrient

Studies that piloted different supplements until they identified one that was readily consumed or that let children choose from a menu were, in general, more likely to improve growth.⁸ ²⁰ Very poor children rarely rejected food in any form, and in these studies the supplement generally had a significant effect.⁸ ¹⁶ ²² ³⁰ In contrast, those trials with adequate nutrients but less impact on growth generally documented incomplete consumption, sometimes because the children did not attend the meal.⁶ ¹¹

In one pilot study, 25% of children rejected a cows' milk supplement even when it was chocolate flavoured, strongly suggesting lactose intolerance.⁸ Use of a specially formulated low lactose milk supplement refined in response to the children's feedback on its palatability had a significant effect on growth.

Most trials in this review provided at least 15% of the recommended daily allowance of energy to the intervention group. Two studies that provided considerably less than 15% of the recommended daily allowance had no significant effect on weight.¹³ ²⁵ However, a study targeting calcium deficiency in teenage girls, which provided less than 15% of the recommended energy levels, did show a positive effect on the primary end point of height gain,¹⁷ suggesting that targeted correction of micronutrient deficiency may be effective.

Bioavailability

Low bioavailability was occasionally invoked as an explanation for lower than expected effect of a feeding programme. Grillenberger and colleagues, for example, proposed that the milk supplement might have decreased the absorption of iron and zinc.¹⁹

Compensatory reduction in food intake at other times

A few studies documented a compensatory adjustment in appetite (supplemented children ate less at the next meal).⁸ ²⁴ Four studies in which the benefit of supplements was less than expected were done in very poor areas in Peru,²⁰ Jamaica,²³ Kenya,²² and India.¹² The authors of two of these studies concluded that children who had been given a substantial supplement at school were provided with less food at home (substitution).¹² ²⁰ The authors of the Jamaican study looked for substitution at home in a second study 15 years later²⁴ and found no evidence that it was still occurring.

Supplementation occurs too late

Authors of a study published in 1962 speculated that the failure of their feeding programme in children aged 7-12 was because the children were “too far along the track of malnutrition” and recommended that subsequent studies should target younger children.¹³ Our statistical analysis, which showed significantly greater gains in weight in younger children,¹ supports the notion that the earlier feeding supplementation occurs, the better it is for growth, although this was not the case for cognitive outcomes.

Programme is not implemented as planned

Deviation from the study protocol can be an important problem. One study reported, for example, that: “It was originally intended that Group 1 would be a control group. However it was impossible to obtain cooperation without distributing some supplements to all the boys.”¹³ Asking school staff to withhold food from hungry children when others are getting fed, or trying to stop children sharing food with their friends, was poor study design as well as ethically questionable. Consultation with the target population at design stage seems to help prevent such problems by producing an intervention that engages staff and incorporates their practical wisdom of what is workable. Lieberman and colleagues, for example, originally intended to randomise their participants by pupil but a steering group drawn from the local community rejected this and they eventually allocated by school, with one experimental and one control school.⁶

Some studies that did not apply selection criteria to schools, and where there was weak or absent sign-up to the research dimension, reported a high rate of sample attrition or non-fidelity of the intervention.⁹ ¹⁰

Non-fidelity of the intervention was also explained by staff and senior management viewing the research as a way of gaining funding for a cash strapped organisation or community, and having little or no interest in the scientific elements. Shemilt et al, for example, evaluated a randomised controlled trial of school breakfast clubs in socioeconomically deprived parts of England in the early 2000s. At the time, there was a strong push for all schools in deprived areas to provide breakfast clubs. This led to the absurdity that “at second follow-up 72.2% of pupils in the intervention arm and 77.0% of pupils in the control arm had a breakfast club operating at their school.”¹⁰

Confounding or external variables

Several authors commented on factors—both internal (such as the wide variation in timing of puberty growth spurt² ¹⁷) and external (infections and infestations,²⁶ seasonal variation,¹¹ etc) that would have reduced the measured impact of the programme, making a (potentially) real difference non-significant.

Discussion

A complex, community based intervention inevitably operates at multiple levels, and controlled trials of such interventions must be interpreted in their appropriate historical and policy context. Although the factors listed in boxes 1 and 2 should be seen as preliminary and non-exhaustive, we believe they will be useful to policymakers who need to know not merely whether school feeding programmes work but what sort of programme (if any) to put resources into. Our analysis supports concentrating school feeding on pupils with documented nutritional deficiencies, and for a development phase (working in partnership with the local community to optimise and pilot an intervention) before the programme is tested in an experimental trial—a finding that fits with the UK Medical Research Council's recommendations on the design of complex interventions.³⁵ Consultation with practitioners and the local community may result in important changes in study design, which in turn may require a change in sample size, so should not be seen as mere formality.

Process data from some trials suggest that in situations of absolute poverty even severely malnourished children may not benefit from school feeding programmes because of substitution at home. In these very specific (and increasingly rare) circumstances, further research should take account of this theory. For example, the feeding protocol might be designed to provide a higher energy meal (to compensate for the food that will be withheld at home), or give food as a mid-morning snack (perhaps less likely to be substituted than a meal) or a different intervention might be used (such as rations to take home or income supplementation).

We focused only on the 18 trials included in the Cochrane review, but the analysis would undoubtedly be enriched by inclusion of descriptive studies, theoretical papers, and grey literature. Moreover, we were unable to distinguish between something that was not done and something that was done but not reported because of the stringent word count constraints of medical journals.

Is it time to shift the balance in what we define as quality from an exclusive focus on empirical method (the extent to which authors have adhered to the accepted rules of controlled trials) to one that embraces theory (the extent to which a theoretical mechanism was explicitly defined and tested)? If authors of trials of complex interventions were required to meet minimum quality standards for theory as well as method, far fewer systematic reviews might conclude that “more primary research is needed.”

Summary points

Systematic reviews rarely give detailed information on the context, mechanisms, and outcomes of interventions and the theories that underpin them
This realist review describes the theory and processes in 18 trials of school feeding programmes
Programmes should be aimed at children with documented nutritional deficiencies
Programmes are more likely to be effective when designed in partnership with the local community and interventions are piloted
In situations of absolute poverty even severely malnourished children may not benefit from school feeding programmes because they may receive less food at home

Supplementary Material

[extra: Measurement issues raised by the realist review]

Click here to view.

[extra: Historical context]

Click here to view.

Notes

We thank the authors of the original Cochrane systematic review. We also thank Ray Pawson and Penny Hawe for helpful comments on an earlier draft.

Contributors and sources: TG conceptualised the study, adapted the method from Pawson's original, drew up the original spreadsheet, conducted the initial realist analysis, and drafted the paper. EK and VR independently read all the primary empirical studies, contributed to a substantial revision of the realist analysis, and helped revise the paper. TG is the guarantor.

Funding: None

Competing interests: EK was given travel funds to present findings from this review at an experts seminar on school feeding in Rome and to attend an expert panel on school meals in Santiago.

References

1. Kristjansson E, Robinson V, Petticrew M, Macdonald B, Krasevec J, Janzen L, et al. School feeding for improving the physical and psychosocial health of disadvantaged elementary school children. Cochrane Database Syst Rev 2007;(1):CD004676.

2. Baker IA, Elwood PC, Hughes J, Jones M, Moore F, Sweetnam PM. A randomised controlled trial of the effect of the provision of free school milk on the growth of children. J Epidemiol Community Health 1980;34:31-4. [PMC free article] [PubMed]

3. Bro RT, Shank L, Williams R, McLaughlin TF. The effects of an in-class breakfast program on attendance and on-task behavior of high-school students. Child Family Behavior Therapy 1994;16(3):1-8.

4. Bro RT, Shank L, Williams R, McLaughlin TF. Effects of a breakfast program on on-task behavior of vocational high-school students. J Educ Res 1996;90:111-5.

5. Corry Mann HC. Diets for boys during the school age London: Medical Research Council, 1926.

6. Lieberman HM, Hunt IF, Coulson AH, Clark VA, Swendseid ME, Ho L. Evaluation of a ghetto school breakfast program. J Am Diet Assoc 1976;68:132-8. [PubMed]

7. Orr JB. Milk consumption and the growth of school-children. Lancet 1928;i:202-3.

8. Paige DM, Cordano A, Huang S-S. Nutritional supplementation of disadvantaged elementary school children. Pediatrics 1976;58:697-703. [PubMed]

9. Shemilt I, Mugford M, Moffatt P, Harvey I, Reading R, Shepstone L, et al. A national evaluation of school breakfast clubs: where does economics fit in? Child Care Health Dev 2004;30:429-37. [PubMed]

10. Shemilt I, Harvey I, Shepstone L, Swift L, Reading R, Mugford M, et al. A national evaluation of school breakfast clubs: evidence from a cluster randomized controlled trial and an observational analysis. Child Care Health Dev 2004;30:413-27. [PubMed]

11. Tisdall FF, Robertson EC, Drake TGH, Jackson SH, Brouha L, Ellis RG, et al. The Canadian Red Cross School Meal Study. Can Med Assoc J 1951;64:477-89. [PMC free article] [PubMed]

12. Agarwal DK, Agarwal KN, Upadhyay SK. Effect of mid-day meal programme on physical growth and mental function. Indian J Med Res 1989;90:163-74. [PubMed]

13. Bailey KV. Rural nutrition studies in Indonesia IX: Feeding trial on schoolboys. Trop Geog Med 1962;14:129-39.

14. Chandler AM, Walker SP, Connolly K, Grantham-McGregor SM. School breakfast improves verbal fluency in undernourished Jamaican children. J Nutr 1995;125:894-900. [PubMed]

15. Chang SM, Walker SP, Himes J, Grantham-McGregor SM. Effects of breakfast on classroom behaviour in rural Jamaican schoolchildren. Food Nutr Bul 1996;17:248-57.

16. Devadas RP, Jamala S, Surabhi A, Murthy NK. Evaluation of a food supplement to school children. Ind J Nutr Diet 1979;16:335-41.

17. Du X, Zhu K, Trube A, Zhang Q, Ma G, Hu X, et al. School-milk intervention trial enhances growth and bone mineral accretion in Chinese girls aged 10-12 years in Beijing. Br J Nutr 2004;92:159-68. [PubMed]

18. Du X, Zhu K, Trube A, Fraser DR, Greenfield AH, Zhang Q, et al. Effects of school-milk intervention on growth and bone mineral accretion in Chinese girls aged 10-12 years: accounting for cluster randomisation. Br J Nutr 2005;94:1038-9. [PubMed]

19. Grillenberger M, Neumann CG, Murphy SP, Bwibo NO, van't Veer P, Hautvast JGAJ, et al. Food supplements have a positive impact on weight gain and the addition of animal source foods increases lean body mass of Kenyan schoolchildren. J Nutr 2003;133:3957-64S.

20. Jacoby E, Cueto S, Pollitt E. Benefits of a school breakfast programme among Andean children in Huarez, Peru. Food Nutr Bull 1996;17:54-64.

21. Jacoby ER, Cueto S, Pollitt E. When science and politics listen to each other: good prospects from a new school breakfast program in Peru. Am J Clin Nutr 1998;67:795S-7S. [PubMed]

22. Murphy SP, Gewa C, Liang LJ, Grillenberger M, Bwibo NO, Neumann CG. School snacks containing animal source foods improve dietary quality for children in rural Kenya. J Nutr 2003;133(11 suppl 2):3950-6S.

23. Powell C, Grantham-McGregor S, Elston M. An evaluation of giving the Jamaican government school meal to a class of children. Hum Nutr Clin Nutr 1983;37:381-8. [PubMed]

24. Powell CA, Walker SP, Chang SM, Grantham-McGregor SM. Nutrition and education: a randomized trial of the effects of breakfast in rural primary school children. Am J Clin Nutr 1998;68:873-9. [PubMed]

25. Neumann CG, Bwibo NO, Murphy SP, Sigman M, Whaley S, Allen LH, et al. Animal source foods improve dietary quality, micronutrient status, growth and cognitive function in Kenyan schoolchildren: background, study design and baseline findings. J Nutr 2003;133:3941-9S.

26. Siekmann JH, Allen LH, Bwibo NO, Demment MW, Murphy SP, Neumann CG. Kenyan school children have multiple micronutrient deficiencies, but increased plasma vitamin B12 is the only detectable micronutrient response to milk or meat supplementation. J Nutr 2003;133:3972-80S.

27. Whaley SE, Sigman M, Neumann CG, Bwibo NO, Guthrie D, Weiss RE, et al. The impact of dietary intervention on the cognitive development of Kenyan school children. J Nutr 2003;133:3965-71S.

28. Zhu K, Zhang Q, Foo LH, Trube A, Ma G, Hu X, et al. Growth, bone mass, and vitamin D status of Chinese adolescent girls 3 y after withdrawal of milk supplementation. Am J Clin Nutr 2006;83:714-21. [PubMed]

29. Zhu K, Du X, Cowell CT, Greenfield H, Blades B, Dobbins TA, et al. Effects of school milk intervention on cortical bone accretion and indicators relevant to bone metabolism in Chinese girls aged 10-12 y in Beijing. Am J Clin Nutr 2005;81:1168-75. [PubMed]

30. Zhu K, Du X, Greenfield H, Zhang Q, Ma G, Hu X, et al. Bone mass in Chinese premenarcheal girls: the roles of body composition, calcium intake and physical activity. Br J Nutr 2004;92:985-93. [PubMed]

31. Pawson R, Greenhalgh T, Harvey G, Walshe K. Realist review--a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy 2005;10(suppl 1):21-34. [PubMed]

32. Pawson R, Greenhalgh T, Harvey G, Walshe K. Realist synthesis — an introduction ESRC working paper series .

33. Dixon-Woods M, Booth A, Sutton AJ. Synthesizing qualitative research: a review of published reports. Qual Res 2007;7:375-422.

34. Simeon DT. School feeding in Jamaica: a review of its evaluation. Am J Clin Nutr 1998;67:790-4S.

35. Campbell M, Fitzpatrick R, Haines A, Kinmonth AL, Sandercock P, Spiegelhalter D, et al. Framework for design and evaluation of complex interventions to improve health. BMJ 2000;321:694-6. [PMC free article] [PubMed]

Articles from BMJ : British Medical Journal are provided here courtesy of
BMJ Publishing Group