HIGHLIGHTS
- Abt conducted a first-of-its-kind evaluation on the effectiveness of the USDA’s McGovern-Dole school feeding and literacy interventions to improve student performance in Mozambique.
- Abt used a mixed methods design to quantitatively examine the causal impact of interventions with a qualitative investigation to understand the “what” and “why” of implementation and outcomes.
- Abt’s evaluation found that the projects improved students’ literacy, though they had little impact on attendance and attentiveness.
PROJECT
McGovern-Dole Mozambique Program Impact Evaluation
The Challenge
In most provinces in Mozambique, schools are overcrowded, student-teacher ratios are high, and reading and math test scores are low. Many Mozambican children suffer from food insecurity, including poor diet quality, which hinders their ability to learn.
The U.S. Department of Agriculture (USDA) McGovern-Dole program takes steps to improve student literacy by enhancing teacher and school administrator training, creating school meal programs, and supporting student attendance and attentiveness through nutrition and health interventions. To determine the effectiveness of these strategies, Abt led an evaluation of two McGovern-Dole projects in Mozambique.
The Approach
USDA/Foreign Agricultural Service supported an evaluation of the effectiveness of McGovern-Dole interventions: one implemented by Planet AID, Inc in Maputo province from 2015-2020 and one implemented by World Vision, Inc. in Nampula province from 2015-2021.
Abt’s evaluation focused on the outcomes for students in third grade, who had up to two years of exposure to the interventions, and students in the fifth grade, who had up to four years of exposure to the interventions.
Abt used a quasi-experimental difference-in-differences impact evaluation design to compare average school-level performance at the beginning of the projects to average school-level performance at end of the projects four years later on the following student outcomes: literacy, attendance, and attentiveness. Abt also addressed secondary evaluation questions concerning the effects on teachers and schools.
Additionally, Abt assessed whether impacts differed by student gender and age by analyzing the intervention effects on boys versus girls as well as younger students compared with older students.
The Results
Through our evaluation, we found that students in schools that implemented the combined school feeding and literacy interventions substantially improved their reading comprehension scores, even though attendance and attentiveness appeared unchanged. Students in schools implementing only the school feeding intervention did not improve average mean reading comprehension relative to students in comparison schools, suggesting that teacher training likely explains the impacts on student reading performance. The lack of effects on attendance and attentiveness suggests that the impact on literacy was not because students were in school more often or more attentive but that the instruction was more effective.
Abt also found little difference between the results for boys and girls and younger and older students. However, Abt found that schools could better support the girls. Impacts on student reading comprehension scores did not vary in meaningful ways between boys and girls, but literacy outcomes show that boys performed better than girls in general. This reading achievement gap is consistent with the general pattern in Mozambique, particularly in rural settings.
Ultimately the study team offered the following recommendations for McGovern-Dole programming, many of which have applicability beyond Mozambique:
- Expand stakeholder awareness of the McGovern-Dole program theory of change and engagement in identifying student and school outcomes of interest
- Improve teachers’ capacity to support student development of early reading skills
- Include local African languages in early-grade literacy development
- Increase gender sensitivity and improve instruction focusing on girls.