WASHINGTON, 19th July 2022—The effects of most international development spending remain unknown, despite the potential for evaluation to save governments and aid agencies hundreds of millions of dollars and improve lives, according to a new large-scale analysis.
Experts from governments, aid agencies, leading universities, and nongovernmental organizations across the world—including the World Bank, the African Population and Health Research Center, Yale and Harvard Universities and many others, as well as current and former Ministers from low- and middle- income countries—are calling for more funding to be allocated to rigorously evaluating how well a policy or program works.
The group’s report, “Breakthrough for Policy Use: Reinvigorating Impact Evaluation for Global Development,” published today by the Center for Global Development, examines how evidence is used in aid and development programs, drawing on findings from more than 150 studies, as well as consultations with nearly 100 policymakers and researchers.
Experts found that—while rigorous impact evaluation can now be done quickly and at a relatively low cost, with huge potential for returns—just 10 percent of aid assessments carried out can be characterised as impact evaluations. Evaluations often prioritize trip reports and meeting summaries, rather than measuring progress and effects of interventions.
The report outlines how evidence from evaluations is leading to policy change with tangible, large-scale impact on people’s lives; informing policies to help lower the price of mosquito nets and deworming medicines for those who need them and to create low-tech remote education programs during COVID-19, for instance. But the experts say despite this progress—and the 1,600 evaluations of international development policies published in 2020—there is a long way to go.
“COVID-19 has made it clearer than ever that making decisions based on incomplete or outdated evidence costs lives and livelihoods. Researchers need to understand the real-time decisions policymakers are making every day and how evidence can help them, whether they’re responding to pandemics, trying to improve girls’ education, or collecting taxes. What’s more, the potential return is immense: $1 million spent on proper impact evaluation could save hundreds of millions in mistargeted or ineffective spending.” said Amanda Glassman, executive vice president at the Center for Global Development and co-chair of the working group.
When impact evaluations are conducted, they aren’t always useful to, or used by decision makers, the experts found, as they often fail to respond to policymaking priorities, questions, and timelines.
A key issue is that researchers based in the global south are commonly excluded from evaluations in their own countries, fuelling calls to “decolonize development.” Three-quarters of published impact evaluations of social policies in low- and middle-income countries do not have authors based in low- or middle-income countries.
“We now know the benefits to involving researchers with local expertise in research and evaluation projects: you identify more relevant questions, get more useful findings, and policymakers in developing countries are more likely to trust and use results when deciding how to spend limited resources,” said Santhosh Mathew of the Bill & Melinda Gates Foundation’s India Country Office.
“We’re past the days where it makes sense to design a development program in Washington, deploy it in Kenya, and assess it in Washington. We don’t see Kenyan researchers as the main authors of studies on US policy. Why should we accept different trends elsewhere? Researchers with local expertise are there and ready, but they are often overlooked,” added Dr. Catherine Kyobutungi, Executive Director of the African Population and Health Research Center and member of the working group who produced the report.
Other key findings include:
- Impact evaluations are not carried out in the areas where most aid is spent. While over one-third of official development assistance is spent on transportation, energy, and civil society, for instance, just 7 percent of development impact evaluations are in these sectors.
- A staggering two-thirds of impact evaluations do not include gender or equity analyses.
- Only around one in five impact evaluations consider the impact of a program per dollar spent on it.
The expert group lays out recommendations for solving these issues. While action in this area will be largely led by low- and middle- income countries, they call for major donors, including USAID, the World Bank and philanthropies to provide funding for evaluations which are relevant to the priorities of low- and middle-income country policymakers; build incentives for evidence use into their own practices; and support partnerships with locally immersed researchers.
“We hear a lot about what the ‘best buys’ in development are, whether this issue or that issue should get the most money. But the real best buy is making sure we know what really works, and that policymakers know it too,” concluded Ruth Levine, co-chair of the working group and chief executive officer of IDinsight.
The full report is available at www.cgdev.org/evidence-to-impact.