Impact Evaluation in Africa (for Africa and by Africans)

The Journal of African Economies has a new special issue out in November on "Impact Evaluation in Africa", co-edited by Marcel Fafchamps of Oxford and  Andrew Zeitlin of Georgetown who is a part-time colleague here at CGD.While each of the papers is interesting on its individual merits, the collection itself addresses two broad debates.1. What is impact evaluation, and how should we be doing it?All the papers in the volume implicitly subscribe to what you might call the experimental social scientists’ mantra: “To find out what happens when you change something, it is necessary to change it.”  (ht Andrew Gelman)Note that that does not mean you need to randomize it.  Rather, what the papers in this volume share is a healthy preoccupation with clean identification -- stats jargon for demonstrating that in a particular case, correlation actually is causation.Contrast this with recent work at DFID that seeks to "broaden the range of designs and methods for impact evaluation". This appears to mean focusing less on quantitative data and counterfactual reasoning.  This JAE collection is a nice riposte to such efforts to water down the meaning of the term impact evaluation.2. What can impact evaluation do?One criticism of impact evaluations in development economics -- one that I've been known to utter myself -- is that they focus on small things.  To get at causal mechanisms it's often easier (and for academic work, even preferable) to work in somewhat artificial settings like laboratory experiments, or evaluations of small-scale pilots, rather than analyzing actual public policies at a state or national level.The special issue contains two great examples of economists' best efforts to resist that trade-off, using quasi-experimental methods to evaluate big government policies at scale, while still paying careful attention to causality.  Adrienne Lucas and Isaac Mbiti's  paper on the impact of Kenya's "free primary education" reform on gender differentials in enrollment looks at a nation-wide policy of huge social importance and relevance to the entire region.  Similarly, John Hoddinott and co-authors' analysis of Ethiopia's productive safety net program of transfers to small farmers is a laudable effort to bring impact evaluation methods to bear on observational data from a huge government program.Lastly, it's worth noting that the special issue brings together papers co-authored by some of the most interesting African scholars doing  serious applied research in development -- Isaac Mbiti of Kenya, Harounan Kazianga of Burkina Faso, and Alemayehu Seyoum Taffesse of Ethiopia.It's great to see impact evaluation in Africa being done rigorously, by some of Africa's top scholars, and not shying away from big questions of relevance to African policymakers.