The Guardian’s Poverty Matters Blog quotes Michael Clemens and his paper on the evaluation of the Millennium Villages development project.
From the article:
Until now, evaluations have been cautiously positive, such as this one (pdf) by the Overseas Development Institute. Now, Michael Clemens, one of the authors of the paper, is forthright: "The paper deliberately makes no conclusion about the wisdom or effectiveness of the intervention itself, except for the modest claim that the intervention shouldn't be massively scaled up until its effects have been reliably estimated. To me that's uncontroversial; Africans have urgent needs, but they have urgent needs for things that work, and many of them have been disappointed by well-intended outsiders in the past." Ouch. In the polite world of development wonkery, this counts as a pretty tough snub.
Clemens criticizes the midterm evaluation of the project published in June 2010 for claiming credit for positive trends which were happening in other districts without the project. In particular, he points out what anyone visiting Africa notices – mobile phone ownership is growing exponentially – so it seems odd to claim, as the midterm evaluation does, that this increase, seen in the Millennium Villages as elsewhere, is an 'impact' of the project. Clemens argues that the project is not being rigorously evaluated, so we can't really know what it is achieving. And he proposes a new method of evaluation.