Good News: Microfinance Research Not As Bad As Medical Research

Jonathan Morduch brings us good tidings for the new year.

Rich Rosenberg blogged yesterday about an Atlantic article that profiled John Ioannidis’s critique of medical research. The article reminded me of a meeting held in Washington a few years ago. Consumers and producers of microfinance impact studies were brought together to discuss the research agenda. One participant, who is not a researcher, concluded dolefully that microfinance research lags far behind medical research.My immediate thought was that that claim was probably false: not because microfinance research is so far ahead, but because much medical research seems full of problems. If you read beyond the newspaper headlines, it’s usual to see simple correlations between health conditions and a given diet/activity/lifestyle quickly---and falsely---assumed to be causally determined. Sample sizes are small. Lots of hypotheses get tested, but just a few get published.

In contrast,

The [randomized microfinance evaluations] do better because they are designed from the ground up to test a particular (and narrow) set of hypotheses. That greatly curtails the opportunity for “fishing expeditions” and the chance that one of your 57 hypotheses happens to be statistically significant by a fluke. The questions tend to be far more focused in the microfinance context. Does access to microfinance increase business profit? Business investment? Household consumption? Those microfinance hypotheses usually stem from a clear theoretical model and should show up in clear patterns. That’s far different from many medical studies, in which a much greater range of plausible hypotheses exist (along with a greater range of incorrect hypotheses).

Without disagreeing with Jonathan, I would probably have accentuated the negative in comparing research on microfinance and medicine. In my experience, John Ioannidis's message that a lot of research published in prestigious journals is wrong does carry over to economics in general and microfinance in particular. Cases in point are the papers that Jonathan and I replicated. Ioannidis helps us understand why something many thought proved---that microcredit reduces poverty on average---was not. But, with Jonathan, I would also sound this positive message: RCTs are a cut above---even in medicine.