As a Matter of Policy, Put Policies to the Experimental Test

Columnist Tim Harford in today's Financial Times (free with registration, I think):

It is a shame, then, that there is so little appetite from politicians for the same standards of evidence outside medicine. In fact it is more than a shame – it’s a scandal. While randomised trials are not going to tell us when to raise interest rates or get out of Afghanistan, there are many policies that could and should be tested with properly controlled trials. Is Jamie Oliver right to emphasise healthy school meals? Run a trial. Should young offenders be sent to boot camp, or to meet victims of crime? Run a trial. What can we do to persuade households to use less electricity? Run a trial.Yet such trials are not common in the US, and downright rare in the UK. There is no financial, ethical or practical excuse for this. Trials are cheap. (Even if they were expensive, solid practical knowledge is well worth paying for.) This is not a question of carrying out dangerously speculative crank experiments, but simply adding the essential ingredient of randomisation to a standard pilot project that would have happened anyway. Randomising is often what distinguishes proper evidence from statistical mush, by removing biases in the setting of experiments – such as running pilots only in the most needy areas.

What I like about this column is that Harford does not allow himself to become bogged down in the quagmire questions about what randomized controlled trials (RCTs) can and can't tell us. That has been my focus as I have worked to understand what we know about the impacts of microfinance. He takes RCTs' superiority as given (where they are practical) and proceeds to a moral call: those who advise and make policy on everything from education to child rearing should institutionalize experimentation. That, in order to support vital societal learning about what helps children learn math, say, what causes breast cancer, or whether sleeping face down kills babies.