Important null results in development economics | VoxDev

Skip to main content

More

New AI search

Ask a question or use keywords

Search

Search

Supported by

Important null results in development economics

VoxDev Blog

Published 11.04.25

Oliver Hanney

Managing Editor, VoxDev

Despite the bias against publishing null results, they are important for policy, helping to kill bad ideas.

I recently had the pleasure of hosting the closing panel at the CSAE Conference. I will write a separate post summarising this discussion, but one point that stuck out was made by Ranil Dissanayake, who highlighted a key role that evidence can play - killing bad ideas. It’s one thing encouraging a policymaker to fund and implement a new programme, it’s much easier to convince them that an existing programme is not having it’s intended impact. Whether it’s providing textbooks or the Millennium Villages Project, economic research has played an important role in stopping ineffective interventions, freeing up resources to (hopefully) be put towards more productive uses.<br>While these results ended up influencing policy, there are a host of well-documented reasons why there is a bias against publishing null results in economics journals. I am not going to get into these here. Instead, I thought I would highlight research featured on VoxDev that did not find statistically significant impacts. In my experience, these often result in more interesting, thoughtful summary articles. Instead of going through the motions, authors need to more carefully interrogate why there was no impact. This often means they adopt a more policy-friendly lens, thinking through implementation and other factors more important to policymakers than academics more carefully.<br>Microfinance: Transformative impacts? Nulls across studies<br>One area which held much promise and hype, but has since come under greater scrutiny, is microcredit. Evidence from the first generation of microcredit RCTs, highlighted in our recently updated VoxDevLit on Microfinance, highlights that access to microcredit does not translate into increases in profit, income, labour supply, average consumption or gains in social indicators. Rachael Meager outlines the appropriate method to understand these average impacts across studies here.<br>At this point, it is worth highlighting that there are different types of nulls – many of the studies in the microcredit literature are imprecise, meaning that while we can’t rule out zero effects (i.e. the null) we also can’t rule out the possibility there were large effects that studies were not powered to pick up – for examples of precise nulls, read on.<br>These insights from the first round of RCTs on microcredit have encouraged researchers to dig deeper and interrogate findings more carefully – rather than assuming this intervention would benefit all, asking when it could work and for who.<br>Improving parenting practices: Cheap alternative? Null at scale<br>Can we improve early childhood parenting practices at scale with low-cost technology? Improving parenting practices through home visits and group meetings has benefitted child development in low- and middle-income countries, but these policies are expensive and impractical to scale. Given this, Irma Arteaga, Andreas de Barros and Alejandro Ganimian evaluated a programme in India which sent automated phone calls with parenting advice to thousands of caregivers of young children, aiming to replicate the success of intensive home-visit programmes but at a fraction of the cost. However, the evaluation found the intervention was not effective in improving child-rearing practices or child development outcomes​​. Cheap, easily scalable interventions can fail to achieve impact at scale. Recently, John A. List discussed his work on this “voltage drop”, and how we can identify effective policies at scale.<br>Free contraception: Popular hypothesis meets precise null<br>A popular hypothesis in development is that unmet need for contraceptives is a key driver of high fertility rates. Pascaline Dupas, Seema Jayachandran, Adriana Lleras-Muney and Pauline Rossi tested this by conducting a large three-year RCT in rural Burkina Faso, which provided free access to all modern contraceptives to thousands of married women aged 17 to 35 via local health clinics. The outcome was striking: removing all financial constraints to accessing contraceptives had no detectable effect on birth rates, suggesting that in this context, high fertility reflected desired family size rather than lack of contraceptive access​.<br>Laptops: 10 years on, still a null<br>The One Laptop per Child programme was a widely publicised initiative that gave students in over 40 developing countries a personal laptop. A large scale experiment in Peru found no effects on test scores or enrolment after 15 months. Santiago Cueto, Diether W. Beuermann, Julian Cristia, Ofer Malamud and Francisco Pardo follow up 10 years on, and find that laptops did not improve academic...