1 The Question

Over two decades, randomised controlled trials (RCTs) and natural experiments transformed development economics, culminating in the 2019 Nobel Prize to Banerjee, Duflo, and Kremer for an "experimental approach to alleviating global poverty." Thousands of rigorous studies now tell us what worked in a particular place at a particular time: deworming raised school attendance in western Kenya; a specific microcredit product did little for average consumption in Hyderabad; a teacher-incentive scheme raised test scores in rural Andhra Pradesh. The credibility of each estimate, within its setting, is hard to dispute.

The contested question is what happens next. When a finance minister in Lagos or a donor in Washington reads that an intervention "worked" in a trial elsewhere, are they learning something about their decision? Do credibly identified causal effects travel across regions, populations, implementing organisations, and scales? Or does the very precision that makes these estimates trustworthy at home make them treacherous abroad? This is the external-validity debate, and in development economics it is not academic hair-splitting: billions of dollars and millions of lives ride on whether evidence generalises.

2 The Case That Estimates Do Travel (Enough to Act On)

Credible local knowledge beats uncredible global theory. The first argument is comparative. Before the credibility revolution, development policy rested on cross-country growth regressions and structural models riddled with endogeneity, producing conclusions that flipped with every new specification [Deaton, 2010]. An RCT may be local, but it answers a question without bias. Proponents in the tradition of Banerjee and Duflo [2011] argue that accumulating many well-identified local answers is a sounder route to knowledge than grand, untestable generalisations. A clean estimate that applies somewhere is more useful than a biased estimate that pretends to apply everywhere.

Mechanisms generalise even when magnitudes do not. A sophisticated version of the pro-generalisation case distinguishes effect sizes from mechanisms. The exact number a

0.3 standard-deviation test-score gain may not transport, but the behavioural mechanism (information relieves a binding constraint; incentives change effort) often does. Replication across sites is precisely how the field separates the portable mechanism from the local magnitude. When deworming, iron supplementation, and information campaigns are tested repeatedly across countries, the pattern of results teaches a general lesson even if no single coefficient is universal.

Theory and structure can extend reach. Advocates also point out that external validity is not binary but a modelling task. With a credible local estimate plus a theory of how effects depend on context, one can extrapolate responsibly reweighting complier populations, adjusting for differences in baseline conditions, or embedding the experimental moment in a structural model. The marginal-treatment-effect and policy-relevant-treatment-effect frameworks exist exactly to map a local estimate to a new policy question. On this view, "do estimates travel?" is the wrong question; the right one is "what must we assume, and measure, to carry them?"

3 The Case That They Do Not (And Pretending Otherwise Is Dangerous)

The same parameter genuinely differs across contexts. The strongest sceptical argument is empirical, not philosophical. Pritchett and Sandefur [2015] show that when the same intervention is evaluated in multiple settings, the treatment effects differ so much that the average across studies is a poor predictor of the effect in any new site "context matters" is not a hedge but a measured fact. Vivalt [2020] quantifies this directly: pooling impact evaluations within an intervention type, she finds substantial heterogeneity, so that the predictive value of an existing body of evidence for a new context is limited, and government-implemented programmes show systematically smaller effects than researcher-run pilots.

Scaling changes everything. Even holding the population fixed, an effect estimated in a small trial can vanish or reverse when the programme goes to scale. Bold et al. [2018] ran the same contract-teacher intervention in Kenya through an NGO and through the government: the NGO version replicated the famous positive effect; the government version produced no effect at all. General-equilibrium feedbacks, political-economy frictions, and implementation quality all absent from the pilot dominate at scale [Muralidharan and Niehaus, 2017]. The pilot estimate is internally valid and externally irrelevant to the policy actually contemplated.

The complier population is local by construction. A natural-experiment or IV estimate identifies a local average treatment effect the effect for those whose behaviour the instrument shifted [Deaton, 2010]. The compliers in a Kenyan deworming study (children near the margin of attendance, given that delivery system, that disease burden) are not the compliers anywhere else. There is no reason the LATE estimated here equals the policy-relevant effect there, and aggregating LATEs across instruments and sites need not converge to anything interpretable.

4 Where the Two Sides Actually Disagree

Stripped down, the disagreement is narrower than the rhetoric suggests. Both camps accept that (i) local estimates are credibly identified and (ii) effects are heterogeneous across contexts. The fault line is about the default. Optimists treat a credible estimate as the prior for a new setting, to be updated as local information arrives; sceptics treat the burden as falling on anyone who would import an estimate, demanding an explicit, defended model of why it should transport. The optimist fears paralysis if nothing generalises, evidence-based policy is impossible. The sceptic fears false confidence a precisely estimated irrelevance dressed up as a global truth.

A second, deeper divide concerns the unit of knowledge. To the optimist, science accumulates through replication: enough trials of a mechanism eventually map its dependence on context. To the sceptic, the heterogeneity is so structured by unobserved institutional and equilibrium factors that no feasible number of trials pins it down, and the honest output is a context-specific estimate plus wide uncertainty about transport.

5 What Evidence Would Help Resolve It

The debate is, encouragingly, an empirical one, and several research programmes can adjudicate it.

• Coordinated multi-site replications. Pre-registered, harmonised trials of the same intervention across many countries (as in the multi-country microcredit and graduation-programme studies) directly measure how much effects vary and which moderators explain the variation. The more variation is explained by observable context, the more estimates can be made to travel.
• At-scale evaluations paired with pilots. Routinely evaluating the government-implemented, full-scale version alongside the NGO pilot [Bold et al., 2018] quantifies the "voltage drop" from efficacy to effectiveness and identifies its sources.
• Structural models disciplined by experiments. Embedding experimental variation in models with explicit general-equilibrium and political-economy channels lets researchers test whether out-of-sample predictions hold, turning external validity into a falsifiable claim.
• Meta-analytic prediction tests. Following Vivalt [2020], one can hold out a study and ask how well the rest of the literature predicts it. Improving that out-of-sample accuracy is a concrete, measurable goal.

The reasonable synthesis is that neither blanket extrapolation nor blanket scepticism is defensible. A credibly identified estimate is a genuine piece of knowledge, but it is knowledge about a context; carrying it elsewhere is a modelling act that must be argued and, wherever possible, tested. The credibility revolution solved the problem of internal validity with remarkable success. Its unfinished business the central methodological frontier of development economics is to make external validity equally rigorous, so that "it worked there" becomes the beginning of an analysis rather than the end of one.

References

1. Banerjee, A. V., and Duflo, E. (2011). Poor Economics: A Radical Rethinking of the Way to Fight Global Poverty. Public Affairs.
2. Bold, T., Kimenyi, M., Mwabu, G., Ng'ang'a, A., and Sandefur, J. (2018). Experimental evidence on scaling up education reforms in Kenya. Journal of Public Economics, 168, 1-20.
3. Deaton, A. (2010). Instruments, randomization, and learning about development. Journal of Economic Literature, 48(2), 424-455.
4. Muralidharan, K., and Niehaus, P. (2017). Experimentation at scale. Journal of Economic Perspectives, 31(4), 103-124.
5. Pritchett, L., and Sandefur, J. (2015). Learning from experiments when context matters. American Economic Review: Papers & Proceedings, 105(5), 471-475.
6. Vivalt, E. (2020). How much can we generalize from impact evaluations? Journal of the European Economic Association, 18(6), 3045-3089.