Introduction

For most of the history of econometrics, data scarcity was the binding constraint. Sample sizes of a few hundred or a few thousand observations forced researchers to be parsimonious, to rely on strong parametric assumptions, and to accept wide confidence intervals. The credibility revolution was partly a response to this scarcity: with limited data, researchers learned that transparent identification strategies and simple, robust estimators outperformed elaborate structural models estimated with weak data.

That world is changing. Administrative tax records, electronic health systems, social media platforms, satellite imagery, and retail scanner data now make datasets of millions or even hundreds of millions of observations routinely accessible [Chetty et al., 2014]. The US Census Bureau's Longitudinal Employer-Household Dynamics file links every formal employment spell in the United States to employer characteristics. Credit card transaction records capture near-universal consumer behaviour in real time.

Does the shift from data-scarce to data-abundant environments simply mean sharper estimates? Often yes—but the change is more fundamental than that. Large n breaks the econometrician's world in several ways: standard errors shrink so small that any effect, however substantively trivial, becomes statistically significant; heterogeneity that was previously undetectable becomes estimable; entirely new identification strategies become possible; and, paradoxically, new forms of model misspecification become a binding constraint when they were previously dwarfed by sampling noise.

This article maps the opportunities and the pitfalls of causal inference with abundant data. We draw on recent theoretical work by Abadie et al. [2020] and the framework developed in Abadie et5 al. [2025].

2 What Changes When n Is Large

2.1 The Statistical Significance Filter Dissolves

The classical p-value framework was designed for a world in which Type II error—failing to detect a true effect—was the dominant concern. In large administrative datasets, Type II error vanishes almost entirely. With a million observations, a difference-in-means estimator with a standard deviation of outcome σ_Y = 2 and a treatment effect of τ = 0.001 yields a t-statistic of roughly:

which is not significant. But with n = 10,000,000, the same effect yields t ≈ 3.5, highly significant by any conventional threshold.

This is not a problem with the data; it is a problem with how researchers use statistical significance as a proxy for "important." With large datasets, effect size and substantive significance must be reported alongside p-values. Confidence intervals remain informative, but the conversation shifts from "is the effect non-zero?" to "is the effect large enough to matter?" [Imbens, 2021].

2.2 Design-Based Inference Becomes More Compelling

Abadie et al. [2020] drew a sharp distinction between sampling-based and design-based uncertainty. In the traditional framework, we treat the sample as a random draw from a superpopulation, and uncertainty reflects sampling noise. In the design-based framework, we treat the population as fixed and uncertainty reflects the randomness in treatment assignment.

With large administrative datasets, the population is often nearly fully observed—there is no sampling step. The US income tax records cover essentially every tax filer. In this setting, sampling uncertainty is near zero, and uncertainty about causal effects comes entirely from the randomness in the assignment mechanism. Design-based standard errors, which account for this correctly, can be much smaller (or larger) than sampling-based standard errors, depending on the clustering structure of the assignment.

This has immediate implications for inference after DiD, RD, and IV estimation in large administrative panels. Abadie et al. [2023] showed that whether standard errors should be clustered, and at what level, depends on the assignment mechanism, not on the correlation structure of the residuals. In a large panel where treatment varies at the county-year level, clustering at the county level is justified by the design, not by worries about autocorrelated errors.

2.3 New Identification Strategies Become Feasible

Large datasets open identification strategies that are impractical at small scales. Within-person designs compare outcomes for the same individual at different points in time, eliminating all fixed individual-level confounders. With millions of workers, researchers can ask: when the same worker switches from a high-wage to a low-wage firm, what fraction of the wage drop is attributable to the firm rather than to worker-level selection? Card et al. [2018] used a two-way fixed effects decomposition on Austrian Social Security data to decompose wages into worker and firm components.

Peer effect identification becomes tractable at large scale. With millions of elementary school students, quasi-random variation in classroom peer composition can be exploited even after conditioning on a rich set of school-by-year fixed effects [Chetty et al., 2011]. With small samples, such conditional variation is insufficient for precise estimation.

Regression discontinuity designs gain statistical power when large administrative datasets provide dense mass near the cutoff. The usual bandwidth-bias tradeoff is resolved partly by the abundance of observations: optimal bandwidth selectors such as Calonico et al. [2014] can be applied with very narrow bandwidths, limiting confounding by global trends without sacrificing precision.

3 The Pitfalls of Abundance

3.1 Heterogeneity Is Everywhere

The standard assumption of a constant treatment effect τ was always wrong, but with small samples, heterogeneity was indistinguishable from noise. With large datasets, heterogeneity is everywhere and unmistakeable.

This creates an identification challenge that is underappreciated. The TWFE estimator of DiD, for instance, is a weighted average of all pairwise 2×2 DiD comparisons. When treatment effects are heterogeneous across units and over time, some of these weights are negative [Goodman-Bacon, 2021]. With a small sample, this bias may be small relative to sampling noise. With a large sample, the bias is the dominant source of error.

Similarly, IV estimates identify the Local Average Treatment Effect (LATE) for compliers. In a large dataset with many different instruments—different judges, different bureaucrats, different geographic cutoffs—each estimates a LATE for a different complier population [Imbens, 2010]. Aggregating these estimates into a single number may not recover any substantively interesting parameter.

3.2 Model Misspecification Dominates

In small samples, the bias from a misspecified functional form is usually swamped by sampling variance. In large samples, bias is the dominant source of error, and it does not decrease as n grows.

This is Lehmann's textbook lesson, but it is violated routinely in large-dataset empirical work. A linear probability model for a binary outcome misspecifies the conditional expectation function; in a small sample, the resulting bias in estimated marginal effects is small. In a dataset with 100 million observations, the same model generates a very precise but potentially misleading estimate.

The practical implication is that flexible, nonparametric methods become more important as n grows. Machine learning estimators—gradient boosted trees, random forests, neural networks—are valuable not because they are causal but because they reduce misspecification bias in the nuisance components of a causal estimator. The DML framework [Chernozhukov et al., 2018] was partly motivated by this logic: use machine learning to estimate 𝔼[Y|X] and 𝔼[D|X], then partial out X before estimating the causal parameter of interest. In large datasets, the machine learning step is both more important (because misspecification bias is larger relative to variance) and more reliable (because the estimators have enough data to converge).

3.3 Multiple Comparisons at Scale

Large administrative datasets invite large-scale searches for treatment effect heterogeneity. With data on millions of individuals, researchers can estimate subgroup-specific effects for hundreds of demographic and geographic cells. Some of these will appear significant by chance.

The multiple testing problem—which was already a concern in small-scale randomised experiments [List et al., 2019]—becomes acute at administrative-data scale. Adjustments such as the Benjamini-Hochberg false discovery rate procedure are essential when reporting subgroup effects from large-scale analyses [Benjamini and Hochberg, 1995].

3.4 Privacy, Confidentiality, and Measurement Error

Large administrative datasets are not clean. Records are linked probabilistically, introducing record linkage error. Tax records misclassify self-employment income. Medical records use billing codes designed for reimbursement rather than research. These measurement errors are often uncorrelated with treatment status, causing attenuation bias in estimated treatment effects. With large n, the confidence intervals shrink around a biased point estimate.

Moreover, access to large administrative datasets typically requires navigating Federal Statistical Research Data Centres, data use agreements, and confidentiality protocols. The institutional constraints on using these data can create selection effects: the research questions that get asked are those that can be addressed within the institutional framework, not necessarily the most important questions.

4 A Framework for Thinking Clearly

Abadie et al. [2025] propose viewing causal inference in data-rich environments through the lens of three distinct uncertainty sources: (1) sampling uncertainty from drawing a sample from a superpopulation; (2) assignment uncertainty from the randomness in who receives treatment; and (3) estimation uncertainty from using finite-dimensional summaries of a potentially infinite-dimensional parameter. In large datasets, (1) vanishes, (2) becomes the dominant source of uncertainty, and (3) depends on the quality of the nuisance estimators used.

This decomposition has a practical implication: in large administrative datasets, the most important research design choices concern the assignment mechanism—natural experiment quality, exclusion restriction plausibility, overlap condition—not the sample size. A well-designed study with a million observations is worth less than a poorly-designed study with ten million observations only if the natural experiment is genuinely better in the latter case. Scale does not substitute for identification.

5 Conclusion

The move from sparse to abundant data is one of the most consequential shifts in empirical social science in the last two decades. It has enabled research programmes—on intergenerational mobility, employer wage premia, health effects of pollution—that were simply impossible with survey data. But it has also created new pathways to misleading inference: statistical significance that means nothing, heterogeneity that is measured but not understood, and bias from model misspecification that survives indefinitely into larger samples. Researchers who exploit administrative data well are not those who trust its scale, but those who remain sceptical of their own identifying assumptions.

References

1. Abadie, A., Athey, S., Imbens, G. W., and Wooldridge, J. M. (2020). Sampling-based versus design-based uncertainty in regression analysis. Econometrica, 88(1):265-296.
2. Abadie, A., Athey, S., Imbens, G. W., and Wooldridge, J. M. (2023). When should you adjust standard errors for clustering? Quarterly Journal of Economics, 138(1):1-35.
3. Abadie, A., Agarwal, A., and Shah, D. (2025). Causal inference in a data-rich environment. arXiv preprint, 2504.01702.
4. Benjamini, Y. and Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B, 57(1):289-300.
5. Calonico, S., Cattaneo, M. D., and Titiunik, R. (2014). Robust nonparametric confidence intervals for regression-discontinuity designs. Econometrica, 82(6):2295-2326.
6. Card, D., Heining, J., and Kline, P. (2018). Workplace heterogeneity and the rise of West German wage inequality. Quarterly Journal of Economics, 128(3):967-1015.
7. Chetty, R., Friedman, J. N., Hilger, N., Saez, E., Schanzenbach, D. W., and Yagan, D. (2011). How does your kindergarten classroom affect your earnings? Evidence from Project STAR. Quarterly Journal of Economics, 126(4):1593-1660.
8. Chetty, R., Hendren, N., Kline, P., Saez, E., and Turner, N. (2014). Is the United States still a land of opportunity? Recent trends in intergenerational mobility. American Economic Review Papers and Proceedings, 104(5):141-147.
9. Chernozhukov, V., Chetverikov, D., Demirer, M., Duflo, E., Hansen, C., Newey, W., and Robins, J. (2018). Double/debiased machine learning for treatment and structural parameters. Econometrics Journal, 21(1):C1-C68.
10. Goodman-Bacon, A. (2021). Difference-in-differences with variation in treatment timing. Journal of Econometrics, 225(2):254-277.
11. Imbens, G. W. (2010). Better LATE than nothing: Some comments on Deaton (2009) and Heckman and Urzua (2009). Journal of Economic Literature, 48(2):399-423.
12. Imbens, G. W. (2021). Statistical significance, p-values, and the reporting of uncertainty. Journal of Economic Perspectives, 35(3):157-174.
13. List, J. A., Shaikh, A. M., and Xu, Y. (2019). Multiple hypothesis testing in experimental economics. Experimental Economics, 22(4):773-793.