1 Introduction

How do we estimate the causal effect of incarceration on future crime? Of granting bail on court appearances? Of patent approval on firm innovation? Each of these questions confronts a fundamental endogeneity problem: the treatment jail, bail, patent is assigned partly on the basis of characteristics that also predict the outcome. Defendants who are detained pre-trial are more likely to be convicted, but also more likely to have committed crimes that warrant detention in the first place.

The leniency design is one of the most elegant solutions economists have devised. The key observation is that in many administrative systems, cases are quasi-randomly assigned to decision-makers judges, examiners, loan officers, bankruptcy trustees who differ in their propensity to grant treatment. These idiosyncratic tendencies, driven by judge-specific preferences or ideologies rather than case characteristics, provide a source of quasi-random variation in treatment that satisfies the conditions for a valid instrumental variable [Angrist et al., 1996].

Over the last two decades, leniency designs have spread far beyond criminal justice. Patent examiners vary in grant rates; bankruptcy judges differ in whether they allow firms to continue operating; immigration judges diverge in asylum grant rates; loan officers vary in approval propensities. A new comprehensive treatment by Goldsmith-Pinkham et al. [2025] provides what amounts to an operator's manual for implementing these designs, clarifying the required assumptions, diagnosing potential violations, and recommending the appropriate estimators. This article synthesises their contribution and the broader leniency design literature.

2 The Basic Setup

Let Di ∈ {0,1} denote whether individual i receives the treatment (e.g., pre-trial detention), Yi denote the outcome (e.g., conviction, employment), and Ji ∈ {1,...,K} denote the judge to whom case i is assigned. Define the leave-one-out leniency of judge j as:

the average treatment rate among all cases assigned to judge j except case i itself. This leave-one-out construction, due to Angrist et al. [1999], removes the mechanical correlation between an individual's own treatment and the leniency measure. The estimated leniency λ̂j(-i) serves as the instrument Zi for treatment Di. In the two-stage least squares (2SLS) framework:

where Xi are case characteristics included as controls. The coefficient β1 recovers the local average treatment effect (LATE) for compliers individuals whose treatment status is switched by the judge's leniency [Angrist et al., 1996].

3 The Identifying Assumptions

Three conditions are required for the leniency instrument to be valid [Goldsmith-Pinkham et al., 2025]:

3.1 Quasi-Random Assignment

Cases must be assigned to judges in a manner that is as-good-as-random, conditional on observed case characteristics. Formally, Ji ⊥ (Yi(0), Yi(1), Di(0), Di(1)) | Xi. This is typically guaranteed by institutional rules: courts randomly assign cases by calendar slot, day of week, or a rotating docket. However, the researcher must verify that defendants cannot sort into particular judges' dockets, and that judges do not strategically request or avoid certain case types.

Testing quasi-random assignment: A standard approach regresses judge assignment on baseline covariates. With a large number of judges, joint F-tests across all covariates for each judge are impractical; instead, one can test whether the leniency measure Zi predicts pre-determined case characteristics after conditioning on any administrative controls (court, year, case type). A clean design should show no predictive power.

3.2 Exclusion Restriction

The judge leniency instrument must affect the outcome only through the treatment not directly. The identifying concern is that "strict" judges may impose harsher sentences conditional on guilt (the incarceration length channel), or write more thorough opinions (the quality channel for patent examiners), so that judge leniency affects Yi through channels other than the binary treatment indicator Di. This assumption is typically justified by arguing that, conditional on treatment receipt, the identity of the judge is not further informative about Yi. In criminal justice applications, researchers sometimes address this by showing that leniency does not predict outcomes for those always treated regardless of leniency.

3.3 First-Stage Monotonicity

The standard LATE interpretation requires that judge leniency affects everyone's treatment probability in the same direction: Di(λ') ≥ Di(λ) for all i when λ' > λ. That is, no defendant would be treated under a lenient judge but not treated under a strict judge.

Goldsmith-Pinkham et al. [2025] distinguish between average and individual monotonicity. Individual monotonicity (no defiers) is stronger and often implausible: some defendants may be treated precisely because a strict judge views their case more seriously. Average monotonicity requiring only that the first stage be positive on average is weaker and typically satisfied. The LATE interpretation changes subtly when individual monotonicity fails.

4 The Many-Instruments Problem

A key econometric complication arises when the number of judges K is large relative to the sample. In this case, the leniency instrument is constructed from a large number of judge fixed effects, creating a "many-instruments" problem analogous to that identified by Staiger and Stock [1997]: 2SLS with many instruments is biased toward OLS in finite samples.

4.1 Jackknife IV (JIVE)

Angrist et al. [1999] proposed the Jackknife Instrumental Variables Estimator (JIVE) to address this bias. JIVE replaces the fitted values from the first stage with leave-one-out predictions essentially instrumenting each observation with the judge effects estimated from all other observations. The leave-one-out leniency measure in equation (1) is precisely the JIVE first-stage instrument.

4.2 Unbiased JIVE (UJIVE)

Goldsmith-Pinkham et al. [2025] recommend the Unbiased Jackknife IV Estimator (UJIVE), which removes additional finite-sample bias present in the original JIVE. The UJIVE leniency instrument is defined as:

where &Dtilde;i' = Di' - Xi''γ̂ are residuals from a regression of treatment on covariates, and nj is the number of cases assigned to judge j. Unlike 2SLS with judge fixed effects, UJIVE remains consistent even as the number of judges grows with the sample size. Goldsmith-Pinkham et al. [2025] show through Monte Carlo experiments that UJIVE outperforms JIVE and 2SLS in settings typical of leniency applications.

!Figure 1: DAG representation of the leniency design. The red dashed arrow from judge to outcome represents a violation of the exclusion restriction. The identifying assumption is that this path is absent.

5 Inference and Standard Errors

A subtle but important point concerns standard errors. Goldsmith-Pinkham et al. [2025] argue that, unlike many IV applications, non-clustered (heteroskedasticity-robust) standard errors are often appropriate in leniency designs. The reasoning is design-based: if judge assignment is truly random within the conditioning set, the residual variance structure does not require clustering at the judge level. Clustering at the judge level will typically produce conservative standard errors. The appropriate choice depends on whether the randomisation is at the individual or judge level and whether one is targeting a design-based or model-based variance.

6 Applications Beyond Criminal Justice

The leniency design has been profitably applied across a remarkably wide range of settings:

Criminal justice. Kling [2006] uses judge assignment to instrument for incarceration length and finds modest positive effects on employment. Aizer and Doyle [2015] find that juvenile incarceration substantially reduces graduation rates (by 13 percentage points) and increases adult crime. Dobbie et al. [2018] use bail judges to show that pre-trial detention causally increases conviction rates and reduces earnings.

Bankruptcy. Bankruptcy judges vary in whether they allow firms to continue operating or liquidate. Judge assignment provides a quasi-random source of variation in whether a firm reorganises versus liquidates, enabling estimation of the causal effect of bankruptcy protection on firm outcomes and worker wages.

Patent examiners. Technology center directors at the USPTO randomly assign patent applications to examiners who vary in grant rates. Researcher have used this variation to estimate the causal effects of patent protection on firm performance, follow-on innovation, and market entry.

Immigration. Immigration judges differ substantially in asylum grant rates. Random assignment of asylum seekers to judges provides variation in refugee status that can be used to estimate the causal effects of asylum on labour market outcomes and integration.

Consumer lending. Loan officers differ in approval propensities. Regulatory leniency designs have been used to study the effects of credit access on consumption, default, and financial health.

7 Diagnostics and Falsification Tests

Goldsmith-Pinkham et al. [2025] recommend a battery of diagnostic tests:

1. Balance test: Regress each pre-determined covariate on the leave-one-out leniency instrument Zi, conditional on court-year fixed effects. None should be statistically significant.
2. First-stage strength: Report the first-stage F-statistic for UJIVE to confirm relevance.
3. Bunching/sorting test: Plot the empirical distribution of cases across judges by observable case quality measures. Deviations from uniformity may indicate gaming.
4. Exclusion plausibility: For binary outcomes, test whether leniency predicts the outcome among those who are always treated (never affect the margin). Any effect would suggest a direct channel.
5. Monotonicity check: Estimate judge-specific first-stage coefficients and check whether any are negative (indicating defiers for that subpopulation).

8 Interpreting the LATE in Leniency Settings

The LATE recovered by a leniency design applies to compliers individuals who receive treatment only when assigned to a more lenient judge. In criminal justice, compliers are typically marginal defendants: those whose cases fall in an uncertain zone where some judges would detain them and others would not. This has important implications for policy interpretation. If policymakers are interested in the effects of changing detention policy for all defendants, the LATE for marginal defendants may not be directly relevant. However, compliers in leniency designs are often exactly the population of interest for incremental policy reforms, since reforms are most likely to affect cases on the margin. The Marginal Treatment Effect (MTE) framework [Heckman and Vytlacil, 2005] provides a richer way to extrapolate from the leniency LATE to treatment effects for other populations, at the cost of stronger functional form assumptions.

9 Conclusion

Leniency designs represent one of the most credible and widely applicable identification strategies in applied economics. By exploiting quasi-random assignment to decision-makers who differ in their treatment propensities, researchers can recover causal effects of consequential treatments incarceration, bail, patent approval, credit that would be impossible to identify through random assignment or other strategies. The key practical lessons from Goldsmith-Pinkham et al. [2025] are: use the leave-one-out leniency instrument to construct the IV; prefer UJIVE over 2SLS with judge fixed effects when judges are numerous; document quasi-random assignment carefully; test monotonicity and the exclusion restriction using the falsification exercises described above; and be thoughtful about standard errors, which may not require clustering at the judge level. As leniency designs continue to migrate into new institutional settings financial regulation, administrative courts, healthcare allocation the practitioner's toolkit assembled by Goldsmith-Pinkham et al. [2025] provides an indispensable reference.

References

1. Aizer, A. and Doyle, J. J. (2015). Juvenile incarceration, human capital, and future crime:Evidence from randomly assigned judges. Quarterly Journal of Economics, 130(2):759803.
2. Angrist, J. D., Imbens, G. W., and Rubin, D. B. (1996). Identification of causal effects usinginstrumental variables. Journal of the American Statistical Association, 91(434):444–455.
3. Angrist, J. D., Imbens, G. W., and Krueger, A. B. (1999). Jackknife instrumental variablesestimation. Journal of Applied Econometrics, 14(1):57–67.
4. Dobbie, W., Goldin, J., and Yang, C. S. (2018). The effects of pre-trial detention on conviction, future crime, and employment: Evidence from randomly assigned judges. AmericanEconomic Review, 108(2):201–240.
5. Goldsmith-Pinkham, P., Hull, P., and Koles´ar, M. (2025). Leniency designs: An operator’smanual. arXiv preprint arXiv:2511.03572.
6. Heckman, J. J. and Vytlacil, E. (2005). Structural equations, treatment effects, and econometric policy evaluation. Econometrica, 73(3):669–738.
7. Kling, J. R. (2006). Incarceration length, employment, and earnings. American EconomicReview, 96(3):863–876.
8. Staiger, D. and Stock, J. H. (1997). Instrumental variables regression with weak instruments.Econometrica, 65(3):557–586.