1 The Gap Between Frequentist and Bayesian Inference in DML

Double/debiased machine learning (DML) is now a standard method for estimating causal parameters in high-dimensional settings [Chernozhukov et al., 2018]. The basic insight is elegant: by using machine learning to partial out the influence of high-dimensional controls from both the treatment and the outcome, and then regressing the residualised outcome on the residualised treatment, one can obtain √n consistent estimates of causal parameters even when the dimension of confounders grows with n.  

The frequentist DML estimator yields point estimates and confidence intervals based on the asymptotic normal distribution of the Neyman-orthogonal score function. But many researchers have natural reasons to prefer a Bayesian approach. Prior information may be available—for instance, about the plausible range of a labour market elasticity or the direction of an environmental treatment effect. Posterior distributions provide a natural way to summarise uncertainty and to propagate that uncertainty into downstream decision analysis. And Bayesian credible intervals have an appealing direct probabilistic interpretation that frequentist confidence intervals lack.  

The challenge is that off-the-shelf Bayesian estimation of the partially linear model runs into a fundamental problem: regularisation-induced confounding. If a Bayesian model shrinks the coefficients on the controls in the outcome equation, the estimated coefficient on the treatment absorbs part of the confounding that was supposed to be removed. The posterior for the causal parameter is then biased, in a way that does not vanish as n grows.  

DiTraglia and Liu [2025] propose a solution: Bayesian DML (BDML), a fully Bayesian approach that achieves valid posterior inference for the causal parameter in the partially linear model while using machine learning to estimate the high-dimensional nuisance functions.  

2 The Partially Linear Model

The model of interest is the partially linear regression (PLR):  

where Y is the outcome, D is the treatment of interest, X is a high-dimensional vector of confounders, θ is the scalar causal parameter, g and m are unknown nuisance functions, and (ε, ν) are mean-zero errors.  

Under standard regularity conditions (including that 𝔼[ε|D,X] = 0 and that the supports of m and ν have sufficient overlap), θ is identified as the coefficient in the projection of the residualised outcome Ỹ = Y − g(X) on the residualised treatment D̃ = D − m(X):  

The frequentist DML estimator replaces the population expectations with sample analogues and uses cross-fitting to avoid overfitting bias in the nuisance estimators.  

3 The Regularisation-Induced Confounding Problem

To see why naive Bayesian estimation fails, consider a simplified version with p covariates and a Bayesian linear regression:  

The shrinkage prior β ~ 𝒩(0, τ²Iₚ) is the Bayesian analogue of ridge regularisation. When D is correlated with X, the shrinkage on β leaves some confounding variation in D unexplained by X in the posterior. This variation gets absorbed into the posterior for θ, biasing it away from the true causal parameter.  

The bias does not diminish as n → ∞ as long as the dimension p grows proportionally with n. In typical high-dimensional settings where machine learning is used precisely because p is large, this is a fundamental problem.  

4 The BDML Solution

DiTraglia and Liu [2025] propose a reformulation that sidesteps regularisation-induced confounding. The key insight is to model the causal parameter θ through the reduced-form covariance structure rather than through a direct regression coefficient.  

Consider the joint distribution of (Y, D) conditional on X. The residuals (Ỹ, D̃) from the projections on X follow a bivariate normal distribution (under linearity):  

The causal parameter can be recovered as θ = σ_YD / σ_DD—a simple transformation of the reduced-form covariance matrix. By modelling the covariance matrix Σ directly and placing a prior on Σ (e.g., an inverse-Wishart), one can recover a posterior for θ = σ_YD / σ_DD without the regularisation-induced confounding problem.  

The BDML procedure:  

1. Use machine learning (e.g., random forests, gradient boosting) to estimate ĝ(X) and m̂(X) via cross-fitting.  
2. Compute residuals ε̂ᵢ = Yᵢ − ĝ(Xᵢ) and ν̂ᵢ = Dᵢ − m̂(Xᵢ).  
3. Place a prior on the covariance matrix Σ of (ε, ν) and compute the posterior for Σ from the residual pairs.  
4. Transform the posterior for Σ into a posterior for θ = σ_εν / σ_νν.  

5 Theoretical Guarantees: The Bernstein-von Mises Theorem

A key theoretical result in DiTraglia and Liu [2025] is a Bernstein-von Mises (BvM) theorem for BDML. The BvM theorem is the Bayesian analogue of asymptotic normality: it states that, under regularity conditions, the posterior distribution for θ concentrates around the true parameter at rate n⁻¹/² and its shape converges to a normal distribution with variance equal to the semiparametric efficiency bound.  

The practical implication is that Bayesian credible intervals from BDML are asymptotically valid as frequentist confidence intervals—the two interpretations agree to first order. This makes BDML attractive for researchers who want the computational and interpretive benefits of Bayesian inference without sacrificing the frequentist validity guarantees that are standard in applied economics.  

The BvM result requires that the nuisance estimators converge at rates faster than n⁻¹/⁴, which is the same condition required for frequentist DML. Under this condition, the choice between Bayesian and frequentist DML becomes a matter of preference: the former provides a posterior distribution, the latter provides a point estimate and asymptotic confidence interval, but both are asymptotically equivalent in terms of inference on θ.  

6 Prior Information and Finite-Sample Performance

In finite samples, BDML and frequentist DML differ. The Bayesian approach incorporates prior information about θ, which can substantially reduce posterior variance when the prior is informative and approximately correct. In a simulation study, DiTraglia and Liu [2025] show that BDML outperforms frequentist DML in terms of mean squared error when the sample size is small relative to the number of confounders, and the performance advantage decreases as n grows—consistent with the asymptotic equivalence result.  

This finite-sample advantage is practically important. In many economic applications, the dataset has 500-5,000 observations and several hundred potential confounders. In this regime, informative priors on the order of magnitude of the causal effect can provide meaningful regularisation without the confounding bias that afflicts naive Bayesian regression.  

7 Comparison to Frequentist DML

8 Identifying Assumptions

BDML shares the same identifying assumptions as frequentist DML:  

1. Unconfoundedness: 𝔼[ε|D,X] = 0, meaning no unobserved confounders.  
2. Overlap: Var(D|X) > 0 for all X, so there is residual variation in treatment after conditioning on confounders.  
3. Nuisance rate condition: The machine learning estimators of g and m converge at sufficiently fast rates.  

The Bayesian framework does not relax these assumptions; it provides a different inferential approach under the same identification conditions.  

9 Extensions and Active Research

DiTraglia and Liu [2025] also discuss extensions to the partially linear IV model, where the treatment D is instrumented by Z and unconfoundedness is replaced by an exclusion restriction. A separate strand of research has developed scalable Bayesian DML for very large datasets using variational Bayes approximations [Ray and van der Vaart, 2025].  

The connection between BDML and Bayesian additive regression trees (BART) is also being explored: BART can serve as the machine learning estimator for the nuisance functions, providing a fully Bayesian pipeline from raw data to posterior inference on θ.  

10 Conclusion

Bayesian DML fills an important gap in the causal inference toolkit. For researchers with genuine prior information about their estimand, or who need to propagate inferential uncertainty into downstream decision analysis, the Bayesian approach is natural. The BvM theorem ensures that this preference does not come at the cost of frequentist validity. As implementations become more computationally accessible, BDML is likely to find broad adoption alongside its frequentist counterpart.  

References

1. Chernozhukov, V., Chetverikov, D., Demirer, M., Duflo, E., Hansen, C., Newey, W., and Robins, J. (2018). Double/debiased machine learning for treatment and structural parameters. Econometrics Journal, 21(1):C1-C68.  
2. DiTraglia, F. J. and Liu, L. (2025). Bayesian double machine learning for causal inference. arXiv preprint, 2508.12688.  
3. Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., and Rubin, D. B. (2014). Bayesian Data Analysis. Chapman and Hall/CRC, 3rd edition.  
4. Ray, K. and van der Vaart, A. (2025). Semiparametric Bernstein-von Mises theorems for Gaussian process priors. Probability Theory and Related Fields, 185(3):1-42.  
5. Semenova, V. and Chernozhukov, V. (2021). Debiased machine learning of conditional average treatment effects and other causal functions. Econometrics Journal, 24(2):264-289.  
6. Wager, S. and Athey, S. (2018). Estimation and inference of heterogeneous treatment effects using random forests. Journal of the American Statistical Association, 113(523):1228-1242[cite: 16].