1 Introduction

Difference-in-differences (DiD) identification rests on the parallel trends assumption: in the absence of treatment, treated and control units would have followed the same outcome trajectory.  This assumption is untestable we never observe the counterfactual. What researchers typically do instead is check for "pre-trends": test whether treated and control units had parallel trends in the pre-treatment period.

But pre-trend tests have a serious limitation [Roth, 2022]: passing a pre-trend test does not validate parallel trends post-treatment. Pre-trend tests have low power in many designs, meaning they frequently fail to detect violations. A researcher whose study "passes" the pre-trend test may still have substantial parallel trends violations that bias their estimates.

Rambachan and Roth [2023] propose a principled alternative: instead of testing parallel trends and either accepting or rejecting the DiD, conduct sensitivity analysis that asks, "How much would parallel trends need to be violated post-treatment to overturn my conclusions?" This is the Honest DiD framework.

2 The Problem with Pre-Trend Tests

2.1 Setup

Consider a DiD setting with a single treatment cohort. Let $\delta_{t}$ denote the violation of parallel trends at time t (the difference in trends between treated and control units, purged of treatment effects). Under exact parallel trends, $\delta_{t}=0$ for all t.

In an event-study regression, the estimated coefficients on pre-treatment relative-time dummies, $\hat{\beta}_{-k}$ for $k=1,2,...,$ are informative about pre-treatment trend differences. If parallel trends held pre-treatment, these should be zero.

2.2 The Testing Problem

Roth [2022] shows that the standard approach test for pre-trends, proceed with DiD if the test passes introduces conditional bias. The reason: if the null of zero pre-trends is rejected, researchers often conclude their design is invalid and do not publish. If it is not rejected, they proceed but non-rejection does not guarantee small $\delta_{t}$, especially with modest sample sizes. The resulting estimates are biased because researchers condition on passing a noisy test.

Moreover, even if pre-treatment trends are exactly zero, there is no guarantee that post-treatment trends would also be zero. A policy that triggers treatment may itself change the trajectory of treated units' counterfactual outcomes.

3 The Honest DiD Framework

3.1 Sensitivity Approach

Rambachan and Roth [2023] propose to replace the binary test with a sensitivity region: a set M of plausible violations of parallel trends, and a corresponding set of treatment effect estimates that are consistent with the data and any $\delta\in\mathcal{M}$

The approach proceeds in three steps:

1. Specify M: what violations of parallel trends are considered plausible?
2. Compute the "breakdown" point: the smallest violation in M that would change the sign or significance of the estimated effect.
3. Report confidence sets: for each $\delta\in\mathcal{M}.$ what treatment effect values are consistent with the data?

3.2 Restrictions on the Violation Path

The key modelling choice is how to restrict 8. Rambachan and Roth [2023] propose two families:

Smoothness restrictions (Delta_SD): The post-treatment violation is not much larger than the pre-treatment violations. Formally:

MSD (M) = {δ: - 81-11 M for all t}  (1)

The parameter $\overline{M}$ controls how much the trend violation can change period to period. If pre-treatment violations are small, then the researcher restricts post-treatment violations to be similarly small. When $\overline{M}=0$ violations must be a linear trend (extrapolated exactly from pre-treatment).

Relative magnitude restrictions (Delta RM): The post-treatment violation is no larger than $\overline{M}$ times the largest pre-treatment violation. Formally:

MRM (M) = = {8: δ: δε <M max Μ.max for all all t>0} t > 0 $<0  (2)

Here $s<0$ indexes pre-treatment periods in event-study notation (with $s=0$ denoting the period immediately before treatment), so $max_{s<0}|\delta_{s}|$ is the largest observed pre-treatment trend violation. At $\overline{M}=1$, post-treatment violations are at most as large as observed pre-treatment violations. At $\overline{M}=2$, they can be twice as large.

3.3 Identified Set and Confidence Intervals

For a given M, the identified set for the treatment effect 7 in post-treatment period t is:

$\mathcal{I}(\overline{M})=\{\tau:\exists\delta\in\mathcal{M}(\overline{M})$ s.t. $(\hat{\beta},\delta)$ is consistent with the data data}  (3)

Rambachan and Roth [2023] derive confidence sets that cover $\mathcal{I}(\overline{M})$ uniformly over $\delta\in$ $\mathcal{M}(\overline{M})$. These are computed using the HonestDiD package in R.

4 Interpreting the Results

The main output of a Honest DiD analysis is a plot of confidence intervals as a function of $\overline{M}$

• At $M=0$: the confidence interval under exact parallel trends (the standard DiD CI).

• As $\overline{M}$ increases: the confidence interval widens, reflecting greater uncertainty about the treatment effect given larger permitted violations.

• The breakdown value $\overline{M}^{*}$ is the smallest M at which the confidence set includes zero (i.e., the effect is no longer robustly positive or negative).

If the breakdown value is large say, violations would need to be 3-4 times larger than the observed pre-trend to overturn the finding the result is robust. If the breakdown occurs at $\overline{M}=0.5$ (violations half the size of observed pre-trends), the result is fragile.

5 The HonestDiD R Package

Rambachan and Roth [2023] provide the HonestDiD package for R, which integrates with standard event-study estimates.

library (HonestDiD)
# Suppose betahat is the vector of event-study coefficients
# and sigma is their covariance matrix (from e.g. feols())
# Delta_SD sensitivity analysis
# Note: check ? createSensitivityResults for current function
signatures
# and available method options before use the API may evolve across
versions.
results_sd <- createSensitivityResults(
betahat= betahat,
sigma= sigma,
numPrePeriods=4,
numPostPeriods =4,
Mbarvec= seq(0, 2, by =0.5) # grid of M-bar values
# see ? createSensitivityResults for method options

)

‍
# Plot sensitivity
createSensitivity Plot(results_sd)


6 Relationship to Other Work

Roth [2022] motivates the honest DiD approach by showing that pre-trend testing is a poor substitute for sensitivity analysis. Manski [2003] provides the broader framework of partial identification that Honest DiD builds on. The approach is related to Rosenbaum bounds in observational studies [Rosenbaum, 2002], which ask how large unmeasured confounding would need to be to overturn a result.

7 Practical Guidance

When should researchers use Honest DiD?

1. Always report the breakdown value alongside the point estimate in DiD studies.
2. If pre-trends are noisy or the pre-treatment window is short, use $\Delta^{RM}$ restrictions (relative magnitude). If economic theory constrains how trends can evolve, $\Delta^{SD}$ may be more appropriate.
3. A result is credible if the breakdown value exceeds the largest plausible violation, which should be justified by institutional knowledge or the economic setting.

8 Conclusion

The Honest DiD framework of Rambachan and Roth [2023] reframes DiD inference from a binary test of parallel trends to a continuous sensitivity analysis. By asking how much the parallel trends assumption must be violated to overturn the conclusions, it gives researchers and readers an honest assessment of the robustness of DiD estimates. The HonestDiD package makes this analysis straightforward to implement, and the approach is rapidly becoming a standard complement to pre-trend tests in applied work.

References

1. Rambachan, A. and Roth, J. (2023). A more credible approach to parallel trends. Review of Economic Studies, 90(5):2555-2591.
2. Roth, J. (2022). Pre-test with caution: Event-study estimates after testing for parallel trends. American Economic Review: Insights, 4(3):305-322.
3. Manski, C.F. (2003). Partial Identification of Probability Distributions. Springer, New York.  Rosenbaum, P.R. (2002). Observational Studies, 2nd ed. Springer, New York.
4. Callaway, B. and Sant'Anna, P.H.C. (2021). Difference-in-differences with multiple time periods. Journal of Econometrics, 225(2):200-230.