1 What Problem Does This Tool Solve?

The parallel trends assumption is the identifying assumption of difference-in-differences (DiD). It states that in the absence of treatment, the average outcome in the treated group would have evolved on the same time path as in the control group. This assumption is untestable using post-treatment data alone. Researchers typically test for pre-trends by checking whether the treated and control groups were parallel before treatment began, but Roth [2022] showed that pre-trend tests are often severely underpowered and can give false reassurance.

Rambachan and Roth [2023] proposed a sensitivity analysis approach: rather than assuming parallel trends holds exactly, allow for deviations of bounded size and trace out how treatment effect estimates change as the bound varies. The HonestDiD R package implements this approach. It takes a standard DiD event-study as input and returns sensitivity confidence intervals for each post-treatment period, along with a breakdown value the smallest deviation from parallel trends at which the treatment effect is no longer significantly different from zero.

2 Installation and Setup

The HonestDiD package is available from GitHub:

# Install devtools if needed

install.packages("devtools")

# Install HonestDiD from GitHub

devtools::install_github("asheshrambachan/HonestDiD")

library(HonestDiD)

library(fixest) # for the base event-study regression

library(dplyr)

library(ggplot2)

The package requires an event-study regression output (coefficient vector and covariance matrix) as input. It is agnostic about how the regression was run; fixest is recommended for its speed and staggered DiD support.

3 The Sensitivity Framework

Rambachan and Roth [2023] parameterise deviations from parallel trends via two classes of restrictions on the vector of pre-treatment violations δ_pre = (δ_-K, ..., δ_-1 and post-treatment violations δ_post = (δ₁, ..., δ_L).

• Smoothness restriction (Δ^SD). Deviations from parallel trends cannot change too quickly. Formally, |δ_t+1 - δ_t| ≤ M for all t, where M ≥ 0 is a tuning parameter. When M = 0, parallel trends holds exactly. As M grows, larger and more rapidly changing violations are permitted.

• Relative magnitude restriction (Δ^RM). Post-treatment violations are no larger (in magnitude) than a multiple of the largest pre-treatment violation: |δ_t| ≤ M̄ · max_s |δ_s^pre| for t > 0. This formalises the intuition that pre-trend tests are informative about the likely magnitude of post-treatment bias.

For each value of the restriction parameter (M or M̄), the package computes the identified set for the treatment effect and a confidence interval that is valid uniformly over all violations within the specified class. When the confidence interval excludes zero for all values of the parameter up to some threshold, that threshold is the breakdown value: the treatment effect estimate is robust to violations up to that size.

4 A Worked Example

We illustrate using a simulated DiD event-study. In practice, the researcher would replace the simulated data with their own application.

library(HonestDiD)

library(fixest)

library(dplyr)

# --- Step 1: Run the base event-study regression

# Assume we have a long panel: variables y (outcome), id (unit),

# t (time period), treat (=1 if ever treated),

# and rel_t (event time relative to treatment, NA for never-treated)

# feols with Sun-Abraham-style interactions

# (or any standard TWFE event-study)

es_reg <- feols(y ~ i(rel_t, treat, ref = -1) | id + t,

               data = mydata,

               cluster = ~id)

# Extract coefficients and covariance matrix

beta <- coef(es_reg)

sigma <- vcov(es_reg)

# --- Step 2: Set up HonestDiD inputs

# Select pre-treatment and post-treatment coefficients

# Suppose rel_t runs from -4 to +3 (ref = -1, so -4, -3, -2 are pre; 0, 1, 2, 3 are post)

# HonestDiD needs: betahat (pre + post coefficients), sigma (their covariance),

# numPrePeriods, numPostPeriods

numPrePeriods <- 3   # periods -4, -3, -2

numPostPeriods <- 4  # periods 0, 1, 2, 3

# Positions of pre and post coefficients in the full vector

pre_idx <- 1:numPrePeriods

post_idx <- (numPrePeriods + 1):(numPrePeriods + numPostPeriods)

betahat <- beta[c(pre_idx, post_idx)]

sigma <- sigma[c(pre_idx, post_idx), c(pre_idx, post_idx)]

# --- Step 3: Sensitivity analysis under Delta RM

# Grid of relative magnitude parameters M_bar from 0 (exact PT) to 2

Mbar_grid <- seq(0, 2, by = 0.25)

results_rm <- createSensitivityResults_relativeMagnitudes(

 betahat = betahat,

 sigma = sigma,

 numPrePeriods = numPrePeriods,

 numPostPeriods = numPostPeriods,

 Mbar = Mbar_grid,

 alpha = 0.05

)

# Print sensitivity table

print(results_rm)

# --- Step 4: Sensitivity analysis under Delta SD

# Grid of smoothness parameters M from 0 to 0.05

M_grid <- seq(0, 0.05, by = 0.005)

results_sd <- createSensitivityResults(

 betahat = betahat,

 sigma = sigma,

 numPrePeriods = numPrePeriods,

 numPostPeriods = numPostPeriods,

 Mvec = M_grid,

 alpha = 0.05

)

# --- Step 5: Plot the sensitivity results

createSensitivityPlot_relativeMagnitudes(

 robustResults = results_rm,

 originalResults = data.frame(

   lb = betahat[post_idx] - 1.96 * sqrt(diag(sigma)[post_idx]),

   ub = betahat[post_idx] + 1.96 * sqrt(diag(sigma)[post_idx])

 ),

 rescaleFactor = 1

)

The output is a plot with the standard event-study confidence interval at M = 0 (exact parallel trends) and widening intervals as M increases. The breakdown value the smallest M at which zero enters the confidence interval is reported automatically.

5 Interpreting the Results

• Breakdown value as a robustness metric. If the breakdown value is 0.10 under Δ^RM the treatment effect is significant as long as post-treatment violations are no larger than 10% of the maximum pre-treatment deviation. Whether 0.10 is "large" or "small" depends on context: for a well-designed DiD with very small pre-trends, even M̄ = 0.5 might be considered very generous.

• Comparison to naive pre-trend testing. The conventional approach fails to pass a pre-trend test at p < 0.05 only if pre-trends are large enough to be statistically detectable. The Rambachan and Roth [2023] approach reports what the researcher is implicitly assuming when they do or do not reject: it makes the assumption explicit and continuous rather than binary.

6 Key Options and Pitfalls

• The reference period matters. HonestDiD requires that the event-study coefficient for period -1 (the period immediately before treatment) is omitted (normalised to zero). If a different period is used as the reference, the pre-treatment coefficient vector must be adjusted accordingly.

• Choice of restriction class. Δ^SD (smoothness) is natural when secular trends are the main concern; Δ^RM (relative magnitude) is natural when the researcher believes the pre-trend plot is informative about post-treatment violations. Report both; they should tell a consistent story.

• Staggered adoption. The package handles a single treatment cohort event-study directly. For staggered adoption, run separate event-studies by cohort using Callaway and Sant'Anna [2021] or Sun and Abraham [2021] and apply HonestDiD to each. The HonestDiD package includes a honest_did() wrapper for did package output.

7 Conclusion

HonestDiD operationalises the Rambachan and Roth [2023] sensitivity analysis in a straightforward R workflow. Any standard DiD event-study can be passed to the package; the output is a breakdown value and widened confidence intervals that are valid under bounded violations of parallel trends. Reporting this sensitivity analysis has become a best practice in applied DiD research, providing transparency about the assumption-dependence of findings and enabling readers to calibrate their own degree of scepticism about the parallel trends assumption.

References

1. Callaway, B. and Sant'Anna, P. H. C. (2021). Difference-in-differences with multiple time periods. Journal of Econometrics, 225(2):200-230.
2. Rambachan, A. and Roth, J. (2023). A more credible approach to parallel trends. Review of Economic Studies, 90(5):2555-2591.
3. Roth, J. (2022). Pre-test with caution: event-study estimates after testing for parallel trends. American Economic Review: Insights, 4(3):305-322.
4. Sun, L. and Abraham, S. (2021). Estimating dynamic treatment effects in event studies with heterogeneous treatment effects. Journal of Econometrics, 225(2):175-199.[cite: 14]

‍