Difference-in-differences attempts to measure causal effects using changes in outcomes across different groups. One of the key assumptions of differences-in-differences specification are that pre-period trends are similar across these groups. But what happens if these pre-period trends do appear? Can the use of propensity score matching solve the problem. According to a paper by Daw and Hatfield (2018), the answer is typically ‘no’:
When preperiod outcome level is correlated with treatment assignment, an unmatched analysis is unbiased, but matching units on preperiod outcome levels produces biased estimates. The bias increases with greater preperiod level differences and weaker serial correlation in the outcome. This problem extends to matching on preperiod level of a time‐varying covariate. When treatment assignment is correlated with preperiod trend only, the unmatched analysis is biased, and matching units on preperiod level or trend does not introduce additional bias.
To better explain, Melissa Garrido gives an example on the Incidental Economist blog.
Consider a hypothetical study of the impact of a counseling program on hospital satisfaction scores among bereaved spouses. The treatment group would include spouses who received counseling. The comparison group might include spouses of decedents from nearby hospitals that do not offer the counseling program. You notice that the treatment group has lower mean satisfaction scores than the comparison group and try to correct this by matching the groups on pre-death satisfaction.
This means that your analytic sample now contains the comparison individuals who have below-average pre-death satisfaction (relative to the mean of the original, pre-matched group of comparison individuals) and the treatment individuals who have above-average pre-death satisfaction (relative to the mean of the original group of treatment individuals). Over time, satisfaction levels of the matched comparison individuals may increase towards the overall comparison group mean, and satisfaction levels of the matched treated individuals may decrease towards the overall treatment group mean. Even with a null effect of counseling, it may appear as if counseling decreased satisfaction with care. This is bias due to regression to the mean.
In short, propensity score matching is acceptable in some cases, but in general be very weary about combining these two econometric techniques.
Source:
- Daw, Jamie R., and Laura A. Hatfield. “Matching and Regression to the Mean in Difference‐in‐Differences Analysis.” Health services research (2018).