# Interrupted Time Series and Regression Discontinuity 2019-12-07¢ Interrupted time series...

date post

13-Apr-2020Category

## Documents

view

0download

0

Embed Size (px)

### Transcript of Interrupted Time Series and Regression Discontinuity 2019-12-07¢ Interrupted time series...

Interrupted Time Series and Regression Discontinuity Design

Marcelo Coca Perraillon

University of Colorado Anschutz Medical Campus

March 2019

1 / 90

Outline

Interrupted time series

Basic design

Estimation

Validity issues

Extensions

Overview of regression discontinuity

Meaning and validity of RDD

Several examples from the literature

Estimation (where most decisions are made)

Sharp RRD example: Nursing home ratings

Fuzzy RDD example: Early intervention therapies

2 / 90

Code/data

You can get code and example dataset from my website (click on Code on left menu):

http://tinyurl.com/mcperraillon

3 / 90

http://tinyurl.com/mcperraillon

ITS

Interrupted time series

Interrupted time series is a fairly basic before and after analysis

You have probably learned, for good reasons, that before and after analyses are suspicious

Yet, in some circumstances, and under some assumptions, they can be fairly convincing designs

Today, I’ll separate design and estimation which in general is the way to go

See Rubin (2007,2008). Bottom line: “[O]bservational studies can and should be designed to approximate randomized experiments as closely as possible. In particular, observational studies should be designed using only background information to create subgroups of similar treated and control units, where similar here refers to their distributions of background variables.”

4 / 90

ITS

Interrupted time series

Time series: observations for a single variable made consecutively over time

Could be same units or different units

Example: prescription numbers/rates over time for different people in each time period or the same group of people measured at different points over time

At some point during the observation period there is change that creates an “interruption” in the time series

For example, a policy change like Medicare part D implementation in 2006 or the release of a black label warning for antidepressants use in children would create an “interruption” or a change in payment policy in Medicaid

The idea is to use that interruption to measure the effects of policies

5 / 90

ITS

Example Medicaid in New Hampshire imposed a three-drug limit that restricted medication reimbursement among among chronically ill poor patients with cardiac and other chronic illnesses The outcome: the mean number of prescriptions declined by half Soumerai et al (2017, 1987).

6 / 90

ITS

Example Medicare Part D changes in prescriptions for Medicare beneficiaries Schneeweiss et al (2009) Use pharmacy claims for the elderly–problems with measurement and establishing insurance coverage

7 / 90

ITS

Thinking about effects

Note that in the two examples effects can be described in different ways, which has implications when thinking about the estimation part

Is a change in the level? Is it a change in the rate? In a model, intercept vs slope changes

Is the change continuous or does it decay? In the Medicaid example, there was another change prompted by the concerns about prescription declines

Is the effect immediate or delayed?

In health policy, changes often happen before the actual policy implementation. Example, Medicaid expansion or Part D

8 / 90

ITS

Estimation

Estimation is (relatively) simple with models parametrized in different ways depending on whether one assumes a change in level or a change in slope or if other features are also incorporated

For example: y = β0 + β1t + β2post + β3t ∗ post + �, where post = 1 if after the policy intervention and t is time

In the pre-period: E [ypre |x] = β0 + β1t In the post-period: E [ypost |x] = (β0 + β2) + (β1 + β3)t Note that several tests are possible. If β2 = β3 = 0 then there is no policy effect

If β2 = 0 but β3 6= 0 then there was a change in the slope, but not the level after the interruption. β3 is the change in the rate (remember, interactions in the linear model are differences of differences). The slope after the policy is given by β1 + β3

9 / 90

ITS

Estimation

For the Medicaid example above, a Wald test for β2 would be more relevant since there doesn’t seem to be much happening in the slope

A model like y = β0 + β2post + � could be a better fit

If you think that a change in slope and not level is a better assumption, splines are an option.

With splines, we can model a change in the slope with no change in the level

10 / 90

ITS

Splines

The policy change (“interruption”) happens at time t = k . In splines lingo k is the knot

The model is y = β0 + β1time + β2(time − k)+� The (z)+ is called a truncated line function and is defined as being equal to z if z is positive and zero otherwise

So (time − k)+ will be equal to time (time − k) after the policy change and and zero if at policy knot or before

The only difficult part about splines is to get the coding right, the rest is (relatively) easy

See Stata’s mkspline command. You can do all sort of things with splines (they don’t have to be linear). Sometimes they are called piecewise regression

11 / 90

ITS

Splines

Model: y = β0 + β1time + β2(time − k)+ + � Before the policy change the model is A) : E [y ] = β0 + β1age

After: If time > k the model is: E [y ] = β0 + β1time + β2(time − k) Same as centering (more on it soon) so when time > k we can rewrite is B): E [y ] = (β0 − β2 ∗ k) + (β1 + β2)time If β2 = 0, then the slope before and after is the same (so A and B are the same)

Note that β2 is the incremental change in slope

The trick of using the truncated function is that it allowed us the possibility of a different slope after k

See Lopez Bernal (2017) for other parametrizations

12 / 90

ITS

Statistical issues

The models I wrote above are linear models implicitly assuming errors are iid and �i ∼ N(0, σ2) or, equivalently, �i ∼ N(β0 + β1X1i + · · ·+ βpXpi , σ2) Of course, two issues: 1) The outcomes drives distributional assumptions. If counts, for example, a negative binomial or Poisson model would better

2) The errors cannot be iid since they can’t be independent. Autocorrelation is a feature of time series

Although the linear model is unbiased, the standard errors would be wrong

Autoregressive integrated moving average (ARIMA) or other solutions are commonly used. There are test for autocorrelation, like the Breusch-Godfrey test

Another issue (seasonality)

13 / 90

ITS

Validity

Intuitively, the main validity issue is whether other changes happening after the policy/interruption could have caused the change in y

Your textbook also mentions instrumentation (changes in procedures changes measurement) and

Selection (the composition of the group changes after policy change)

Compelling interrupted time series designs tend to be short-term, with changes that are hard to explain otherwise

14 / 90

ITS

Extensions

Adding a nonequivalent no-treatment control group time series

As the duck test describing an example of abductive reasoning goes: If it looks like a duck, swims like a duck, and quacks like a duck, then it probably is a duck

Yes, this is the same as difference-in-difference models... See Wing et al. (2018)

Note how the parallel trend and the common shocks assumptions of DiD are extensions of the issues discussed in Chapter 6

Please read Chapter 6. Don’t underestimate your textbook

15 / 90

ITS

And now for something completely different...

Switching gears: regression discontinuity design

16 / 90

Introduction

Basics

Method developed to estimate treatment effects in non-experimental settings

Provides causal estimates of treatment effects. Those estimates are Local Average Treatment Effects (LATE)

Design limits external validity in some cases

Good internal validity; some assumptions can be empirically verified

Relatively easy to estimate but it has some complications

First application: Thistlethwaite and Campbell (1960) (Does the last name Campbell sound familiar?)

17 / 90

Introduction Thistlethwaite and Campbell

Thistlethwaite and Campbell

They studied the impact of merit awards on future academic outcomes

Awards allocated based on test scores

If a person had a score greater than c , the cutoff point, then she received the award

The wrong way of analyzing: compare those who received the award to those who didn’t

Thistlethwaite and Campbell realized they could compare individuals just above and below the cutoff point

By now I find this idea intuitive but it’s not at first. It helps if you think that choosing the point c is arbitrary or measured with error

Say, it’s 1200. But why not 1210? Or 1190? We know that the test scores used to give scholarships is related to future academic outcomes but there is no solid relationship between 1200 and the

Recommended

*View more*