Chapter Four. Effectiveness Data and Analysis

In this chapter, we provide the data sources and assumptions used to develop our estimates of the effectiveness of implementing the 14 interventions.

Estimates of Each Intervention’s Effect on Injuries and Deaths

There is a range of potential benefits associated with the implementation of motor vehicle interventions. Across all interventions, a primary benefit is the reduction in injuries and deaths associated with crashes. For offender interventions (e.g., alcohol interlocks), however, implementation may also lead to increased employment or quality of life among offenders affected by the intervention who choose not to drive while impaired. Although the full range of benefits should be considered as part of the debate, we have chosen to focus our estimates on the primary benefit of reduced injuries and deaths because that is what the literature best supports.

To estimate the state-specific effect of each intervention, we sought to identify, from the existing literature, the “best” estimate of the intervention’s effect on injuries and deaths and apply it to each state. We began by reviewing the literature for each intervention and documenting it in the associated fact sheet. We started with the literature cited in the Countermeasures That Work report (UNC Highway Safety Research Center, 2011) and then searched via electronic databases (e.g., MEDLINE, Web of Science, Social Sciences Abstracts) for any additional studies that had been published in the interim. For each intervention, we assessed the existing literature and chose studies based on several criteria, which are described in order of importance below.

First, we included studies that provide information on the primary outcomes of interest, such as crashes, injuries, and deaths, as opposed to less direct outcomes, such as recidivism. Second, we reviewed the methodologies used to estimate the effect of the intervention, selecting those with rigorous study designs. For example, we gave preference to studies that use comparison-group designs with before-and-after measurements to examine the impact of an intervention (rather than, e.g., control states or other geographic areas) over those that use only a before-and-after design. Third, we considered the dates when the interventions were implemented and gave preference to studies that examined interventions that were implemented more recently. We expect that, all else equal, estimates derived from more-recent experiences will be more applicable and provide better estimates of what might be expected to occur if an intervention were implemented in the near future. Fourth, we consider where the intervention was implemented. All else equal, we favored studies looking at interventions in the United States because these estimates are more likely applicable than interventions elsewhere.

In addition, we relied on meta-analyses and systematic reviews when available to identify studies accepted and cited in the field. These were also helpful in understanding whether our selected study was an outlier in the literature. Although we do not always refer specifically to an existing meta-analysis, we have frequently used it as background to inform the selection of the preferred study. In most cases, we selected a single study that was deemed to be the best match to the criteria. This made it easier to ensure that the assumptions underlying the estimate were carried through our calculations.

Once we selected a study, we abstracted several pieces of information. First, we extracted estimates on the intervention’s effect on motor vehicle injuries and deaths. In studies in which these outcomes were not examined, we extracted information on the reported outcome, such as the impact on recidivism or crashes. Second, we documented the specific data set and baseline used to generate the effectiveness estimate. This is especially important because many studies use different baselines for their analysis. For example, Fell, Tippetts, and Levy, 2008, found that sobriety checkpoints reduce alcohol-related deaths. It would be incorrect to apply the reduction conclusion to all fatal crashes.

In many cases, the literature on these interventions does not consider the intervention’s effect on both motor vehicle injuries and deaths. In these circumstances, we made some assumptions to translate the estimates in the literature to effects that we can use. In many cases, we adopted the methodology applied in Preusser et al., 2008, which assumes proportional impacts on both injuries and deaths; that is, if the intervention reduced deaths by 10 percent, we assumed that injuries were reduced by 10 percent as well. We believe that this is a reasonable assumption for most of the interventions, although we recognize that this is a limitation of our methodology.

We presented our initial selection of estimates from the literature to experts at CDC for review. They provided input on the selected estimates and suggested additional studies and sources of information for several of the interventions. We reviewed the additional information and revised our selected estimates accordingly. The intervention’s estimated effect on injuries and deaths, the source for the estimates, and any assumptions made to translate the estimate are presented in Table 4.1. Following Table 4.1, we describe how the effectiveness is empirically determined in the source documents.

Table 4.1. Basis for Estimates of Fatality and Injury Reduction

Intervention	Estimated Effect	Source
Red-light cameras	17% of deaths at intersections with signals	Hu, McCartt, and Teoh (2011) conducted panel data analysis and found that red-light cameras reduce fatal crashes by 17%. We assume proportional responses on injuries.
Speed cameras	12% reduction in speed-related crashes	Cunningham, Hummer, and Moon (2005) studied North Carolina speed limit–enforcement cameras and found a 12% reduction in speed-related crashes. We assume a proportional response in injuries and deaths.
Alcohol interlocks	24% reduction in crashes of those with previous DWI	DeYoung, Tashima, and Masten (2005) studied California interlock program, comparing DWI offenders with interlock restrictions and those without. They found a 24% reduction in crashes. We assume a proportional response on both injuries and deaths.
Sobriety checkpoints	8.1% reduction in alcohol-related deaths	Fell, Tippetts, and Levy (2008) studied demonstration projects using FARS data. They studied 7 programs, and we take the average effect as our main estimate.
Saturation patrols	17.9% reduction in alcohol-related deaths	Fell, Tippetts, and Levy (2008) cited a 17.9% drop in fatal crashes in Michigan. We assume a proportional response on injuries.
Bicycle helmets laws	15% reduction in cyclist deaths	Grant and Rutner (2004) studied the effect on juvenile cyclist deaths. We assume a proportional effect on injuries.
Motorcycle helmets laws	28.9% reduction in motorcyclist deaths	Sass and Zimmerman (2000) looked at the effect on motorcyclist deaths. We assume a proportional effect on injuries.
Primary enforcement of seat belt laws	7% reduction in deaths involving passenger vehicles	Farmer and Williams (2005) studied the effect on passenger deaths. We assume proportional effect on injuries.
Seat belt enforcement campaign	5.4% reduction in deaths involving passenger vehicles	Solomon, Ulmer, and Preusser (2002) studied the effects that CIOT campaigns have on seat belt usage. Using Preusser et al. (2008), we converted this to a 5.4% reduction in injuries. We assume proportional effects on both injuries and deaths.
License plate impoundment	27% reduction in recidivism for those with previous DWI	Leaf and Preusser (2011) studied the effect on recidivism. They estimated that DWI offenders subject to impoundment had a 27% reduction in recidivism relative to offenders not subject to impoundment. We assume proportional effects on both injuries and deaths.
Limits on diversion and plea agreements	11% reduction in recidivism for those with previous DWI	Wagenaar et al. (2000) presented estimates on several outcomes. We use a summary estimate of an 11% reduction that is reported in the Countermeasures That Work report. We assume proportional effects on injuries.
Vehicle impoundment	30.4% in crashes for those with previous DWI	DeYoung (1999) studied the decrease of crashes due to DWI offenders. We assume a proportional effect on injuries and deaths due to drivers with a previous DWI.
In-person license renewal	9% reduction in fatal crash involvement rates for drivers ages 55+	Tefft, 2014, compares states with in-person license renewal and those without. The author found a 9% decrease in fatal crashes for ages 55+ with little evidence that this reduction varies significantly by age in this range.
Higher seat belt fines	7.2% reduction in fatalities involving passenger vehicles	Houston and Richardson, 2005, uses changes in state-level seat belt fines to estimate that a $1 increase is associated with a 0.152-percentage-point increase in seat belt use, implying an 11.4% increase for a $74 fine. Using Preusser et al., 2008, this increase translates to a 7.2% decrease in injuries and deaths.

NOTE: All effects on injuries are assumed to be the same as those on deaths except with sobriety checkpoints, for which the effect on injuries is 20 percent.

Red-Light Cameras

Hu, McCartt, and Teoh, 2011, compared the change in per capita fatal crash rates between 1992–1996 and 2004–2008 in cities with red-light camera enforcement and those in cities that did not have such enforcement in those years. Their analysis included 62 cities, using FARS data, a Poisson regression model, and accounting for city fixed effects by including pre- and post- periods. They focused on fatal crashes at intersections with signal lights. They found decreases in both the treatment and comparison groups, but the treatment-group decrease was 17 percent larger.

Speed Cameras

Cunningham, Hummer, and Moon, 2005, analyzed the introduction of speed cameras in Charlotte, North Carolina, along 14 key corridors using data from 2000 to 2004 and similar “comparison sites” as controls. This is a differences-in-differences design, meaning that the study compares changes in the number of crashes in sites where cameras were introduced and changes in the number of crashes where cameras were not introduced. They estimated a 12-percent reduction in crashes.

Alcohol Interlocks

DeYoung, Tashima, and Masten, 2005, compared California drivers with interlocks and those without, using a propensity-score design to try to adjust for differences between the two groups. Because drivers who receive interlocks may be different from those who do not, the authors controlled for the probability of receiving an interlock based on observable characteristics. They ran a hazard model and looked at days until first crash as the outcome. The results suggest that interlocks reduce the probability of involvement in a crash by 24 percent (p. 20). Other samples were also analyzed and got different results, but this one is the most relevant for our purposes. The literature on interlocks typically focuses on the effect on recidivism, rather than the effect on crashes, due to small sample sizes. DeYoung, Tashima, and Masten, 2005, is one of a small number of studies that looks at the effect on crashes. As such, there is not broad consensus in the literature on interlocks’ effects on crashes. Still, the DeYoung study used a solid design and offers insights on the potential effects that alcohol interlock use can have on the outcome of interest.

Sobriety Checkpoints

Fell, Tippetts, and Levy, 2008, using FARS data from 1987 to 2003, reported estimating several interrupted time series to test for effects of the implementation of sobriety checkpoints in seven states. NHTSA had funded these demonstration projects, and the authors tested for changes in the ratio of drinking to nondrinking drivers in fatal crashes. We interpret the results as the effect on alcohol-related deaths, accounting separately for the total number of deaths. In practice, the authors actually used neighboring states as comparison groups. The paper reports only the results for each state. We aggregated the results to arrive at an 8.1-percent reduction.

Elder, Shults, et al., 2002, conducted a systematic review of the effects of sobriety checkpoints. We focus on the paper’s evaluation of the effects of selective breath testing (SBT) checkpoints. They reported a median finding in the literature of a 20-percent reduction in fatal and nonfatal injury crashes.

Saturation Patrols

Fell, Langston, et al., 2008, studied the introduction of sobriety checkpoints or saturation patrols in seven states. Michigan implemented highly publicized saturation patrols in 2002 and 2003. Using FARS data, the authors estimated the change in fatal crashes relative to vehicle-miles traveled. They estimated a significant decrease of 18 percent in the number of alcohol-related deaths.

Bicycle Helmet Laws

Grant and Rutner, 2004, used the adoption of bike helmet laws to study their impact on juvenile cyclist deaths in the FARS. They estimated a Poisson model with state and year fixed effects and found a 15-percent reduction in deaths.

Motorcycle Helmet Laws

Sass and Zimmerman, 2000, used panel data for all 50 states for 1976 through 1999, conditioning on state fixed effects and used state-level changes in motorcycle helmet laws to estimate the relationship between such laws and deaths. Their outcome variable was the log of motorcyclist deaths per capita, and they estimated their specification using ordinary least squares (OLS). Evaluated at the mean death rate in the sample, their estimates suggest that helmet laws reduce the per capita motorcyclist death rate by 28.9 percent.

Primary Enforcement of Seat Belt Laws

Farmer and Williams, 2005, looked at changes in death rates by comparing ten states that switched from secondary enforcement to primary enforcement and 14 states that remained with secondary enforcement between 1989 and 2003. They found a 7-percent decrease in the FARS in the switching states compared with the control states.

Seat Belt Enforcement Campaign

Solomon, Ulmer, and Preusser, 2002, studied how CIOT campaigns affect belt use. They compared changes in belt use in ten states that implemented CIOT and use in four states that conducted enforcement with limited advertising and four other states with only enforcement. They found that the full-implementation states increased seat belt usage rates by 8.1 percentage points relative to the enforcement-only states. We use evidence found in Preusser et al., 2008, regarding the effect that seat belt use has on the risk of death in a crash to translate the increase in seat belt use to a 5.4-percent reduction in deaths.

License Plate Impoundment

Leaf and Preusser, 2011, studied first-time DWI offenders in Minnesota. They compared people with BACs of 0.20 to 0.22 and people with BACs of 0.17 to 0.19. Although these groups should be similar, the Minnesota law allowed for license plate impoundment only for those with BACs of at least 0.20. Their outcome variable was recidivism, and they found that the group subject to license plate impoundment had a lower rate of recidivism. We calculate the decrease as a 27-percent decrease in recidivism, which we use to project a 27-percent decrease in crashes involving people with previous DWI convictions.

Limits on Diversion and Plea Agreements

Wagenaar et al., 2000, reviewed 52 studies of plea-agreement restrictions and found reductions on several measures, including recidivism. The Countermeasures That Work report aggregates the findings and reports a reduction of 11 percent, which we apply to drivers with previous DWI convictions. Unfortunately, we do not have much better evidence than this number, so we assume an 11-percent reduction in injuries and deaths due to limits on diversion and plea agreements.

Vehicle Impoundment

DeYoung, 1999, focused on four jurisdictions in California with data on impoundments and driver records. In 1995, California began impounding vehicles for some driving offenses. The author compared the one-year driving records of subjects with impounded vehicles and the records of a control group. This control group was made up of people who would have had their vehicles impounded under 1995 California law but did not because they committed their driving offenses in 1994. We focus on the results using crashes as the outcome. The author found that the group with impounded vehicles was involved in 24.7 percent fewer crashes when selecting on first offenders. For repeat offenders, the reduction was 37.6 percent. According to data published in the study, 55.8 percent of the sample were first-time offenders, so we use a weighted average of the two results to arrive at a reduction of 30.4 percent. We assume that this effect applies to those with suspended or revoked licenses due to DWI convictions.