Evaluating Place-Based Transportation Plans

Research Team: Adam Millard-Ball (lead), Jason Karpman, and Melody Ng

UC Campus(es): UCLA

Problem Statement: California has increasingly turned to place-based, community-driven programs such as Transformative Climate Communities (TCC), the Community Air Protection Program (CAPP), and Regional Climate Collaboratives (RCC) to address the twin priorities of climate change and environmental justice. Transportation improvements are at the heart of these programs because of the potential to mitigate air pollution, greenhouse gas emissions, and inequities in access to transportation. However, these efforts are inherently difficult to evaluate as they often involve a diverse set of projects with different timelines and locations. Moreover, evaluators often face the challenge of isolating the effects of individual programs. Carefully selected control sites can support this effort, but no two communities are exactly alike, limiting the ability of evaluators to make meaningful comparisons.

Project Description: This research addresses how place-based transportation plans are being evaluated, and what insights from the broader policy and plan evaluation research literature might inform evaluation design. The researchers complemented a review of the evaluation literature with six expert interviews with 15 people. They found that California agencies and their community partners have high expectations for evaluations of place-based transportation plans. So far, however, those evaluations have been less successful in providing detailed information on outcomes and the causal impact of interventions. This does not reflect the shortcomings of the evaluation teams, but rather the inherent challenges in holistically assessing a diverse set of projects on different implementation timelines in a project area with porous boundaries. There is also a fundamental difficulty with the evaluation scale. California’s place-based transportation plans have often been evaluated individually. But in general, evaluations, particularly quantitative causal inference methods, are most effective with a larger number of projects or sites. The researchers suggest a two-pronged approach to addressing the tensions that we identify between place-specific knowledge and generalizable conclusions. The first prong, at the site level, would emphasize process evaluations and assessment of outputs and outcomes. The second prong would focus on impacts across multiple sites and the extent to which place-based transportation programs have a causal role.

Status: Completed

Budget: $25,000