3 A comment on Colantone et al. (2024, APSR)
3.1 Abstract
3.1.1 The paper’s main claim
Background:
major ban on polluting cars introduced in Milan, which was strongly opposed by the populist right party Lega.
The policy identifies the most polluting categories of vehicles and bans them from accessing and circulating within the area.
Main claim:
~'This policy caused people [who were financially hurt] to vote for Lega'. [or more generally to be more right wing]
3.1.2 Main evidence for this claim:*
the difference in the differences of outcomes by emission category, that is, between Diesel versus Petrol owners of Euro4, and Diesel versus Petrol owners of Euro5."
In general this is a ‘difference in differences’ without a time dimension, although some of their regressions control for previous voting record.
Our main dependent variable is an indicator that takes the value 1 if the respondent reports voting for Lega, and 0 otherwise.
the average estimated effect is 13.5 percentage points … the baseline rate of support for Lega in the sample was 24.4%. … owning a car affected by the vehicle ban raised the probability of voting for Lega in the subsequent elections by 55% above the baseline rate"
3.1.3 Secondary claim(s)/other key claims:
electoral change did not stem from a broader shift against environmentalism, but rather from disaffection with the policy's uneven pocketbook implications
They also argue that ~‘people who were compensated did not shift in response to the policy’.
These are implicitly claims of (1) null or bounded effects and (2) differential effects (for those compensated vs. those not compensated.) This is important to the authors’ theoretical interpretation of the effect, and its generalization to political behavior as a whole.
3.1.4 Replication success
The paper’s results could be reconstructed from the data and code provided, with only a moderate amount of effort
We also recoded the construction of their replication data from the survey data they provided and their description. There were no discrepancies.
We considered a handful of alternate specifications and robustness checks. The qualitative results were preserved in every case we examined.1
3.2 Introduction
[Brief overview of the original study, its research question, data, and methods]
3.2.1 Original Study
[Brief description of the original study’s main findings and conclusions]
3.3 Computational Reproducibility
[Description of the process of computationally reproducing the original study’s results using the provided replication package]
See chapter (ref?)(computational),
3.3.1 Discrepancies and Coding Errors
[Discussion of any discrepancies or coding errors found during the computational reproduction process]
3.4 Robustness Reproduction and Replication
See chapter (ref?)(robustness),
[Explanation of the robustness checks and replication efforts undertaken, including changes to the sample period, clustering of standard errors, or other modifications]
3.4.1 Regression Model
[Description of the regression model(s) used in the original study and any modifications made for the robustness checks or replication]
3.4.1.1 Results: [Outcome Variable 1]
3.4.1.1.1 Clustering
[Discussion of the results when the clustering of standard errors is modified, including any changes in the sign, magnitude, or statistical significance of the estimates]
3.4.2 Results: [Outcome Variable 2]
[Repeat sections 3.2.1 and 3.2.2 for additional outcome variables]
3.5 Conclusion
[Summary of the main findings from the replication effort and their implications for the original study’s conclusions]
3.6 References
See
However, because of time constraints, we were not able to complete all of the robustness checks and methodological comparisons we had hoped to do.↩︎