Ecological and Logit Predictions of Liberal Ratings

[version 2.0]*

After having my head under the hood of Segal-and-Spaeth’s bivariate ideology model over the last week, I discovered something interesting: ecological regression does not effect how efficiently Segal/Cover scores predict aggregate liberalism on the Court. I had suspected the opposite. Although the two sets of predictors are quite similar and their overall difference is small, there does appear to be one interesting pattern: logit regression predicts liberal justices slightly better while ecological regression predicts conservative justices slightly better. Overall, ecological regression predicted 18 justices better; logit regression predicted 14 better. The best way to demonstrate these findings is by examining the following table and a graph. (Click bold words to access. Look at the output yourself. The graph really shows the story the best. A picture is worth a thousand words).

To explain the findings better, consider the Court’s two newest voters. Just how conservative will Alito and Roberts be? I’ve uploaded another table that analyzes the data.[1]  Alito’s newspaper reputation for political values was -0.8 (see Jeff Segal’s website). That means Alito’s career liberalism is estimated to be 0.355 using a logit model and 0.343 using an ecological model. Justice Roberts’ newspaper reputation, by contrast, was -0.76, making his estimated liberalism 0.364 (logit) and .351 (ecological). In short, both of the new justices are predicted to have a liberal tendency roughly equal to the aggregate tendencies of O’Connor, Kennedy and Powel (to say nothing of the policy differences that will comprise this tendency).

Now the big question: how accurate are these forecasts? The answer is: not incredibly. The average mistake that these forecasts generate is 13 points for ecological regression and 13.1 for logit regression. That means that, on average, Alito could be the next Rehnquist or the next (almost) Stewart. But at least we have a reasonable basis (before he takes the bench) for knowing that he is not the next Stevens or Brennan. Interestingly, if you regress the predictions generated by newspaper reputation against the reality that eventually emerges (the true aggregate liberalism), the R-squared is .4117 for ecological predictions and .4081 for logistic ones. Hence, ecological regression does not affect how well Segal/Cover scores can forecast predictions of an aggregated tendency. But in either case the quality of the forecast that emerges is “partly cloudy.”

---

[1] The table shows not only the difference between ecological and logit estimations, but also the difference between estimations based upon modeling decisions that exclude Truman and Roosevelt appointees versus those that do not. There is some controversy about whether these justices should be excluded. My view is that they should not be. I hope to author an entry about that point later.

* excised unnecessary final paragraph and changed the title.