Ideology Models Only Account for 12.5% of the Votes!

I need to make a clarification about something. On several occasions – both in here, on Howard’s list and on ELS – I stated that newspaper reputation accounts for only 24% of the votes that justices cast in civil liberties cases over the last (almost) 60 years. That figure may be somewhat misleading. I grabbed it from phi-p, which is a correlation statistic used in contingency-table analysis. But whether or not it is misleading, one of the things that is undeniable is that in the ecological regression of civil liberties votes endorsed by numerous political science scholars, only 12.5% of the total votes cast in civil liberties are responsible for the 41% of the variance in the liberal index that newspaper reputation “explains.” Only 12.5%!! Let’s take a closer look.

As I have demonstrated previously, the R-squared in an ecological regression involving Segal/Cover scores and career liberal ratings in civil liberties cases from 1946-2004 is 0.41. Keep in mind that this figure is not the explained variance of the votes; it is the explained variance in the numbers comprising the liberal index. To equate the one with the other is an ecological fallacy. To see just how damaging this fallacy is, I have provided a table which looks closely at what this R-squared statistic is reporting. The table can be accessed here.

The table is useful for several reasons. First, it breaks down the explained and unexplained variance that occurs in the numbers comprising the liberal ratings. It also, however, breaks down the number of justice votes implicated by those percentage points. As one can plainly see, the number of votes that accompany the explained portion of the regression is only 12.5% of the total number of votes that comprise the entire regression (31,049). That means the regression is only able to rely upon 12.5% of the votes to explain 41% of the variance in the ratings.

Another thing that is interesting is how each justice is affecting (or driving) the R-squared. This can be located in the column to the far right, called ERL (Explained Ratings Load). This column is simply referring to the contribution that each justice is making to the R-squared statistic (the explained variance). The justices who are actually driving the statistic the most are the ones who carry the highest proportion of influence (“load”). Scalia is a good example; he accounts for over 6% of the R-squared by himself. Justice Rehnquist is right behind him.

But now, however, examine the column titled EVL (Explained Voting Load). This column shows the proportion of votes that are hiding behind each of these “rating loads.” This is simply the proportion of the 12.5% of the total votes to which each justice contributes. Here we find something else of interest. First, there are, as one might expect, some justices who “artificially” contribute to the R-squared by having “payoffs” in their percentage points that are not matched by their votes. Good examples of this are justices Goldberg, Fortas, Rutledge, Jackson and Thomas (to some extent). To see this better, examine the following graph (the red lines of those justices – the percents – are disproportionate to the grey lines, the votes). Also, of the 12.5% of the votes that are needed to drive the 41% of the ratings, the bulk of the work comes disproportionately from four key justices: Rehnquist, Blackmun, Brennan and Douglas.

I’ll have more to say about this in a couple of days. I’ve got to run now.