Aggregates and Votes in Quantitative Ideology Models

(sent to law courts)

Hi Howard.

Good to hear from you again. Haven't heard from you in a long, long time. As I think you know, I am quite aware that Jeff uses an ecological model. About two years ago, I spent a great deal of time comparing ecological models with logit models. So I know what is being said. One of the things I found out about those comparisons is that it is a mistake to think that an ecological model is "something different." It isn't; it just requires much more cautious interpretation. To really interpret it, one needs to "look under the hood," so to speak (before aggregation). In fact, I would disagree with you to the extent you suggest that Jeff's model gets some sort of free pass from conducting logit diagnostics underneath those percentages. Any responsible researcher would do so. In fact, that was one of the central flaws that stung the quantitative ideology research program in the first place -- it committed major methodological sins in the substantive interpretation of its ecological offering. And so I would never be from the school or thought that tried to say that one who analyzed aggregates was doing something unconnected with what those figures are summarizing "underneath it all."

The reason why I had asked Jeff for the latest version of his ecological model is for two reasons. First, if he is truly using the aggregated data from the entire data set as the dependent variable, the distribution of votes will be more leptokurtic. And as I believe was your central point, an ecological model is only analyzing the variance of an index. And before we analyze index variance, we would want to know to what degree values within the index cluster around the mean. Because the more leptokurtic the index, the less we would be substantively impressed with a high account of its variance. If you look at the logit model, this becomes instantly clear. The more the values are non-directional, the more the model goes in the tank. So if Jeff is now trying to offer a high correlation in the variance of a an index that isn't varying as much as before to begin with, someone really needs to catch that -- at least for Paul's sake. (And others).

Now, let me do this much more easily. In my 2006 piece, I did something that I thought was very interesting. I went ahead and took Jeff's world "as is." I took his ecological model on its face and dissected it. What I did was break down what the r-squared in that regression was really reporting, by converting the explained versus residual sum of squares into the equivalent number of votes that accounted for each portion. When I did this, I found that only 12.5% of the total votes cast were "explained" by an ecological regression of a civil liberties INDEX. So the headline would be: model correlates with 60% of index variance, and, in doing so, explains only 12.5% of the votes accounted for by the summaries constituting the index. (This is a good illustration of how analyzing votes is supplementary to aggregates and not something of a different kind).

One more point. Howard, may I ask something of you? Why is it that you continue to say that Jeff has an "attitudinal model" here? You didn't say this in the Law and Society piece from a long, long time ago. I think we need to be clear. Jeff has only a model that has variables that gather something from the external world. What he names it is not germane. Hence, what he has is something in the nature of small-group media-perception scores constructed using a political stereotype. He then regresses that in an ecological model against the summary rates in which justice-approaches to legal issues end up favoring particular claimants. Who those claimants are is determined by what we might call the Harold-Spaeth "client list," which is another construction. I mean, there is no one on the plant who thinks that every single issue the court decides in bankruptcy cases, tax cases, economic cases, etc. etc. etc. are "liberal and conservative" because one side had to win. And so you have a forced stereotype score being regressed against an assigned client-winning profile. This is NOT a model that measures attitudes. I don't think it can even accurately claim to measure journalist attitudes for crying out loud.

So why is it that political scientists talk this way? No other science talks this way. Real science is supposed to accurately describe what is measured in the external world. All you have here is a contrived media perception score regressed against a constructed claimant-winning profile. It is not an "attitude model." And it surely isn't "the justice ideology and the votes."

When are political scientists doing this work going to actually adopt basic principles of science, such as rigidly explaining phenomenon under study in the external world?

Howard, as always, regards and thanks. (Please do write me again in the future like you used to in the past).