Hi Howard.

Good to hear from you again. Haven't heard from you in a long, long time. As I think you know, I am quite aware that Jeff uses an ecological model. About two years ago, I spent a great deal of time comparing ecological models with logit models. So I know what is being said. One of the things I found out about those comparisons is that it is a mistake to think that an ecological model is "something different." It isn't; it just requires much more cautious interpretation. To really interpret it, one needs to "look under the hood," so to speak (before aggregation). In fact, I would disagree with you to the extent you suggest that Jeff's model gets some sort of free pass from conducting logit diagnostics underneath those percentages. Any responsible researcher would do so. In fact, that was one of the central flaws that stung the quantitative ideology research program in the first place -- it committed major methodological sins in the substantive
interpretation of its ecological offering. And so I would never be from the school or thought that tried to say that one who analyzed aggregates was doing something unconnected with what those figures are summarizing "underneath it all."

The reason why I had asked Jeff for the latest version of his ecological model is for two reasons. First, if he is truly using the aggregated data from the entire data set as the dependent variable, the distribution of votes will be more leptokurtic. And as I believe was your central point, an ecological model is only analyzing the variance of an index. And before we analyze index variance, we would want to know to what degree values within the index cluster around the mean. Because the more leptokurtic the index, the less we would be substantively impressed with a high account of its variance. If you look at the logit model, this becomes instantly clear. The more the values are non-directional, the more the model goes in the tank. So if Jeff is now trying to offer a high correlation in the variance of a an index that isn't varying as much as before to begin with, someone really needs to catch that -- at least for Paul's sake. (And others).

Now, let me do this much more easily. In my 2006 piece, I did something that I thought was very interesting. I went ahead and took Jeff's world "as is." I took his ecological model on its face and dissected it. What I did was break down what the r-squared in that regression was really reporting, by converting the explained versus residual sum of squares into the equivalent number of votes that accounted for each portion. When I did this, I found that only 12.5% of the total votes cast were "explained" by an ecological regression of a civil liberties INDEX. So the headline would be: model correlates with 60% of index variance, and, in doing so, explains only 12.5% of the votes accounted for by the summaries constituting the index. (This is a good illustration of how analyzing votes is supplementary to aggregates and not something of a different kind).

One more point. Howard, may I ask something of you? Why is it that you continue to say that Jeff has an "attitudinal model" here? You didn't say this in the Law and Society piece from a long, long time ago. I think we need to be clear. Jeff has only a model that has variables that gather something from the external world. What he names it is not germane. Hence, what he has is something in the nature of small-group media-perception scores constructed using a political stereotype. He then regresses that in an ecological model against the summary rates in which justice-approaches to legal issues end up favoring particular claimants. Who those claimants are is determined by what we might call the Harold-Spaeth "client list," which is another construction. I mean, there is no one on the plant who thinks that every single issue the court decides in bankruptcy cases, tax cases, economic cases, etc. etc. etc. are "liberal and conservative" because one side
had to win. And so you have a forced stereotype score being regressed against an assigned client-winning profile. This is NOT a model that measures attitudes. I don't think it can even accurately claim to measure journalist attitudes for crying out loud.

So why is it that political scientists talk this way? No other science talks this way. Real science is supposed to accurately describe what is measured in the external world. All you have here is a contrived media perception score regressed against a constructed claimant-winning profile. It is not an "attitude model." And it surely isn't "the justice ideology and the votes."

When are political scientists doing this work going to actually adopt basic principles of science, such as rigidly explaining phenomenon under study in the external world?

Howard, as always, regards and thanks. (Please do write me again in the future like you used to in the past).

Dr. Sean Wilson, Esq.
Assistant Professor
Wright State University
Redesigned Website: http://seanwilson.org/
SSRN papers: http://ssrn.com/author=596860
Twitter: http://twitter.com/seanwilsonorg
Facebook: http://www.facebook.com/seanwilsonorg
New Discussion Group: http://seanwilson.org/wittgenstein.discussion.html

Jeff Segal wrote in response to Paul's Finkelman's mail, "The predictive value is this: for the justices appointed since Warren, the editorial scores correlate at about .8 with the percentage of times the justices vote liberally."
--------------

First, for any given justice, flipping coins will predict that their score will be 50. So the question becomes how well these media-impression workshops that Jeff recreates improves upon this efficacy. This is called Proportional Reduction in Error (PRE). The PRE on the logit models do show improvement upon blind guessing at 50, but several things must be noted:

1. No one guesses in the blind. Whether these scores are worth their labor is a function of what other perception workshops would tell us. I bet that polling empirical scholars would be better than constructing something from editorials. No one who watches the scores would expect anything more than a 60-ish number anyway, especially when you consider what that number really is.

2. The scores only improve blind guessing (at 50) by about 24%. But if you take away the extremely- directional justices -- the ones no longer on the Court -- the number is 9%. (Subtracted: Rehnquist, Brennan, Marshall, Fortas, Douglas and Goldberg).

3. If you consider the whole docket, of course, all bets are off. You have a statistically-insignificant model from 1948-2004. (about 60,000 so-called "votes"). Model is logit. The PRE is terrible anyway.

4. A couple additional things:

People need to ask themselves to what extent the model really indulges metaphysics. Think about it. As a scientist, you know that the media-perception scores are only a form of prognostication. That's what Jeff has done. He's turned their content into a prediction for either a justice's state of mind or his or her work consequences for criminal-plus claimants.

But if journalists really knew this, the story would be one of clairvoyance or perhaps conspiracy (like insider trading). There is nothing in those editorials different from what, say, informed list members might believe about these things. If Jimmy the Greek predicted numbers well for six weeks in a row, would you go off and say that science was the cause, or that metaphysics (or corruption) was? I think luck would be the real cause. My point is there is nothing special about journalists feelings in this respect. Many of us could do better than a coin flip. There is no need to make either metaphysics or science out of this.

One last point. If Jeff's measures have any significance to anything, it probably is similar to the correlation that young children have in picking presidential elections. That's what it reminds me of. But there, what we say is that this is "carrier evidence." That it shows image perception at some base level of psychology. Here the mistake is not to ask the same question: why is it that a small media-perception work group constructed during the confirmation ritual has any relationship whatsoever to a yes-no tally of claimants winning in criminal-plus cases? The answer really only lies in this:

1. The 6 to 8 extreme justices that drive the results
2. It's an easy game. Pick from 35 to 45 for republicans; 55 to 65 for Democrats - and you'll do fine.

And, if you can find some sort of naturally-occurring process that generates numbers like this -- like media perceptions of a president's pick -- now you have something really neat. It makes the whole thing look automated.

Regards and thanks.

(P.S. -- Paul, see my paper if you want a technical overview of Jeff's model. It is on SSRN, below my signature)

Dr. Sean Wilson, Esq.
Assistant Professor
Wright State University
Redesigned Website: http://seanwilson.org
SSRN papers: http://ssrn.com/author=596860
Twitter: http://twitter.com/seanwilsonorg
Facebook: http://www.facebook.com/seanwilsonorg
New Discussion Group: http://seanwilson.org/wittgenstein.discussion.html

... just a few points on measurement error. (I had thought these ideas had finally left the discipline).

First, the only segment of the docket being talked about here are what these people call "civil liberties cases," which is roughly 1/2 of the Court's workload in the data set. When you look at this from 1948-2004, the breakdown of this HALF of the set is: criminal cases (40%); civil rights (30%), First Amendment (16%), Due Process (8%), Privacy (2%) , and Attorneys (2%). So, for simplicity, let's call this the "criminal-plus rights claimants."

When Jeff speaks of "measurement error," what he means is that the media-impression scores for any given justice are disappointing to him when looking at the overall tendency of a justice's craft to have favored criminal-plus rights claimants. Apparently, what he would like is for his media-impression workshops to be able to really nail the rate at which criminal-plus rights claimants win their cases.

There are several obvious problems here:

1. the stuff in the media editorials (that Jeff codes) are not confined to or centered around issues in criminal law and civil rights (70% of the docket concern). And when they are, it is usually just a discreet, hot-button thing. Hence, the content of the one measure has nothing to do with the issues justices actually end up considering.

2. Also, the coding philosophy used here is confused. It indulges the idea of political values existing as exemplar issues in American political psychology, stuffed into one-dimensional space. (Guns, butter, taxes, abortion, big-case controversies like firefighters, speeches about presidential power, affirmative action positions, etc.). Anything mentioned along these lines gets you "coded." I think Jeff even codes based upon whether the journalists uses the world "liberal" in the editorial. Let's call this the "stereotype picture."

The problem here is that when justices decide cases before them that involve criminal-plus rights claimants, the issues in the cases very rarely involve "stereotype politics." Many times, the issues are a real snooze and make only a technical point. Or its only a little extension here or a little take away there. And so, you have this disjuncture between the philosophy of "liberal" being conjured on the one hand (the stereotype) and the thing you want to call "liberal" on the other, but in good faith can't. (At least not without playing games with language).

3. What is curious about all of this is that the majority of justices for whom we have data do not have any real affinity for criminal-plus rights claimants one way or the other. Assuming most legal issues are tough, one would expect 40-60 to be the basic range. Of course, it wouldn't be during periods of innovation, where new rights paradigms emerge and then recede into an equilibrium. But even though we have this dynamic history in the data set, the majority of justices are really not that directional.

And it is this that causes the failure in Jeff's model, not "measurement error." Indeed, the only errors truly present in these models are specification errors (see points 1 and 2 above), errors with ecological inference (which I'll get to in a moment), and language games.

Really, if you think about it , Jeff's measures are lucky. He's got more measurement luck in the model than error. He's lucky that he has those 8 or so justices with high propensity to decide issues favoring criminal-plus rights claimants -- and those crazy scores of perfect liberal and conservatism. Without those 100% or 0% scores coming out of those media prejudice workshops that he recreates -- scores that attach themselves to justices with 80-20 propensities for criminal-plus cases -- there would be no model here at all.

In fact, just think about it. Use half the docket. Don't use all the justices. Get lucky on the rights revolution thing. Tell everyone you do better on the first 3 years of service (another cut). Then just cry measurement error for all the rest.

I think its worth noting that Segal-Cover scores are statistically insignificant and otherwise extremely paltry for the entire docket (every decision for which researchers have data). They are also statistically insignificant for discreet years of voting. I think one was in the early 1990s (I wrote a paper mentioning it). Also, if you take away those justices who are around the 80-20 mark and who are no longer on the Court -- in essence, replicating today's Court -- you don't have anything to speak of.

It isn't measurement error; it is that the whole idea is faulty.
Regards and thanks.

Dr. Sean Wilson, Esq.
Assistant Professor
Wright State University
Redesigned Website: http://seanwilson.org/
SSRN papers: http://ssrn.com/author=596860
Twitter: http://twitter.com/seanwilsonorg
Facebook: http://www.facebook.com/seanwilsonorg
New Discussion Group: http://seanwilson.org/wittgenstein.discussion.html

Mark:

Whatever may be anyone's private views, the prediction announced here is a function of a model Jeff uses. The logic of the model doesn't work that way you suggest . It doesn't offer predictions in the sense of prognostication. The output is solely a function of the input and the logic of the mathematical specification.

None of the variables account for the points you raise. The dependent variable coded by graduate students is rather "blind." If the decision favors a criminal defendant in a criminal case, for example, it gets thrown in the "liberal bin." If not, it gets the other one. The model doesn't consider what the substantive issue was, whether it shifted over time, whether it was mundane, whether it was big or small, whether the republican party actually supported it, whether anyone even cared, whether it instrumentally created "conservative doctrine" while disposing in favor of the defendant, and so forth. In short, there is no assessment of qualitative factors.

Furthermore, because about half of the docket is excluded from the prediction -- as are various justices from before the ascendancy of Earl Warren -- the prediction is apparently limited to civil liberties cases, and assumes (as all of these sorts of models do) that the past controls the future. (The past meaning only Warren Court forward).

A couple of important points to keep in mind. What drives these models statistically is the presence of justices who had rather extreme tendencies to have decided for or against civil liberties claimants during the Warren and anti-Warren periods of the Court. And more specifically, to have decided criminal cases, which comprise the bulk of the cases that are said to be "civil liberties." (You really could call it the criminal cases and remaining civil liberties docket if you wanted to). Hence, what drives the model are justices like Rehnquist who decided in favor of criminal-plus claimants about 20% of the time (roughly) and the big-time Warren justices, some of whom hit the 80% mark.

Today's justices are more around the 33-65 range -- excluding, I think, Thomas, who is the only one still in the 20s the last time I checked (a few years ago). [I quit doing this work for obvious reasons]. I think Scalia is around 29 or something. (Even he may have made it to 30, I don't know). Jeff's model indeed assumes that the old guard is still there when the prediction is made, because all that the model sees are a bunch of numbers in Stata.

Even so, you will note that the model only produces a 62. Why? Because most justices for which we have data are not that directional when it comes to deciding for or against the claimants. The non-directional justices clog the model.

What is interesting about this is that if Sotomayor does decide whatever civil liberties cases she does -- even if they are not as heavy in criminal cases or anything like the ones from the 60s and 70s -- it makes no real difference. If she comes out a wild 78 or 80 (like the good old days), you can say "the newspaper scores were right about her." But if she winds up at 60, you can say "the model was right." And if she is anything near this side of 50, the industry continues. Next time, the model simply shaves the prediction for someone like her to a 60 or 59 or something (shaving for the mistake). So long as Rehnquist and the Warren people are in the Stata machine, and so long as the docket is shaved, it can't lose. (Plus, take away the old justices).

So in conclusion, there is in fact nothing to the prediction that considers anything you raised.

Regards and thanks.

Dr. Sean Wilson, Esq.
Assistant Professor
Wright State University
Redesigned Website: http://seanwilson.org/
SSRN papers: http://ssrn.com/author=596860
Twitter: http://twitter.com/seanwilsonorg
Facebook: http://www.facebook.com/seanwilsonorg
New Discussion Group: http://seanwilson.org/wittgenstein.discussion.html

someone wrote: "Yes, according to my editorial scores, Sotomayor is the most liberal justice confirmed since Marshall, but remember, there have only been two Democrats confirmed since then, Ginsburg and Breyer."

... let's be clear. According to the "editorial sources," she is thought to have only the score value generated by that procedure. That's all the measure says. Scientifically, the scores do not measure her "ideology" (whatever that means).

And with respect to the announced prediction of what political science calls her "votes," we probably should note a few disclaimers:

1. If you include the whole docket and all of the justices for which there is data, the predictive relationship is extremely paltry.

2. Even if you cherry-pick the docket and the justices, whatever results you get are fundamentally driven by the few justices with extreme propensity for direction under the "liberal index" -- most of whom are no longer there. And even this predicts that Sotomayor will be closer to neutrality (50%) than her alleged reputation (78). And of course, you don't need any newspaper scores to guess that Sotomayor will be in the 60s -- the safe money already has her around 65. (Flipping coins puts her at 50. The PRE on honest logit models was never impressive with these scores).

3. One of the biggest problems these models have is their misleading conclusions. The dependent variable (the so-called "liberal index") is quite peculiar because it doesn't have any empirical or substantive relationship to true "liberal voting." It's just called that by people pretending to do the "science." After all, the great majority of the coding doesn't concern exemplar issues that make up the belief-spectrum in the political system. And it doesn't concern issues that appear in campaigns or the culture war and so forth. In fact, one has to have a great deal of ideology himself or herself in order to see or call this measure "liberal voting." You have to sort of think like a creation scientist would when they study the world. In fact, one might think of political scientists who try to catch "ideology" this way as being sort of "ideology-creation scientists."

When scientists study the external world, they develop rigid designators for the things in the world they have "pinned down." I have always found it extremely curious that in ideology-centered, quantitative political science, no one attempts to talk precisely and honestly about the empirical things they actually observe. If they would, they would find that Sotomayor has a 78 on what appears to be some sort of exemplar-conceived political-issue barometer by a small media/journalist workgroup acting within a discreet time in American politics. And that this thing, when combined with other such prejudice workshops, has some sort of relationship to what we might call a rather badly-conceived yes-no claimant-priority arrangement. (And only when dockets and justices are constructed).

But that's not the way it comes out. It's always comes out as "the ideology going in was measured by the scientists," and "the scientists confirmed that ideology was coming out on the other side." I mean, it reminds me of one who would say, "first they were baptized, and then they went to heaven."

Regards and thanks.

Dr. Sean Wilson, Esq.
Assistant Professor
Wright State University
Redesigned Website: http://seanwilson.org/
SSRN papers: http://ssrn.com/author=596860
Twitter: http://twitter.com/seanwilsonorg
Facebook: http://www.facebook.com/seanwilsonorg
New Discussion Group: http://seanwilson.org/wittgenstein.discussion.html

First, thanks so much for sharing this.

Could you tell us a little about the semantic assumptions in the naming of the scores? For example, if someone were to call them the "legal philosophy space scores," would they be wrong? Or, what if they called them something like the "relative casuistry differential" -- would that be off the mark? When you tell others that you have "common space ideology measured," you surely don't mean "conventional ideology," right? And there is, of course, no way for a judge to decide a case that doesn't result in having an "ideology score?" And, if we were to develop measures of this sort of thing for scholars when they make decisions as a group that require judgment -- even the grading of exams -- they, too, would have "common space ideology?"  

I think I know a little about these scores. I admit I haven't paid great attention to them, but I have paid slight to moderate. And as I remember perusing them a while back, I've always found it curious what political science means when it calls them "ideology scores" and why empirical researchers would adopt non-scientific vocabulary for work such as this. Why not actually call the scores by a scientifically jargonized name, as real science does when it studies something in the external world? You do agree, after all, that the only thing quantitative models actually observe in the external world are the indices themselves, not the things they say they are seeing (e.g., "ideology")?  It seems to me that, somewhere down the road, you all may want to develop a science for the creation of indeces like this that could result in a jargonized lexicon that spoke the language of science.

Because as long as you are out there saying you've got "ideology" empirically observed, you really are in danger of sounding like creation science. There is no place in the external world where "ideology" is; the word itself is a normative conclusion about the status of beliefs. It would be something similar to saying, "I've got their epistemology measured." Imagine someone saying, "I have their correctness measured."  "So and So has a correctness score of X." You could, of course, find things in the external world to measure that bear upon a debate about these things, but you really can't say you have the things measured, because they, themselves, are fundamentally accusations about about the normative content of beliefs.  

I really want to help political science become either real science, or -- better yet -- good philosophy.

Regards and thanks. 

Dr. Sean Wilson, Esq.
Assistant Professor
Wright State University
Redesigned Website: http://seanwilson.org/
SSRN papers: http://ssrn.com/author=596860
Twitter: http://twitter.com/seanwilsonorg
Facebook: http://www.facebook.com/seanwilsonorg

Larry:

On the distinction between knowing-that and knowing-how, and on what your club calls OLP, consider these expressions:

1. (a) I know how to drive to Dayton.  (b) I know how to breathe.

2. (a) I know that route 49 is the best route to Dayton.  (b) I know I have a hand.

The grammars of (a) all call upon the brain to perform two basic tasks: to remove doubt and to coronate this state of affairs. We might say the brain does something like this: FUNCTION(compare/contrast) + CORONATE(alternative). (We could quibble about the commands, but just keep them metaphorical for now). For Wittgenstein, language is only relevant for what it DOES. What it does is what it IS. And hence, the behavior of removing of doubt within the form of life is the only real purpose that "knowledge" serves. Knowledge is not a picture or an algebra; it is simply what one says to one another about the execution of a common mental task.

And so, once you put forth expressions (b), you break this grammar. And in so doing you create a language puzzle. The idea is first to confuse the brain with the expression. What is normally taken as a stipulation -- something self-evident or something not in need of doubt-removing in the form of life -- is now prescribed doubt-removing grammar. This leads to one of two courses of action. The  person either doesn't see the confusion and begins playing the game (which leads to brains-in-vats examples and a cereal-box sort of discussion). Or, they are insightful enough to sees the puzzle itself.

Those who cannot see the game end up trying to logic their way out of the situation. They want to prescribe "knowledge" as a rule or set of properties. As if you would actually go to a philosopher for answers like "so tell me if I knew professor" or "so tell me if the tree is real."    

Wittgenstein ended all of this. He realized that these were only forms of talking, and that philosophy could not prescribe these things. He realized that what expressions (a) were asking the brain to do could not be functionally imported into (b). He showed this not by formal logic, but by simply showing what would be the case if we accepted the new conditions of assertability. He showed that when one rearranges grammar in this way, they only create new forms of speaking that never actually do anything different than before (in terms of what the brain is asked to do). And that the whole point of words like "reality" "physical" "consciousness," etc., were merely language sets used to for helpful communicative brain tasks within the form of life, not "philosophical problems to be solved."

Importantly, we now get to OLP. I would not call W's views OLP. Wittgenstein is not proposing that "ordinary use" (as in common denominator) is what is key. What he is proposing is something more like "functional language philosophy" (FLP). The idea is words are only relevant for what brain tasks get performed. When one says "X is not real, it is in my head," one would want to know "what in god's name are you asking my brain to do to communicate with you?" ("what are your script instructions"). Until the claim is conjugated into a common frame of reference,  the sentence is pointless. It is the common-frame that is key, not for being ordinary but for being functional (behavioral). So your club has it wrong again. It isn't OLP, its FLP.

And it doesn't matter who says this premise or whether they are a general, Larry -- or even whether they are arrogant. What matters only whether you understand it and can receive it yourself as an idea. And whether, as such, you find it flawed or convincing.  

(Sigh). Regards. 

Dr. Sean Wilson, Esq. Assistant ProfessorWright State UniversityNew Website: http://seanwilson.orgDaily Visitors: http://seanwilson.org/homepagelucy.htmlSSRN papers: http://ssrn.com/author=596860

1. I'm glad you share some sympathies toward my orientation. You are not right, of course, if you mean to say that i cannot productively converse on this subject with you (or anyone). Indeed, my contributions would be quite well thought out, and I imagine both of our ideas about the subject would have benefited. I do understand your concerns, however. Talking with a Wittgensteinian can be a headache. 

2. I think the thing that you must understand from my perspective is that I am a philosopher who was trained both as a lawyer and as an empirical watcher of what this group of scholars call (quite colorfully) "judicial politics." Because I am imminently familiar with philosophy of law, philosophy of language, philosophy of science, Wittgenstein, and "the quantitative arts," it bothers me to no end to see the silliness emanating from powerful segments of one social club being repeated by, and incorporated into, another. I am now seeing more often in books here and there, lawyers starting to say things like "segal and spaeth proved this," "martin and quinn measure  ideology," "the attitudinal model says," etc., etc., without the faintest idea of what the proof or claims therein consist of, what is meant in the charge of "ideology," or how any of that is actually accounted for. Truthfully, what happens is this: a law scholar looks for
footnotes because they think it makes for better "scholarship." It doesn't. It makes only a club product.

Listen to me very carefully. What quantitative political science does is it constructs some index and then gives it a NAME rather than a rigid designator, as all other empirical sciences do. They also make-up the index as a CREATIVE act, not from a science of observation. So there is no real science to the instrumentation. All they ever have is a mathematicized rhetoric. They also don't even realize as a social club that science of this sort, if it were to exist, could only observe its index, not the thing they are calling it. And so, you go to these conferences and out they come. "I'm using this ideology measure because it helps me the best," "I'm using the other ideology measure." "I'm using all of them to make people happy." At one job talk where I used to work one person once said (no kidding), "here are the dots. If I was wrong, the dots would be over there." One of my favorite things to look for at one of these conferences is when someone
claims the "attitudinal model" to be an "elegant theory." This always alerts me to the fact that the scholar knows nothing about philosophy and is at the conference for the art of mixing math with rhetoric. (It also reminds me that the club has a "Hell's Angels" component to it).

Not only do none of these indicies actually measure "ideology," very often the speaker doesn't have a refined idea of what this concept entails among its competitors. Lacking requisite understanding in philosophy of law, language and science, the club, nonetheless, produces its books, and they pass this numerology on to one another like communion -- and before you know it, now the lawyers are saying it, too. Now its on Oyez. And then when one of the favorite rituals of these two clubs comes along -- "this one is the liberal one," "that one was active," "this one used his attitudes" "that one cheated because he liked the result" -- out comes the "science" into the carnival.

I mean, really. One ought to film the thing and put it in Seinfeld or something. 

Regards, Ted, and go Steelers.           
 
Dr. Sean Wilson, Esq.
Assistant Professor
Wright State University
New Website: http://seanwilson.org
Daily Visitors: http://seanwilson.org/homepagelucy.html
SSRN papers: http://ssrn.com/author=596860

Ted, why do these scores map "justice ideology" and what in creation are you talking about when you say that? Who's left and right. Can you actually be left one day and center the next (doesn't that change the sense of talking)?

Because it seems to me that the scores are a kind of casuistry differential index of only dispositional choices. And that they are quite different from what other indicies say, and that, to declare them "ideology," one has to adopt a rather peculiar vernacular. (To say nothing, of course, that, when mapping "casuistry space," any set of choices one makes gets mapped. There is no way for the justice to not have "ideology"). So, what relevance does this have to the "who's the real liberal" pie-throwing ritual that one so often sees? More importantly, what relevance does it have to ANY discussion in jurisprudence?

Here's what I am saying to you,  Ted, and to the rest of the lawyers who know where the web site is. If Martin Quinn scores estimate anything, it would more be in the neighborhood of a set of a mathematicized choice-differential evidencing, perhaps, a sort of IDEATION, not ideology. You might say of these scores, "oh look at justice x's ideating pattern relative to y on the issue of every disposition (affirm/reverse)."  (Of course, even this is problematic because there is no measure of ideation, there is only a set of scores generated from a set of choices. You have to infer from the choices that a ideating path of some sort accompanied the choice).  

So now, tell us how this plugs into the prior discussion, because I can't see it.

Regards.

Dr. Sean Wilson, Esq.
Assistant Professor
Wright State University
New Website: http://seanwilson.org
Daily Visitors: http://seanwilson.org/homepagelucy.html
SSRN papers: http://ssrn.com/author=596860

The "research" that you speak of showed no such thing. That is only social club lore. If you put something in a headline long enough, people just repeat it.

Really, as to what can properly be said about justices and ideology by political scientists, it is only that an opinion may CONSTITUTE ideology  -- not whether a brain follows it. Brains have no real choice, really, but to follow what they do. No political scientist, therefore, has come close to proving anything remarkable about jurisprudence or the ethical reality of supreme judging. Among the informed, the old debates live on as strong as before, the lead ideas not being quantitatively determined. We aren't a real science anyway; one should not expect this to be a data-driven enterprise. When people want to know what affects cancer, for example, that is predominately data driven. We wait for the latest news. But it is not so for the idea of whether justices use what you call "ideology." There is no true science here; only a language game and, I think, something fundamentally aesthetic.   

If there were to be movements in this "field," therefore, they would not come from empiricists. They would come from philosophy of bias. That's the real problem. That subject has never really materialized.

Kind regards and "yours in bloody Ludwig"    
 
Dr. Sean Wilson, Esq.
Assistant Professor
Wright State University
New Website: http://seanwilson.org
Daily Visitors: http://seanwilson.org/homepagelucy.html
SSRN papers: http://ssrn.com/author=596860

________________________________
From: Raymond Kessler
To: Margo Schlanger ; Sean Wilson ; LAWCOURT-L@tulane.edu
Cc: conlawprof@lists.ucla.edu
Sent: Sunday, January 25, 2009 4:26:08 PM
Subject: RE: Ginsburg & Stevens (WARNING! My response contains OFFENSIVEMATERIAL)

4.      I also would NOT describe either G or S as “screaming ideologues.”  Both occasionally seem able to set aside their ideologies. (They are not  better or worse than Scalia, Alito and Roberts.  Thomas may be a different story).  Although it is not without its critics, I think the research by Segal, Spaeth,  Eptstein, etc. has pretty much revealed that most of the justices, most of the time, vote their attitudes/politics/ideology, and/or are pursuing policy goals.  IMHO these researchers have revealed that the Emperor has no clothes.  All this talk about precedent and the other legal mumbo-jumbo is window-dressing.

Walter:

That isn't what I was saying, but I think your point is to demonstrate that the sentence is poorly written.

When a brain goes to the store, it deals in apples and oranges. "This" is [you name it]; that is [something else].  One who deploys with these framesets communicates all sorts of things. This is motion, this isn't. This is physical, this isn't. It's like the brain saying SET(frame) + (dichotomy). It's simply a cognitive function. It's called comparison-and-contrast reasoning. You can think of it as a cognitive module. Linguistically, the brain uses this module because it allows for all sorts of useful behavior. Useful not in the sense that it is fun, but because it gets us what we need in our interactions with other humans (as does motioning with the hand or whistling to get attention). All uses of language are a BEHAVIOR.

Now, when you say that consciousness is not physical, all that is happening is that you are not gesturing with the frame-dichotomy attribute. Your brain is doing something in the nature of SET(frame) + (lexicography). Indeed, one who does not have a counter-factual for something has not eliminated any real problems, all they have done is introduced a different way of talking about it into the lexicon. Nothing's non-physical? Fine. Now as to that strange thing I call the "mind's eye," I'll now just have to call it physical Type I compared to physical Type 2 -- which is all I was ever doing in the first place when my brain was deploying the comparison/contrast module. All you have done in this dispute is change the paint color in the room.

Let's do it this way. When someone says consciousness is "non-physical," the response has nothing to do with empiricism and everything with therapy. Your brain's job is simply to ascertain whether the deployment of a dichotomous frameset is appropriate to its context -- if the person is trying to shop, for example (in which case you confine the terms to the function of the communication). But if the person isn't deploying the frameset for that objective  -- if he or she means that the light of god is in their head and is proof of some sort of outer-worldly goodness, you simply treat the matter as a hypothesis and show him or her the latest journalism that your social club (science) has produced. The answer would be "this is where we are on that," not "let me teach you how to talk correctly."

And so the point is this: there's nothing for philosophers to debate. There might be everything for philosophically-minded scientists to conjecture regarding things they would like to test. But there is nothing that takes the form of an argument where the goal is to say "yes it is physical" and "no it is not." All of that is nothing but misunderstanding the language game.

One does not go to the store and say "look at the apple particles move" or "is tomato a fruit?" Is a scorpion a bug? Is the pope a bachelor? Is consciousness "physical?" Do computers "think?" Do animals "believe?"  All of these questions offer the same fallacy: they think that one can make an algebra out of language. Computers do what they do, and if, anthropologically, that comes to having a family resemblance to the way we deploy "think" in other contexts, we'll so christen it in the language game. It means nothing other than the family has another cousin. It means absolutely NOTHING else.

And so if I say to you that the pope is not a bachelor because he is ineligible to date, you misunderstand the rules of the language game when you say, "you are wrong because bachelors are an unmarried male." I was never asserting that.  One does not make a logic out of linguistics; one only makes a culture. Whether I can play "bachelor" this way in the game is a function only of whether my communication is successful and nothing else.

Regards.  

Dr. Sean Wilson, Esq.
Assistant Professor
Wright State University
New Website: http://seanwilson.org
SSRN papers: http://ssrn.com/author=596860

... I shouldn't do this. I haven't read virtually anything of these endless discussions. But what my eyes have been forced to encounter even during the second it takes to delete all of these mails each day has me wondering about something seemingly basic. Surely the point has been made before. (And if so, my apologies).

Let's imagine one day someone sees a desk. He says, "it's still," as in "not moving." But the physicist next to him says something to the effect that the particles are in motion, we just can't see it. And the next day, they have a similar discussion about whether consciousness is "physical," and whether machines "think."

The fallacy in all of these discussions is that the deployment of "motion, physical and think" operate as a STATUS, not as something that is ostentatious (meaning pointing out).  For the physicist, the unit of analysis for "motion" is a particle; for others the unit is what the eyes regularly sees (without scientific instruments). The former bestows "motion" to a brain calculus (motion is a rule); the latter bestows it to the brain's use of comparison and contrast. And so, the two are stung by the language game. They each toss around the same mark or noise -- "motion" -- without understanding that the term only NAMING its underlying thing. For neither would disagree that the desk is "still" in comparison to a race car (ostensibly still) or that it has particles that are in motion. There is absolutely nothing to disagree about.

So as to consciousness. If we find one day that consciousness has properties X that share a family resemblance to other deployments of "physical" in the language game, one might say, "it's physical in a sense" (albeit a technical one). But this does NOTHING to anyone else who chooses to deploy the status of "physical" to distinguish it from the ostensibly non-physical, as one does every day in the grocery store when separating apples from oranges.  It matters not the LEAST whether or not what heretofore has been unexplained in the "mind's eye" turns out to have an atomistic reduction, because it will still have the same distinguishing features that it had prior to this explanation. I mean, it might lead to inventions and so forth -- but it doesn't prove anyone wrong.  That's like saying of the discovery of dust mites that we should now say "there are bugs on me." If I say there are no bugs on me, and you say there are, and it turns out
that tiny dust mites exist -- who is actually right in this nonsense? It seems to me that neither is "wrong." (When I'm asked if I have bugs, I'm still going to say no, as I will describe the desk as "still," and self-awareness as "non-physical," because the deployment of these language sets have the same exact use as before). 

Now there is one caveat. Some people when describing consciousness as "non-physical" and when saying that "computers can't think"  might mean something explicitly metaphysical. They might be saying that spirits of a sort exist. Surely these claims do get refuted, as this is the affect science clearly has. But so long as I do not mean to say that my consciousness is of a spirit sort of thing -- as if to point to it with attribution -- it matters nothing what science discovers when I continue to say it is "non-physical," because I was never talking about that sort of "physical" in the first place.

What I mean is this: if consciousness is physical, than physicality becomes a lexicographic idea, which changes it. Same with motion. We still need semantic place-cards for the ostensibly non-physical because these language sets are useful to us. Just as it is useful to exclude dust mites when saying "no bugs on you." We still need, in short, a non-lexicographic idea.

Regards.

... one can only be a "median justice" if what you are counting is commensurable. Let's do it this way: assume a hypothetical Court with 9 members. The appeal is from a trial court sentence of 5 years for a first time marijuana user. In this hypothetical world, 5 justices vote to overturn the conviction, 4 vote not to.  So the good guys win. Let's say the rationalizations are as follows:

Justice-A: 5 years is too much for a first time marijuana user ("cruel punishment")
Justice-B: 4 years is too much for a first time marijuana user ("cruel punishment")
Justice-C: 3 years is too much for a first time marijuana user ("cruel punishment")
Justice-D: 2 years is too much for a first time marijuana user ("cruel punishment")
Justice-E: the jury was improperly instructed ("jury violation")

Justices-F: I don't count my views on severity because trial judges should have discretion in sentencing ("higher principle")    
Justice-G: I don't count my views on severity because trial judges should have discretion in sentencing ("higher principle")
Justice-H: I don't count my views on severity because trial judges should have discretion in sentencing ("higher principle")
Justice-I: I don't count my views on severity because trial judges should have discretion in sentencing ("higher principle")

One assumes that in this situation, there should be no precedent whatsoever, and that the rule should apply only to the parties. I had always thought this was the case and that Marks was not saying otherwise. I don't see how you can have a rule of law without five heads on board for something other than the outcome. 

The only people who are proximate to each other in this example are A-D. That is the only group for which you can apply median logic. The others are theoretically proximate to views only concerning the issue that they are expressing. (F through I may or may not agree with A through D's points). Of course, on the issue of whether the higher principle should apply here, I suppose F - I are proximate to A-D. But one could construct another hypothetical where they are not.

Proximity logic only works if you have a natural issue spectrum. If you have either/or legal rules and people applying different issues to the same controversy, you can throw out unidimentional logic. I have never understood, frankly, the logic of "counting" these things anyway.  
  
Regards.

Dr. Sean Wilson, Esq.
Assistant Professor
Wright State University
New Website: http://seanwilson.org
SSRN papers: http://ssrn.com/author=596860

I want to briefly discuss another kind of "attitude model" that is often appealed to by political science scholars.   It combines what its creators call “case facts” with measures of ideology into a multi-variate analysis of discreet areas of voting (e.g., search-and-seizure) (Segal and Spaeth 2002, 312-320, 324-326; Segal 1984). This model is called the “case-fact” model.

As others, however, have noted (Friedman 2006, 268), it is regrettable that the variables in these models are actually called “case facts.” In the case of Segal and Spaeth's (2002) search and seizure analysis, for example, the variables are probably better understood as circumstances that have been recognized by the Court to morally guide the process of intrusion. For example,  what is said to be a "case fact” is whether the search is “incident to arrest;” involving warrants, probable cause and warrant “exceptions;” and occurring in the house, car, person or business (314-318).  Obviously, the Court has created legal doctrine that specifically prescribes the propriety of police intrusion under of each of these searching circumstances. It really should not be surprising that a statistical model could be created that extracts searching-guideline criteria from doctrine announced in prior cases and then demonstrates that the criteria is a statistical predictor in the very cases where the doctrines were created or enforced. Simply labeling the  announced circumstantial criteria governing the propriety of a search as “facts,” and then claiming that justices vote in cases based upon "the facts," does not allow one to escape the circularity problem that such variables really measure a doctrinal construct and  “conclusions of law.” (Friedman, 268).

A more general problem is that “facts” themselves are rarely judged by appellate tribunals. It is the trial courts that judge the facts (innocent, guilty). That is the reason why trials frequently culminate in a document titled Findings of Fact. The appellate courts, on the other hand, harvest the already-judged facts into a prudential construct. To see this, consider once again the search and seizure example (318). Although Segal and Spaeth argue that what influences voting is the “fact” of where the search occurred (person, place, business, home, etc.), the reality is that the Court is organizing instances of the behavior of intrusion via comparison and contrast to form a Constitutional “meta-doctrine.” What emerges, then, is a prescriptive order about the propriety of intrusion that could probably be called the “territoriality theory" of Fourth Amendment jurisprudence. This theory says, quite simply, that your privacy rights are substantial in your home but not out in "public" (which is why the location of the search is a statistical predictor). Hence, what is being voted for is the construction of a theory of territoriality that governs a set of searching circumstances and that serves as a sort of a “meta-doctrine” for the whole area of Fourth Amendment jurisprudence. Segal and Spaeth, therefore, really do not have a true “case facts” model; they have a model that catches the residue or "particulars" of a doctrinal construct and then purports to predict the outcomes of searches in the very cases where the construct was created or enforced. The problem is as much about tautology as it is vocabulary.

So does that mean that judges never "judge facts?"  Of course not.  For a jurist to be truly a judger of facts, he or she must make a decision based upon an attribute of a case that is unrelated to legal doctrine.  For example, let us say that a trial judge has to decide how many bloody crime-scene pictures to allow into evidence under the prejudice-versus-probity rule. Because this rule is governed by the abuse-of-discretion standard, there is really not much "law" (as in rules or -- I would argue - meaningful standards) to dictate the decision. The judge is generally free to let in any amount -- or even differing amounts --  of photos he or she desires. Let's say that Judge Judy has two murder cases, A and B. If she admits extra gruesome crime photos for Case A because it is a high profile case involving someone from a wealthy neighborhood, but admits a lower number for Case B because it is a poor neighborhood, that would be an example of using attitudes to judge facts. It would be judging facts because there is no evidence of doctrine sanctioning the "neighborhood-value" theory for the admissibility of crime photos. The judgment is therefore made based upon facts that were never woven into a prudential construct. Neither Judge Judy nor her peers are obliged to follow this criterion when judging photos in the future.

How would an appellate court "judge facts?"  One way would be agenda access. If a state supreme court justice votes to place cases on the docket more often for campaign contributers than otherwise, that would be a ruling based upon facts unrelated to doctrine. The Supreme Court might engage in such behavior if it decided a case based upon a "concealed fact" -- i.e., a fact that was not processed into the explicit doctrine. Let's say that the Court strikes down a sodomy law using rational basis instead of strict scrutiny because, if it uses higher level review, it may have to strike down certain marriage laws as well. (Lawrence v. Texas).  If the rational basis law stays weak in all other contexts, the Court will have "judged a fact" unrelated to doctrine.    

So I guess the ultimate point is this:  when supreme court judges make decisions on the merits, they do not generally "judge facts." They process already-judged facts into a construct that explicitly defines the propriety of the activity in question (e.g., searching).  This is not to say that this process is or is not ideologically driven. For search and seizure, it may well be. It is only to say that Segal and Spaeth's "case facts" model suffers from the objection of tautology, vocabulary and theoretical design.  It cannot be relied upon to demonstrate that judging is mythical or driven primarily by political values.  And it doesn't even show the extent that the justices actually judge "facts" apart from "law" (doctrine). 

Today I want to continue the thought experiment that I began in the entry titled, “5.0: What if Segal/Cover Scores Were Perfect?” In that entry, I showed that if Segal/Cover scores were an autonomous set of preference assignments containing no measurement error, justices having extreme beliefs would have created a highly polarized, clan-driven Court in a world where only political values mattered and the constraints of a judging environment did not exist. I want to refer to that regression as the “autonomy model.” However, I did not consider what my hypothetical world would look like if Segal/Cover scores were, in truth, only a dependent set of preference assignments. How would an institutionally-contextual extremist vote if his or her values were already influenced by a pre-existing cognitive edifice and bargaining structure? To answer this question, I conduct two regression analyses below which I call “fixed-effects” regressions (or “dependency models.”).

Recall that my autonomy model required a one-to-one correspondence between scaled Segal/Cover scores and liberal percentages. (This was the assumption that was problematic for some and necessitated the present detour). The two dependency models I construct below shed this assumption in favor two others. The first dependency model assumes that that extremist justices can only have a range of liberal scores symmetrically matching the most extreme-rated justice in the real world (Goldberg). Because Goldberg reached 90%, I assign all justices with a +1 score a liberal rating of 90%, and all those having a -1 score a rating of 10%. The values are scaled accordingly by the simple formula .5 + (score*.4). I call this my “small dependency” model. In the second regression, I confine extreme-rated justices to career-liberal percentages of 20% and 80%. (You will note that this is an especially forgiving assumption inasmuch several justices in the real world have ratings above 80% -- Douglas, Fortas, Marshall and Goldberg). The values are scaled accordingly by the simple formula .5 + (score*.3). I call this regression my “large dependency” model.

The results for both regressions are found in the attached table. I want to discuss the small dependency model first. As one can plainly see, the small model still provides quite pleasing results (although it is not as perfect as the autonomy model). It has a likelihood-ratio R-squared of 0.2281 – a very good number for a bivariate ideology model – and it reduces the error of classifying votes by 48% (tau-p). It explains about 47% of the overall voting variance according to phi-p. The regression coefficient is also strong. The KDV indicates that as Segal/Cover scores go from -1 (perfect conservative) to +1 (perfect liberal), the discreet change in the predicted liberal rating is .955. Once again, that is almost a perfect overall relationship.[1] The only difference between the autonomy model and the small dependency model, then, is that goodness of fit has dropped slightly (from about 59%) and the KDV is barely lower (from about .999).

Now I examine the large dependency model. As the extreme-valued justices begin to be “squished” into the 80-20 parameter, the model begins to lose some of its potency. It has a likelihood-ratio R-squared of 0.1235 and reduces the error of classifying votes by 35% (tau-p). It explains about 35% of the overall voting variance according to the logic of phi-p. Note that the regression coefficient has lost some of its knock-out punch. The KDV indicates that as Segal/Cover scores go from -1 (perfect conservative) to +1 (perfect liberal), the discreet change in the predicted liberal rating is .707. Although this is still a very good number indeed, it is interesting to note that as justices become “squished” even within a model where scores perfectly match percentages, the coefficient loses its near-perfect relational quality.

Also, keep in mind that this model still assumes that Segal/Cover scores have a perfect correlation with liberal percentages under the assumption that the percentage is a function of .5 + (score*.3). If we were to take the values of the independent and dependent variables and plug them into an ecological regression, the R-squared would be a perfect 1.0. In every regression, in fact, where there is a perfect relationship between Segal/Cover scores and percentages, the value of the R-squared is always 1.0. I bring this up only to show you that the R-squared in an ecological regression is a deficient measure. It cannot discriminate between a model showing autonomy in justice values or a model showing either small or large dependency in those values. Note also that in all the hypothetical voting models so far, the coefficient is statistically significant. Hence, if justices really did vote as an autonomy model suggests – or as a small or large dependency model suggests – each time the result would be an ecological model with statistical significance at .000 and a perfect R-squared.

One final observation. Below are the classplots from four regressions I have recently discussed: (1) the autonomy model; (2) the small dependency model; (3) the large dependency model; and (4) the model that exists in reality. Recall that in reality newspaper reputation is not an especially good regression by any means. It has a likelihood-ratio R-squared of only 0.067 and explains about 24% of the overall voting variance. It’s KDV is also only 41%. The reason why each of these models is different can be clearly seen in the classplots below. In the autonomy model, the polarizing voting is creating an ideology model that belongs in "attitudinal" heaven. As the extreme-valued justices are squished inward, the models begin to lose their anchors. Also, as more and more justices exhibit non-directional voting patterns – as they begin to congregate around the 50% range – the model simply becomes “clogged.” Take a look yourself:

<--Autonomy Model   <--                                                                                                                Small  Dependency

<--Strong Dependency  <--                                                                                                                  Reality

[1] It is important to remember that the KDV reports a sum of all of the changes in the predicted Y as X increases from its minimum to maximum in 10% increments. The discreet change in Y for each 10% change in X is symmetrical but not equal. Some 10% changes in X produce larger values than others. Once again, the KDV simply is a sum of the changes.                      

To have a focused discussion of what this objection really says – and I believe it does say something revealing – it is important to have a clear understanding of what my hypothetical regression assumed. Yesterday’s model assumed a judicial world with the following four attributes: (a) only political attitudes governed judging (the “political assumption”); (b) Segal/Cover scores were perfectly accurate in capturing the directional propensity of those attitudes (“perfect measurement assumption”); (c) the dichotomous coding construct used by political scientists for the model’s dependent variable accurately captured the political choices of the justices (“perfect coding assumption”); and (d) the political attitudes of justices did not change over time (“stability assumption”). If all of this were true in a hypothetical world, why wouldn’t the absolutely biased justices always vote according to their label? (Remember that in the real world Goldberg voted 90% liberal).

Interestingly, I can think of only two answers to this question. The first comes from game theorists. Quite simply, strategy, coalition building and fear of sanction would cause defection from what is maximally desired in the short run in favor of obtaining optimal desires for the long run. In short, justices would occasionally cut their losses in order to obtain a better tomorrow. I will refer to this notion as the “policy game.” The second answer to the question is a little tricky. It says something that seems to violate “the political assumption” listed above, but is actually clever enough to avoid doing so. It says that when newspaper editorialists make claims of extreme political propensity, they simply do so under the assumption that the values being described in the editorials can only be expressed within a preexisting “judging context.” That is, when editorialists say that a nominee is "liberal" they probably assume that he or she will express a relative preference for liberal social policy within the context of the judging environment. Note that this does not say that there is measurement error in the scores; rather, it says that a score of -1 (perfect conservatism) or +1 (perfect liberalism) is simply an indication of a contextual extremity. Hence, that is why one cannot assume a one-to-one correspondence between scaled Segal-Cover scores and aggregate career voting even for an attitudinal model in heaven.

But if either of these options is true, something rather revealing has just occurred. Did you catch it? Because both the “policy game” and the argument-from-context purport to have their effect upon judicial votes during and as a result of the process of judging, Segal/Cover scores can no longer be theorized to be an autonomous measure of political values. Instead, they must be theorized to be dependent or contingent set of values. To see this, consider Antonin Scalia. Segal/Cover scores say, in effect, that there is no person on the planet who is more conservative that he is. (But in fact, given what I have just said, is it true that the scores say this after all?). If we viewed extreme scores as being a measure of autonomous values – scores unto themselves as they would be outside of an interdependent judging context – we would have to regard a one-to-one correspondence between values and percentages as being a plausible way to theorize attitudinal heaven (given assumptions (a) through (d)). But if we regard extremity as a relative and dependent phenomenon -- being capable of expression only within the pre-existing decision structure – then Segal/Cover scores are no longer a measure of something that precedes the judicial environment. Instead, they are simply an indirect and imperfect way of forecasting what the true career propensity for direction will eventually reveal.

Hence, what I am saying is that those who object to a one-to-one proportionality for an attitudinal model in heaven are actually (unknowingly?) conceding that their independent variable is making a value assignment that is expected only to manifest itself within the preexisting structural edifice and bargaining context of the Legal Complex. By conceding that this pre-existing environment exists, one concedes that the measure of “attitudes” is a dependent phenomenon. Stated another way, one cannot say that newspaper reputation is an unmolested look into the political souls of judges, yet object to a model where the evidence of those souls bears a one-to-one correspondence to career percentages in a world where the souls are King and everything is measured properly.

Now, what this really says, properly translated, is that the true indication of the dependent value system used by justices to decide cases within the framework of a legal and strategic environment is not Segal-Cover scores, but rather is the true aggregate tendency itself. That is, assuming that the coding of the dependent variable is not problematic and that propensity for political values is stable across time (assumptions (c) and (d)), it would seem logical to use career propensity for direction as the true proxy for justice values. However, you obviously could not use career numbers to forecast career numbers – tautologies in a non-Wittgensteinian sense are indeed the worst. But you could regress the liberal index against the votes to ascertain how well that index as a proxy for political values explains the choices of the justices. Here’s the headline: the more leptokurtic the distribution of liberal votes is, the less sexy the model will be in terms of goodness of fit; the more polarized the distribution, the hotter it looks. And although this conclusion is a “tautology” in a Wittgensteinian sense – i.e., it is axiomatic – it is nonetheless a meaningful assessment of how "politics" --  as that concept is observed and measured in a bivariate model -- explains judging choices.

Program note: In the near future, I am going to begin creating bivariate ideology models that use career ratings as a value proxy instead of Segal/Cover scores. But I am not going to do this right now because I am not done with my Segal-Spaeth critique. I have a few more things to show about the inadequacy of newspaper reputation before I move on. When I do change the independent variable, I will change the website topic from “Segal and Spaeth Critique” to simply “Bivariate Modeling Issues” and will begin seeing if repairs can be made to the problems that I have demonstrated in these models. One of the issues I hope to properly address is whether the dichotomous coding construct used by political scientists is truly problematic or not. (You will note I have been dancing around that one).   

To do this one must ask a central question: if justices voted exactly according to their newspaper reputation for political direction, what would that look like? One answer might be that a perfect match occurs when the propensity for bias found in the reputation has a one-to-one correspondence with the propensity for bias found in the actual votes. Hence, perfect prediction might simply be the scaled Segal/Cover scores.[1]  A justice having a Segal/Cover score of 0, therefore, would be expected to have a liberal output of 50% if newspaper reputation perfectly measured and predicted an aggregate directional tendency.[2]  (For those who object that perfect prediction would not exist using one-to-one proportionality, hold off for just a second).

So what would reality look like if Segal-Cover scores were perfectly accurate, using one-to-one proportionality? To answer this question, I have conducted a logistic regression of simulated data for all civil liberties votes cast for all justices from 1946-2004. (The data is uploaded on this website and can be accessed below). It was fairly easy to create the simulated votes. I simply changed the vote distribution of each justice in my real data set so that the proportion of liberal votes matched perfectly the scaled Segal/Cover score. For example, Justice White has 2,307 votes in the data set and his Segal/Cover score is 0 (scaled to .5); therefore, his vote distribution was changed to 1154 liberal, 1154 conservative.

The results of the regression are quite interesting and can be found in the following table. As one can plainly see, this regression is immaculate. They should put it in an attitudinal museum. The likelihood-ratio R-squared is .38, which is an excellent number for a bivariate ideology model.[3]  According to PRE (tau-p), knowing the newspaper reputation of a justice increases the ability to classify votes by 59%. Phi-p indicates that the overall voting variance accounted for by the model is about 59% as well, which is the level of explanation that Segal and Spaeth originally thought they had created with ecological regression. The regression coefficient also supports a rosy scenario. The KDV shows that when Segal-Cover Scores change from -1 (most conservative) to +1 (most liberal), the discreet change in the probability of obtaining liberal votes increases by .9999, a near perfect match.[4]  The only way this model can become better as far as goodness of fit is concerned is to eliminate the perfectly non-directional justices having “neutral” political reputation (White, Whittaker and Clark).

To see why this model performs so well, examine the classplot below. As one can plainly see, the model is superb simply because it is anchored with extreme values for the predicted Y, with relatively little obstruction coming from the middle-range values. In short, there are two distinct voting clans that dominate the model. Hence, if Segal-Cover scores were perfectly true (using a one-to-one correspondence), the reality that would exist would probably best be described with the following statement: “Rehnquist votes the way he does because he is extremely conservative; Marshall voted the way he did because he was extremely liberal.” (Segal and Spaeth, 2002, 86; 1993, 65).

There is still one disturbing objection, however: isn’t it ridiculous to have a hypothetical model where Justice Scalia never votes liberal? My answer at this point is to hedge a bit: this was only a hypothetical exercise. Because some may believe that a better model of perfect prediction should be something less than a one-to-one correspondence, I will adjust the numbers tonight and post a second analysis. I’ve got an idea of what to do.

Data: http://ludwig.squarespace.com/file-download/segal-cover.perfectpred.dta

[1] Segal-Cover scores “count” from -1 to +1, which is 200 increments. The liberal ratings “count” from 0 to 100, which is 100 increments. To scale the Segal-Cover scores, simply transform them by the formula 1 – (.5 - (score/2)). 

[2] So long, of course, as the propensity for political direction remained constant in the values of a justice throughout his or her term on the bench, and so long as the dichotomous vote-coding construct used by political scientists accurately captured its subject (a controversial proposition that I have not yet addressed).

[3]. It is extremely rare for a bivariate ideological model to achieve a value of R²L above .4. In the hundreds of bivariate regressions I have performed, I have never seen a value that high. Therefore, I would suggest that at the outset researchers involved in bivariate ideology models adopt a simple rule of thumb: R²L values between .2 and .4 are “quite good results” and values below .1 are a baseline for results that are “not so good.”

[4] The relationship between the discreet change in the probability of Y for each 10% increase in X does not appear to be linear, however. For some 10% changes in X, the discreet change in the probability of Y is more extreme than for other 10% increments. The KDV reported in the table is the sum of all of these changes.

As I have demonstrated previously, the R-squared in an ecological regression involving Segal/Cover scores and career liberal ratings in civil liberties cases from 1946-2004 is 0.41. Keep in mind that this figure is not the explained variance of the votes; it is the explained variance in the numbers comprising the liberal index. To equate the one with the other is an ecological fallacy. To see just how damaging this fallacy is, I have provided a table which looks closely at what this R-squared statistic is reporting. The table can be accessed here.

The table is useful for several reasons. First, it breaks down the explained and unexplained variance that occurs in the numbers comprising the liberal ratings. It also, however, breaks down the number of justice votes implicated by those percentage points. As one can plainly see, the number of votes that accompany the explained portion of the regression is only 12.5% of the total number of votes that comprise the entire regression (31,049). That means the regression is only able to rely upon 12.5% of the votes to explain 41% of the variance in the ratings.

Another thing that is interesting is how each justice is affecting (or driving) the R-squared. This can be located in the column to the far right, called ERL (Explained Ratings Load). This column is simply referring to the contribution that each justice is making to the R-squared statistic (the explained variance). The justices who are actually driving the statistic the most are the ones who carry the highest proportion of influence (“load”). Scalia is a good example; he accounts for over 6% of the R-squared by himself. Justice Rehnquist is right behind him.

But now, however, examine the column titled EVL (Explained Voting Load). This column shows the proportion of votes that are hiding behind each of these “rating loads.” This is simply the proportion of the 12.5% of the total votes to which each justice contributes. Here we find something else of interest. First, there are, as one might expect, some justices who “artificially” contribute to the R-squared by having “payoffs” in their percentage points that are not matched by their votes. Good examples of this are justices Goldberg, Fortas, Rutledge, Jackson and Thomas (to some extent). To see this better, examine the following graph (the red lines of those justices – the percents – are disproportionate to the grey lines, the votes). Also, of the 12.5% of the votes that are needed to drive the 41% of the ratings, the bulk of the work comes disproportionately from four key justices: Rehnquist, Blackmun, Brennan and Douglas.

I’ll have more to say about this in a couple of days. I’ve got to run now.

In my last entry, I showed that newspaper reputation for political direction did not constitute as significant or substantial of an explanation of voting behavior as many political scientists had suggested over the last sixteen (16) years. Although its performance may be perfectly acceptable to some, clearly, others in the discipline seem to wish it would be strong enough to explain the bulk of what the Court does in civil liberties cases (which it does not). In this entry I take up the issue of why ideology models do not perform better.

The answer is straight forward: there are simply too many justices who do not affiliate well with the binary outcome being analyzed by the model. If a logit model was a drain, centrists justices are the clog. And that means that “newspaper ideology models” are really nothing other than a partially-clogged piece of plumbing. To demonstrate this, examine the stepwise analysis (Table 5 in my SSRN paper). It begins with the logit results of Segal and Spaeth’s base ideology model, updated to 2004. It then subtracts the justice with a liberal rating closest to pure non-direction (.5) and re-estimates the regression. The subtraction continues one justice at a time until only those justices with ratings above 66.1 (Ginsburg) and below 33.9 remain. The subtraction increment is 16.1 points above or below 50%. As each median justice is subtracted, the values of the regression increase remarkably. At the very end of the regression, Justice Douglas is added. By this time, the only justices remaining in the analysis are the following ten: Brennan, Burger, Fortas, Goldberg, Marshall, Rehnquist, Scalia, Thomas, Warren and Douglas.

The table is simply amazing. The model has been transformed from “clogged plumbing” to Niagara Falls. It now has a KDV of 78 points and increases the ability classify the direction of votes by 57%. The total voting variance explained by the model is also 57% (phi-p). Or, stated another way, “Rehnquist votes the way he does because he is conservative; Marshall voted the way he did because he is liberal.” To really see the effect of “clan voting,” examine the classification plot below. It shows quite clearly why the model performs so well: there are no justices clustered around the 50% mark, and the model has two solid anchors on each side:

What does this show? It demonstrates is that one cannot create a bivariate ideology model that has the level of explanation that Segal and Spaeth originally believed they had created – a model that explains at least 60% of the Court’s choices – without first removing every justice from the truncated model having a liberal rating within 34% and 66.2% (and adding Douglas).[i] What this also says is that scholars who are championing the idea of an ideologically-driven Court are simply allowing the votes of those justices with the most obstinate judicial personalities to stereotype the majority of the institutional-membership’s voting behavior.

To see this, consider the model in Table 6 of my SSRN paper. It analyzes civil liberties voting from 1946-2004, but excludes 8 of the most directional justices having liberal ratings below 23% and over 77% (Rehnquist, Goldberg, Fortas, Douglas, Marshall, Brennan, Murphy and Warren). One of the reasons why excluding outliers is relevant, of course, is that the current Court no longer contains membership with career propensities beyond the values being excluded. Hence, one could argue that this analysis is a better estimation of the degree to which justice ideology governs votes on the current Court, [ii] at least to the extent researchers claim to have observed such phenomena in the Supreme Court data base.

The result of the subtraction is simply remarkable. The index variance explained by the regression drops to 17%. Total voting variance drops to 9% (phi-p), and PRE is only 11% (tau-p). The coefficient in the logit regression indicates that liberal ratings only increase by 8 discreet points as Segal/Cover scores change by 100% (KDV). The classplot in Table 3, Figure 6, speaks for itself. But what is perhaps more interesting is what happens if the subtraction range is increased by one percentage point in each direction (24% to 76%). It results in the exclusion of only two additional justices (Thomas and Rutledge) and produces a statistically-insignificant ecological model. [iii] It is indeed remarkable that an ecological model is completely unable to explain the liberal ratings of nearly two-thirds (22) of the Court’s membership since 1946, and that, whatever level of explanation it otherwise achieves is simply driven by a small minority of the Court’s most obstinate personalities.

[i] Some may be tempted to object that this manuscript sets up a test of the attitudinal model that requires justices to vote perfectly liberal or conservative before “attitudinalism” can prevail. This is not accurate. The manuscript merely requires that the career choices of the largely non-directional justices be similar to the directional before the bivariate model can be regarded as systemically dominant as proponents of ecological regression claimed. Indeed, what this manuscript demonstrates is only that what researchers empirically operationalized as “attitudinalism” simply plays a smaller role than previously thought. Finally, it should be remembered that there are three pathways to higher numbers in these models: (a) extreme justices voting with less dependency (see Figure 2 in Table 3); (b) middle justices voting like affiliated ones; or (c) some combination of the two.

[ii] Although the career propensities of Justices Alito and Roberts are not yet known, it is perhaps worth mentioning that both justices are predicted to be in the “moderately conservative” range. Alito’s newspaper reputation for political values was -0.8. His career liberalism is estimated to be 0.355 using a logit model and 0.343 using an ecological model. Justice Roberts’ newspaper reputation, by contrast, was -0.76, making his estimated liberalism 0.364 (logit) and .351 (ecological). However, it must be remembered that these forecasts are not remarkably accurate. The absolute value of the average mistake is 13 points for ecological regression and 13.1 for logit regression. Therefore, one cannot say that Alito or Roberts will not be especially directional. This caveat must be kept in mind when considering whether a regression that excludes outliers is a more accurate model of today’s Court.

[iii] There are other voting circumstances where Segal/Cover scores produce statistically insignificant results in civil liberties cases. Newspaper reputation is a statistically insignificant predictor (p values greater than .1) for every civil liberties vote cast by justices in the years of 1950, 1954, 1964, and 1992 (95% confidence interval, two tailed test). The p-values are also greater than .01 for the years 1949, 1951, 1952, 1953, 1965, 1968, 1991 and 1993 (Wilson 2006).

* substantial edit

I have just spent a week laying out an indictment against the bivariate ideology models that political scientists constructed over the last sixteen years. The indictment is predicated on a basic point: the aggregation of voting data and resulting misinterpretation of the R-squared statistic caused the creation of disciplinary misinformation – empirical falsehoods, plain and simple, that were passed along to political science graduate students and the rest of the academic community. Now is the time to correct these falsehoods by constructing a bivariate ideology model that avoids ecological inference and correctly estimates the relationship between model variables.

First, however, keep in mind a couple of things. I have yet to say anything about the propriety of the measures in these models. Some have said that the stuffing of justice opinions into one of two binary outcomes – liberal or conservative – or the fitting of justices’ preferences in unidimentional space is too simplistic to merit serious consideration. I will visit these issues at a later date. But for now, all that I want to do is find a more simplistic truth: how well do these measures actually perform when researchers model them properly and understand the results?

Below are the results of a logistic regression of civil liberties voting from 1946 to 2004. The regression contains 31,049 votes cast by 32 justices over 58 years of service on the high Court. The data is publicly available from the Ulmer project. [1] Let’s discuss goodness-of-fit first. The first indication that the fit of the model is poor is the rather low value (0.067) of the likelihood ratio R-squared. [2] The second indication that the fit is not that great comes from the PRE measures. Of the three that are listed, tau-p is the best for judicial modeling. [3] Tau-p indicates that the model only increases the ability classify liberal votes by 24%. Also, Phi-p, which is calculated using the logic of Pearson’s r, suggests that the overal variance between Segal-Cover scores and justice votes is about 24% The table can be accessed here.

Although analyzing the goodness of fit of a logit regression may not be as easy as an OLS regression, one of the nice things is that both now have the ability to generate a "picture." Take a look at the STATA “classplot” below. It shows why the fit of the model is not very satisfying. The reason is twofold: (1) too many justices are simply “non-directional” (median justices who do not affiliate well with dichotomous choices pull down the model’s fit); and (2) there is not enough “gusto” coming from each end of the value spectrum (no one is predicted to vote in the very extreme ranges of 0 to 29, or 80-100). In short, there is too much traffic around the value of 50% and not enough around 20 or 80. That’s why the numbers are poor.

Now let’s look at the coefficient. I like to focus on what I call the "key" discreet change or value (KDV). This statistic shows the total discreet amount that predicted ratings change as Segal-Cover scores change from their minimum to maximum values. Hence, going from absolute conservatism (-1) to absolute liberalism (+1) -- a 100% change – causes predicted liberal ratings to increase by 41 points, less than half the proportion of the change in values.[i]  Stated another way, newspaper reputation is less than half of the story, even using coefficient logic.  And although these results may be perfectly acceptable to some – they certainly seem to fit a Pritchett framework -- it is quite clear as an empirical matter that they do not: (a) explain the bulk of choices justices cast in civil liberties cases; or (b) establish the mythology of judging by showing the supremacy of political values.   Therefore, the ultimate point is that the reputation a justices obtains for political direction at the time of his or her confirmation does not explain nearly as much of the voting universe as the political scientists who originally constructed or endorsed these models proclaimed. This conclusion is not a matter of opinion; it is true as a simple fact of how data is interpreted and analyzed in a statistical model.

Because some scholars (Segal at al., 1995) believe that ideology models perform better when eliminating justices who predate the Warren Court, it is necessary to consider models that exclude Truman and Roosevelt appointees (the “truncated model”).  The findings are found in Table 4 of my SSRN paper.  It indicates that the truncated model is really no different from the model that contains all of the justices. The fit is poor (0.070 R²_L); the increase in the ability to classify liberal votes is moderate (24%, tau-p); and the total voting variance is about 24% (phi-p). The predicted magnitude of the variable relationship is also roughly equal to a model containing all of the justices (KDV = .418). The classplot below is virtually indistinguishable from the preceding one.  Although it is true that a higher proportion of index variance is present in the shorter list of ratings, only 14.6% of the model votes drive this effect (see Table 8 in my SSRN paper) [ii] and only five justices are responsible for half of it. [iii]  Therefore, the substantive conclusions drawn from a model of a truncated set of justices is really no different from the full model.

REFERENCES:

Menard, Scott. 2002. Applied Logistic Regression Analysis. Thousand Oaks: Sage Publications. 20-27.

Kleckla, W.R. (1980). Discriminant Analysis. Thousand Oaks: Sage Publications. 7-19.

DeMaris, Alfred. 1992. Logit Modeling, Practical Applications. Thousand Oaks: Sage Publications. 53-54.

[1] My particular data set is an integration of the Vinson Court data and the original Supreme Court data (updated through 2004). I transformed the data into a single “justice-centered” set with the help of Paul Collins.

[2] R2L is sometimes called “the McFadden R2.” According to Menard (2002), the statistic has the desirable properties of running from 0 (no fit) to 1 (perfect fit), is not affected by the proportion of cases in the sample having the attribute 0 or 1 (called the “base rate), and is not affected by the sample size of the data. However, it is important to remember that R2L is only an analogue to the OLS R2; the two statistics cannot be directly compared. Clearly, R2L underestimates goodness-of-fit when compared to OLS estimations of continuous-level data (DeMaris 1992, 53-54), and cannot be considered itself an explanation of overall voting variance (Menard, 20-24). It is extremely rare, moreover, for a bivariate ideology model to achieve a value of R2L above .4. In the hundreds of bivariate regressions I have performed, I have never seen a value that high. Therefore, I would suggest that researchers involved in bivariate ideology models adopt a simple rule of thumb: R2L values between .2 and .4 are “quite good results” and values below .1 are a baseline for results that are “not so good.”

[3] Based upon Menard’s (2002, 32-34, 36) reasoning, lambda appears inappropriate for a bivariate ideology model because it assumes that errors without the model take the form of an all-or-nothing guess (Menard, 29). In essence, lambda would only be helpful as a PRE measure if modelers could theoretically make the assumption that in the absence of any knowledge of their X variable, every justice in their sample of cases would vote unanimously in every case, the entire sample being all liberal or all conservative. Obviously, this does not appear to be a reasonable assumption.The measure that is best, therefore, for judicial politics scholars is Klecka’s (1980) index originally proposed for use in discriminant analysis models, generally referred to as “tau.” Following Menard’s terminology (2002, 32), I denote the term “tau” with a “p” – tau-p – to indicate its application to a 2 x 2 prediction table generated by a logit model. Tau-p is simply the best PRE statistic for judicial modelers because it assumes that the goal of the logit model is simply to classify as many liberal/conservative votes that are actually found in the base rate of the sample. Therefore, tau-p does not assume an all-or-nothing guessing scenario. It assumes that the number of liberal and conservative votes to be “guessed” in the absence of knowledge about the values of X is simply the proportion of liberal and conservative votes actually present in the sample. In this sense, Menard says that tau-p is less concerned with prediction logic and more concerned with classification logic. (29,33). Of course, like all PRE statistics, tau-p becomes problematic if data becomes excessively skewed.

[i] But how accurate are these predictions? Regressing the predicted liberal score of each justice against the actual ratings produces an R-squared of .4081, indicating that the logit predictions account for roughly 41% of the index variance. (Table 4 lists this statistic as “ppeR²,” which refers to “predicted probability R²”). Note that this is the same amount of index variance reduced in the ecological model. But note also how misleading this can be: As Table 8 shows, the ecological model only uses 12.5% of the Court’s votes to explain 41% of the index variance. And as both the logit model and Table 7 shows, there does not appear to be much overall political direction in the index in the first place.

[ii] Table 8 analyzes the explained and unexplained variance in the ecological model’s R-squared. As one can plainly see, the number of votes that accompany the explained portion of the regression is only 12.5% for the full model and 14.6% for the truncated model.

[iii] Note on Table 8 that only five extreme justices – Brennan, Fortas, Marshall, Scalia and Rehnquist – contribute 56% of the truncated model’s R². When compared to the full model, however, those same justices cannot carry such “loads” – they account for only 37% of the explained variance.

* substantial editing.

After having my head under the hood of Segal-and-Spaeth’s bivariate ideology model over the last week, I discovered something interesting: ecological regression does not effect how efficiently Segal/Cover scores predict aggregate liberalism on the Court. I had suspected the opposite. Although the two sets of predictors are quite similar and their overall difference is small, there does appear to be one interesting pattern: logit regression predicts liberal justices slightly better while ecological regression predicts conservative justices slightly better. Overall, ecological regression predicted 18 justices better; logit regression predicted 14 better. The best way to demonstrate these findings is by examining the following table and a graph. (Click bold words to access. Look at the output yourself. The graph really shows the story the best. A picture is worth a thousand words).

To explain the findings better, consider the Court’s two newest voters. Just how conservative will Alito and Roberts be? I’ve uploaded another table that analyzes the data.[1]  Alito’s newspaper reputation for political values was -0.8 (see Jeff Segal’s website). That means Alito’s career liberalism is estimated to be 0.355 using a logit model and 0.343 using an ecological model. Justice Roberts’ newspaper reputation, by contrast, was -0.76, making his estimated liberalism 0.364 (logit) and .351 (ecological). In short, both of the new justices are predicted to have a liberal tendency roughly equal to the aggregate tendencies of O’Connor, Kennedy and Powel (to say nothing of the policy differences that will comprise this tendency).

Now the big question: how accurate are these forecasts? The answer is: not incredibly. The average mistake that these forecasts generate is 13 points for ecological regression and 13.1 for logit regression. That means that, on average, Alito could be the next Rehnquist or the next (almost) Stewart. But at least we have a reasonable basis (before he takes the bench) for knowing that he is not the next Stevens or Brennan. Interestingly, if you regress the predictions generated by newspaper reputation against the reality that eventually emerges (the true aggregate liberalism), the R-squared is .4117 for ecological predictions and .4081 for logistic ones. Hence, ecological regression does not affect how well Segal/Cover scores can forecast predictions of an aggregated tendency. But in either case the quality of the forecast that emerges is “partly cloudy.”

[1] The table shows not only the difference between ecological and logit estimations, but also the difference between estimations based upon modeling decisions that exclude Truman and Roosevelt appointees versus those that do not. There is some controversy about whether these justices should be excluded. My view is that they should not be. I hope to author an entry about that point later.

* excised unnecessary final paragraph and changed the title.

In my last entry, I demonstrated that the bivariate ideology models constructed by judicial politics scholars over the last sixteen years had the unfortunate property of introducing ecological inference into the regression analysis. One may wonder why scholars did this to their models given the fact that there was no reason to do so (at least not since the mid 1990s). That is a topic for another day, however. For now, I want to consider a more direct question: what is “wrong” with relying upon ecological inference in a bivariate ideology model?

Although there are many problems with models that aggregate voting data, I want to focus upon one exclusive phenomenon in this entry: goodness of fit and model misspecification. I’ll hit the other problems in my next entry.

A. Goodness-of-Fit and Modeling Flaws

The best way to demonstrate the fit problem is with an example, followed by an interpretation. Let us assume that there are two hypothetical courts, Alpha and Beta, each with five justices who have the following voting data: 

The difference between these two courts is that one has a distribution of liberal votes that is symmetrical, the other is polarized. The Alpha Court is anchored by two extreme justices, followed by those of proximate distance and a centrist. The Beta Court, by contrast, is plagued with two extreme “clans.” In both cases, however, the hypothetical Segal-Cover scores are the same. That is, in the case of Alpha, the scores correlate perfectly with the percentage of liberal votes cast -- think of it as the attitudinal model in heaven -- but in the case of Beta, newspaper editorials were simply not as accurate of a forecast.

If one were to regress the Segal-Cover scores against the votes in each of these courts, what do you think the difference would be in the goodness-of-fit for an aggregated versus non-aggregated model? The answer may surprise you. In the logit model, goodness of fit is around .5 for the Alpha Court (symmetrical) and .8 for Beta (polarized).[1]  However, in the aggregated model, goodness of fit is a perfect 1.0 for the Alpha court and .47 for Beta.[2]  In other words, the fit of the models is moving in an opposite direction.

And now the critical question: how come the aggregated model cannot properly distinguish between symmetrical and polarized voting for purposes of fit like the non-aggregated model can? That is, how can a model of bias report that a voting universe is less explained by the presence of bias when it is dominated by two extreme clans – the “Rehnquist Five” logic – versus when it has symmetrical variety? The answer is straight forward: when data exists in a binary format, cases that do not affiliate well with dichotomous outcomes – those that show little to no favoritism for “0s” or “1s” – are interpreted as not fitting the model framework well. Hence, Justice Stewart on the Alpha Court is pulling down the model’s fit. However, in an OLS regression of continuous-level data, median cases of X only fail to fit their model if they have extreme Y values (are outliers). Hence, “Teddy” on the Beta Court is pulling down the fit of the OLS regression. (Picture a scatter plot: Teddy is the highest from the line. I have the data posted below if you want to play with it).

So what do we make of this? The point is that the only way a median justice will fail to fit very well into an ecological regression that is already anchored by two extremes is if he or she is so extremely directional (biased) as to be an outlier (Teddy). Yet, if that same voting pattern happens in the logit regression, goodness of fit would increase, not decrease, because the more movement one sees “out of the middle,” the better those models perform. Hence, what aggregation does is it transforms poorly fitting cases in one model into perfectly fitting cases in the other. Stated another way, it transforms non-directional justices into optimally-biased justices.

Given what I have just demonstrated, it should be quite clear why aggregating votes is fundamentally objectionable from the standpoint of both measurement logic and model specification. Quite simply, a model that transforms median justices who do not affiliate with an observed measure of bias into cases that actually “jack up” the model’s assessment of how well bias explains the voting universe is nothing other than a kind of sophistry masquerading behind statistical software. It is sophistry because: (a) bias is supposed to be an observed, empirical phenomenon, not a manufactured one; and (b) a model predicated upon the idea that a median-measured justice could lower the overall picture of bias in a voting universe by becoming an extremist is simply an invalid theoretical design.

Some might be tempted to argue, however, that non-directional justices have “moderate ideology,” and that this is their true “bias.” The argument would be that aggregation is good because median-measured justices should bolster fit unless they become extreme. The reply to this view is straight forward: Segal and Spaeth do not have a criteria for observing moderation as a political subject matter at the case level. That is exactly what the whole objection is. What is a vote for moderate ideology? To determine whether non-directional justices are, in fact, expressing preference for a political subject matter that is different from liberalism or conservatism, one would need to observe it with a trichotomous variable that provides acceptable coding criteria for the three distinct ideological choices. Or, one would need to create a continuous level measure of quality liberalism for each choice available to a justice (McGuire and Vanberg). To date, neither of these options have materialized. Even if they ever do, it is doubtful that such an innovation will help the fit of ideology models. The reason is that the justices who we think are liberal and conservative may “defect” quite regularly for centrist alternatives. If this happens with any regularity, the goodness of fit will not be as high as some in our field would like.

Therefore, transforming justices who systematically resist a measure of bias into perfectly-biased justices through the magic of aggregation is a most objectionable way to conduct empirical analysis of the data that is currently available. In short, these ecological models are misspecified.  No longer can political scientists assert as an empirical matter that 60-to-80% of the choices justices make in civil liberties cases arise out of their political values -- at least not to the extent that researchers have observed such phenomena in a data set. There is absolutely no empirical truth in that assertion whatsoever.

OUTPUT FOR ALPHA AND BETA: 

The STATA file:  http://ludwig.squarespace.com/storage/experiment.dta;  Goodness-of-fit tables for the logistic regressions: http://ludwig.squarespace.com/storage/table.alpha-beta.doc

REFERENCES:

McGuire, Kevin T., and George Vanberg. 2005. Mapping the Policies of the U.S. Supreme Court: Data, Opinions, and Constitutional Law, paper presented at the Annual Meeting of the American Political Science Association.  

[1] Logit is estimated with maximum likelihood. The only way to achieve a 1.0 (perfect) goodness of fit in a logit model is if the classification table perfectly predicts complete polarization. There would be no classification errors whatsoever.

[2] For those wanting more information, the logit classification table appears at the end of this journal entry. Fit is assessed with the R-squared analogues of phi-p and tau-p.

* corrected the spelling error in the title; minor editing in the final paragraph.

It is challenging to commence an organized analysis of the problems inherent in Segal and Spaeth’s Supreme Court decision-making literature. Indeed, one could enter this discussion from a number of areas. But rather than beginning where others have, I want to start with an entirely original point that is relevant to my own labors in this field: political science’s attempt to create a “bivariate ideology model.”

What is a bivariate ideology model? It is simply a mathematical model of decision making that estimates the relationship of a lone, single variable -- justice ideology -- upon the choices justices make in Supreme Court cases (votes on the merits). Five or ten years ago, it was common to find political scientists appealing to these models as “proof” of the primacy of politics over “law” and the exposure of a popular “mythology” surrounding Supreme Court judging.[1]  The creators of these models apparently still put forth these views.[2]  The truth, however, is that the bivariate ideology models created by political scientists never established the conclusions commonly attributed to them. For now, however, let us begin with an overview of the basic nature of these models as well as their history.

Any discussion of the history of the bivariate ideology model  in attitudinal literature must begin with Segal and Cover’s (1989) landmark article. It was this article that was said to provide the first systematic explanation of the voting behavior in civil liberties cases using an independent variable not derived from the justices’ votes (557). To accomplish this goal, attitudinal researchers created an empirical index called “Segal/Cover scores,” which were derived from the content of newspaper editorials appearing during justice confirmation hearings. The scholars coded editorials describing nominees as liberal or conservative and scaled the results, creating what in essence is a measure of each justice’s reputation for political bias at the time of his or her confirmation. Importantly, the scholars then decided to regress the scores not against the actual civil-liberties voting data that existed, but against a set of summary percentages derived from that data. In short, they regressed Segal/Cover scores against the justices’ percent-liberal ratings. The results of the regression showed a rather high correlation coefficient of 0.80 (561). Other scholars in 1995 joined in this example and created an updated analysis that again offered a robust correlation of 0.80 (Segal et. al., 1995).

Based upon these studies, many political scientists began concluding that the attitudinal model was now an empirically dominant explanation of justice voting behavior, and that justice ideology governed the bulk of the choices made in civil liberties cases.[3]  So popular did bivariate ecological regression become among judicial politics scholars that, even today, it continues to appear in the literature. Jeff Segal’s (2005) work, in fact, analyzes the Rehnquist Court with a bivariate ecological model that focuses exclusively on the 14 justices who served under William Rehnquist’s tenure as chief justice. He reports a correlation of 0.70 for civil liberties cases and 0.72 for the entire docket. There is also a bivariate ecological model that appears in Epstein, Knight and Martin’s recent work on civil rights voting (2004, 181; Figure 10.3). (Even the New York Times recently printed aggregated scatter plots).[4] Perhaps the best statement, however, of what political scientists thought they had proven with bivariate ecological regression can be found in Epstein and Knight’s (1998) work, which replicates the now-famous scatter plot and declares why it is relevant (35,36):

When it turned out that [Segal and Spaeth] could explain more than 60 percent of the variation in civil liberties votes based solely on the justices’ policy preferences, the researchers concluded that justices come to the bench with a set of policy preferences, which they pursue through their votes, at least in civil liberties cases.

Over the last five to ten years, many political scientists offered similar pronouncements.[5]  Of particular interest is Brisbin’s (1996) assessment of the evidence that appeared in political science’s top journal. It declared that the case in favor of Segal and Spaeth was so cogent that further study of the issue should actually cease (1004). It also gave to the APSR the following observation (1011):

If the fiction of a Court of law and not politics, like the tale of a fire breathing dragon, is now dead, why belabor it through further study? Perhaps it is because the dragon is dead, but like most dead reptiles, he is still twitching. So, for good measure, it is necessary to drive lances into him again and again and then draw and quarter him so that the heresy of a legal model of Supreme Court decision making cannot be regenerated.[6]

To understand why these bivariate ideology models are problematic, one must first understand the data that comprise them. The source is a large, publicly-available resource known as the Supreme Court Data Base, which contains voting and case data for every justice who served on the Court from 1946 through 2004.[7]  The format of the variable that observes the ideological choices of the justices is a simple binary entry coded with a "1" or “2” if the vote is liberal or not (Spaeth 1999, 69-72, 92). The total number of civil-liberties votes accounted for by this resources is over 31,049, covering 58 continuous years of Court activity by 32 justices. By aggregating this data into a handful of percentages and using the same as a dependent variable in a regression model, political scientists introduced ecological inference into their empirical analysis. This appears to have created confusion and exaggeration in the interpretation of model results. The truth is that Segal and Spaeth’s bivariate ideology model only accounts for about one-third of the level of explanation the researchers proclaimed. This is still a reasonable model, of course, but it is nowhere near the level of deconstruction many political scientists had proclaimed – and, in fact, supports a much more limited critique of the role that ideology plays in judging.  

[1]. Segal and Spaeth, The Attitudinal Model Revisited, 1, 8, 10 and 26-27.

[2]. Segal, Jeffrey. 2005. “The Rehnquist Court” Law & Courts. 15 (Spring): 14-17.

[3] Evidence of this is found in the following declarations: (1) “A prominent view, if not the prominent, view of U.S. Supreme Court decision making is the attitudinal model. It supposes that the ideological values of jurists provide the best predictors of their votes …” (Segal, et. al. 1995); (2). “[xx-get this quote]” (Peretti 1999, 105-111); (3) “Spaeth’s conclusion about the value of the attitudinal model is one echoed by many scholars of the judicial process, and not just those working in the area of decision making. ... Justices do not decide a priori to protect minority rights or to legitimate the ruling regime. Rather, they base their votes on their political ideologies, with a consequence being that liberal justices tend to protect minority interests, while conservative ones tend to legitimate the ruling regime” (Epstein 1995,  249-250); (4) “... attitude theory is still regarded by most judicial behavioralists as the most elegant and persuasive model for predicting appellate judge behavior” (Carp and Stidham 2002, 351); (5) “Among many political scientists, aspects of the attitudinal model have become a virtual truism” (Cross 1997, 251, 265); (6) “Today, few political scientists would dispute that, within their discipline, the leading approach to adjudication is the ‘attitudinal model,’ which hypothesizes that Supreme Court justices vote their political preferences or ideologies” (Feldman 2005, 89-90); and (7) “Indeed, these days it is difficult to argue credibly that the model utterly fails to perform its primary task. The evidence in support of its one observable implication – namely, that the policy preferences of the justices help predict their merits votes – is overwhelmingly in its favor” (Epstein 2003). See also, Gillman (2001, 465-466) (asserting that judicial behavioralist scholars believe that “law has almost no influence on the Justices” of the Supreme Court) and Brisbin (1996) (quoted, supra, p. 5).  But perhaps what says it best is Segal and Spaeth’s now famous (2002, 86; 1993, 65) summation of their research, “Rehnquist votes the way he does because he is extremely conservative; Marshall voted the way he did because he was extremely liberal.”

[4] See The New York Times, January 6, 2006.

[5] See Notes 12 and 15.

[6] The author continues later in the article: "If additional empirical analysis is coupled with a politically conscious interpretation of legal texts, judicial research could not just slay any claims for principled, legal models of Supreme Court decisions making, it could slay any prescriptive arguments that endeavor to separate legal decisions from politics. Using multiple levels of analysis deciphering the components of judicial attitudes, judicial scholars could in effect deconstruct any claim that American law is or can be a morally principled effort to write down the rules used to discipline political pathologies" (1014). (Although it is true that some of this exaggerated praise arose not only from the results of ecological regression, but from other models that Segal and Spaeth were producing, it is also true that if you take away the bivariate ecological model, not enough remains in the other models to support such an observation -- at least not empirically).

[7] There are several data sets available. See the Ulmer Project at: http://www.as.uky.edu/polisci/ulmerproject/

* copied paragraphs from manuscript version

Version 1.1*

Many scholars in the judicial politics world do not know this, so I thought I would make a quick note. During a paper I prepared for the 2006 Midwest meeting, I discovered something interesting: Segal/Cover scores on a few occasions actually generate statistically-insignificant parameter estimates for specific years of voting. Newspaper reputation is a statistically insignificant predictor for every civil liberties vote cast by justices in the years of 1950, 1954, 1964, and 1992 (95% confidence interval and a two tailed test). The p-values are also greater than .01 for the years 1949, 1951, 1952, 1953, 1965, 1968, 1991 and 1993.  The table below summarizes the results. The data comes from the Ulmer Project. It is a combination of Vinson Court and updated Supreme-Court data combined into a single, justice-centered data set using stata commands (available through Paul Collins and also the Law and Courts Newsletter). 

What would attitudinal modelers make of this? Would they say that judging was so good in 1992 that no vote was influenced by justice ideology? Was that a special, vintage year or something? Better than the '69 Mets? 

P-values for Segal/Cover Scores above .01 (civil liberties):

Year

P-value

Year

P-value

Year

P-Value

1949

.069

1953

.028

1968

.067

1950

.739

1954

.165

1991

.016

1951

.037

1964

.272

1992

.547

1952

.056

1965

.053

1993

.022

* The original version contained a spelling error in the title and a typo on the table. Both are corrected.