What Are Segal/Cover Scores Measuring Anyway?

In my last journal entry, I discussed the topic of what reality would look like if Segal/Cover scores perfectly predicted an aggregate voting tendency. Instead of matching the scores to the percentages, I was interested in matching the percentages to the scores. So I constructed a model that was based upon a one-to-one correspondence between scaled Segal/Cover scores and resulting liberal percentages. The model showed, in essence, that the Segal/Cover index is an especially directional set of preference assignments that, if taken “literally,” contemplated a strongly polarized Court driven by “clan voting.” A serious objection to the model was mounted, however. Although it is true that the scores of some justices indicate extreme directional propensity in their political views, it is simply ridiculous to assume that justices having perfect conservative or liberal reputations would never cast a vote contrary to their assigned label in all the civil liberties cases decided during their career. And this is true, the objection said, even in a hypothetical world where only “political attitudes” mattered and no measurement error existed on either side of the regression. Today I want to deal with the implications of this objection.

To have a focused discussion of what this objection really says – and I believe it does say something revealing – it is important to have a clear understanding of what my hypothetical regression assumed. Yesterday’s model assumed a judicial world with the following four attributes: (a) only political attitudes governed judging (the “political assumption”); (b) Segal/Cover scores were perfectly accurate in capturing the directional propensity of those attitudes (“perfect measurement assumption”); (c) the dichotomous coding construct used by political scientists for the model’s dependent variable accurately captured the political choices of the justices (“perfect coding assumption”); and (d) the political attitudes of justices did not change over time (“stability assumption”). If all of this were true in a hypothetical world, why wouldn’t the absolutely biased justices always vote according to their label? (Remember that in the real world Goldberg voted 90% liberal).

Interestingly, I can think of only two answers to this question. The first comes from game theorists. Quite simply, strategy, coalition building and fear of sanction would cause defection from what is maximally desired in the short run in favor of obtaining optimal desires for the long run. In short, justices would occasionally cut their losses in order to obtain a better tomorrow. I will refer to this notion as the “policy game.” The second answer to the question is a little tricky. It says something that seems to violate “the political assumption” listed above, but is actually clever enough to avoid doing so. It says that when newspaper editorialists make claims of extreme political propensity, they simply do so under the assumption that the values being described in the editorials can only be expressed within a preexisting “judging context.” That is, when editorialists say that a nominee is "liberal" they probably assume that he or she will express a relative preference for liberal social policy within the context of the judging environment. Note that this does not say that there is measurement error in the scores; rather, it says that a score of -1 (perfect conservatism) or +1 (perfect liberalism) is simply an indication of a contextual extremity. Hence, that is why one cannot assume a one-to-one correspondence between scaled Segal-Cover scores and aggregate career voting even for an attitudinal model in heaven.

But if either of these options is true, something rather revealing has just occurred. Did you catch it? Because both the “policy game” and the argument-from-context purport to have their effect upon judicial votes during and as a result of the process of judging, Segal/Cover scores can no longer be theorized to be an autonomous measure of political values. Instead, they must be theorized to be dependent or contingent set of values. To see this, consider Antonin Scalia. Segal/Cover scores say, in effect, that there is no person on the planet who is more conservative that he is. (But in fact, given what I have just said, is it true that the scores say this after all?). If we viewed extreme scores as being a measure of autonomous values – scores unto themselves as they would be outside of an interdependent judging context – we would have to regard a one-to-one correspondence between values and percentages as being a plausible way to theorize attitudinal heaven (given assumptions (a) through (d)). But if we regard extremity as a relative and dependent phenomenon -- being capable of expression only within the pre-existing decision structure – then Segal/Cover scores are no longer a measure of something that precedes the judicial environment. Instead, they are simply an indirect and imperfect way of forecasting what the true career propensity for direction will eventually reveal.

Hence, what I am saying is that those who object to a one-to-one proportionality for an attitudinal model in heaven are actually (unknowingly?) conceding that their independent variable is making a value assignment that is expected only to manifest itself within the preexisting structural edifice and bargaining context of the Legal Complex. By conceding that this pre-existing environment exists, one concedes that the measure of “attitudes” is a dependent phenomenon. Stated another way, one cannot say that newspaper reputation is an unmolested look into the political souls of judges, yet object to a model where the evidence of those souls bears a one-to-one correspondence to career percentages in a world where the souls are King and everything is measured properly.

Now, what this really says, properly translated, is that the true indication of the dependent value system used by justices to decide cases within the framework of a legal and strategic environment is not Segal-Cover scores, but rather is the true aggregate tendency itself. That is, assuming that the coding of the dependent variable is not problematic and that propensity for political values is stable across time (assumptions (c) and (d)), it would seem logical to use career propensity for direction as the true proxy for justice values. However, you obviously could not use career numbers to forecast career numbers – tautologies in a non-Wittgensteinian sense are indeed the worst. But you could regress the liberal index against the votes to ascertain how well that index as a proxy for political values explains the choices of the justices. Here’s the headline: the more leptokurtic the distribution of liberal votes is, the less sexy the model will be in terms of goodness of fit; the more polarized the distribution, the hotter it looks. And although this conclusion is a “tautology” in a Wittgensteinian sense – i.e., it is axiomatic – it is nonetheless a meaningful assessment of how "politics" --  as that concept is observed and measured in a bivariate model -- explains judging choices.

Program note: In the near future, I am going to begin creating bivariate ideology models that use career ratings as a value proxy instead of Segal/Cover scores. But I am not going to do this right now because I am not done with my Segal-Spaeth critique. I have a few more things to show about the inadequacy of newspaper reputation before I move on. When I do change the independent variable, I will change the website topic from “Segal and Spaeth Critique” to simply “Bivariate Modeling Issues” and will begin seeing if repairs can be made to the problems that I have demonstrated in these models. One of the issues I hope to properly address is whether the dichotomous coding construct used by political scientists is truly problematic or not. (You will note I have been dancing around that one).