On Quantitative Methods in Law and Politics

(In reply to Susan Lawrence, who writes, "Exactly what would a Wittgensteinian analysis of settlement rates; attorneys' fees in class actions; or the effects of defendants' criminal records on case outcomes look like?")

First, let me concede your point. I never meant to say Wittgensteinian method could help for these types of issues. The method is only concerned with dispelling confusions. (One of the ways in which confusions dispel themselves, by the way, is that they turn purely informational: one merely has to look and see).

But let me at least comment on what a philosophic-minded person might worry about with respect to studies you mention. Below are edited passages from a mail

I just wrote to Frank Cross (privately) concerning the statistical analysis of football. They apply equally to the statistical analysis of any human activity (settlement rates, fees, etc).

1. Methods are only helpful for things that humans cannot, themselves, see or experience. (E.g., what effect drug X has on certain kinds of lipids 24 hours after intake). The trouble happens when methods become superimposed on things we CAN experience (e.g., Who was a greater risk taker -- Bradshaw or Favre?). When we impose stats upon things we can, in theory, perceive by watching -- we lose information rather than gain it. The stats mislead. Example: as soon as you talk about a "quarterback rating," you mislead someone who neglected to see Roethlisberger in the AFC championship, and only heard about the low number. The person who watched it always has the advantage.

In fact, what social scientists really do is bend their analysis to suit what they think they are seeing in the first place -- that's how they "check their work." It only gets released if it can act as a good piece of journalism.

2. Stats work best with two companions: (a) things that are naturally commensurable (e.g., the economy and dollars); and (b) things naturally stochastic. Social studies tend to lack these things. You have to construct what you want to show. Same with applying stats to football and settlement analysis. This always compromises the picture.

3. Very frequently, the thing social studies want to show is not something that can easily be scientized. Look at the idea of "inflation" (troubling). Now compare that to "ideology" (much worse). Now compare it to whether a team has to run to win the Super Bowl. Or whether something affects settlement. The settlement question is no different than the Super Bowl. Same exact thing. Your answer yesterday is not the same as tomorrow. You can only provide a kind of journalism on the subject.

You have to ask yourself: what does a regression analysis of settlement rates tell you compared to what (say) a good NPR or 60 Minutes investigation might have? Or what 30 years of experience tells you? Or what a collection of people with major experience might say? What you must understand is that mathematics can only ever be a kind of sculpture when summoned into the service of social studies.

The stats are only good if we think the journalism is accurate. If we think the portrayal fits.

(Also, learning Wittgenstein can help students see this last point)