If I in fact did get it, it's pretty wild.'

Now I want them to go back through time and do it for the entire history of the US senate so I can see the markers shift over time, or if they shift.

It would be nice if it had more title to each vote than just 'roll call number n'.

I don't like that thought.

Following the various links, I found their description of what I feel is a basic issue with their work, how legislators and bills get classified as liberal or conservative in the first place. Each legislator has one or more scores (depending on whether or not you are using a one variable or two variable model) and each bill has the same. The analysis is mapping one set of values against another in a sophisticated way. But how are the legislators and bills scored for each of these values?

According to the paper above, what they did was assign what they thought was a reasonable value to each legislator, then used that value to work out a score for each bill (if more liberals voted for it, it was more liberal). Then these values for each bill are used to adjust the scores for each legislator, and so forth until a stable set of values results.

I wonder about that kind of approach. It may work well, and I simply do not understand the statistical safeguards in place. But I am slightly uneasy about the possibility of a algorithmic artifact in this case. Now, my statistics are limited to what a public administration graduate student is expected to know -- not bad but I am no mathematician, and the word "eigenvalue" is a null for me. Would someone with a bit more background in this kind of work comment on how reliable this kind of approach is, and how well these researchers used it?

But it's still pretty to look at if somebody tries (link is to Chris Lightfoot's excellent weblog). NB: the links at the bottom give you an exciting interactive experience exploring what our UK members are up to, which is more than you usually get from Parliament.

If you looked at enough bills with a sufficient range of ratings on the bills, then you could nail down the approximate threshold of each senator. Alice might be at -2, 3. meaning any bill labeled on one side of that point she would vote "Aye" and any bill on the other side she would vote "Nay".

if this is the approach, the only thing you need to do is rate the bills appropriately. If you fail to rate them consistently, then you won't find a threshold for individual senators, since they'll appear to vote for something on one side, and then vote against something else on that same side.

The accuracy of the statistics would be reflected in teh amount of "error" that they show on the right. i.e. when they draw a line, if the numbers are right, everyone will fall into the graph as predicted by previous bills. If there is a large "error", then (1) some prior bills used to determine the senator's coordinates wasn't assigned an accurate value, (2) the current bill wasn't assigned an accurate value, (3) free will and human inaccuracy caused the senator to vote outside their norm.

Given the error seems fairly small for many individual votes, I'd say they are on to something.

Is there a name for this sort of analysis, that isn't a mouthful of syllables?

* (or, King George the Mad, as I like to say)

If a particular senator were attempting to "straddle" the line, trying to appease to both sides of issues, they should show up more often than otehr senators as being flagged as an "error". Since teh senators coordinates don't move, a lot of errors would mean they switch sides often. Wonder if they have a list of senators in order of largest error to smallest.

Okay, they'd have to be braised. Too tough otherwise. My mind must be twisted by that repubican version of the NYT.

1. Expectation Maximization. The Wikipedia article has the density of, say... lead, here so don't start with that one. Some of the applications may be more suitable to give you a flavor of its capabilities than the general tutorials.

2. Support Vector Machines. Which is not a rallying cry for downtrodden parallel processors, but actually a statistical technique that was developed in the hard AI "machine learning" community.

Both these terms, as well as principal components analysis, are mentioned in a tutorial whose URL I cannot post because it triggers the "questionable content" filter. But search for it.

However, I haven't decided if this means that I now have to think of Hillary (and John Kerry) as liberals or if this analysis says that the senate, on the whole, is a rather conservative body (such that solid moderates end up ranked 23.5).

109 15011 71 CALIFOR D BOXER 27 485 0.944 1.000
109 15021 21 ILLINOI D DURBIN 23 502 0.954 2.000
109 49309 25 WISCONS D FEINGOLD 62 500 0.876 3.000
109 14230 31 IOWA D HARKIN 37 505 0.927 4.000
109 10808 3 MASSACH D KENNEDY ED 17 500 0.966 5.000

The midwest is more liberal that massa-f-ing-chusetts???

That's crazy talk.

But they also propose new legislation, which is the main source of their importance.

In particlular, I'm confused what algorithm they used to determine where the bills and legislators fell on the liberal/conservative axis. What distinguishes a 2.3 from a 2.5 rating?

Thalia (20) Anyone else notice that the supposed centrist McCain is the 4th most conservative voter on that rank ordering? More consistent than Frist and Santorum in voting conservatively.

When McCain first began getting a lot of buzz about his presidential campaign, I noticed that the media and punditry seemed to discuss him as if he were a liberal Republican, a throwback to Jacob Javits.

Later, they seemed shocked that he espoused conservative positions.

He was a self-described conservative all along, though.

I will admit that a lot of my early liking for McCain was based on his straightforward language. Then he got savaged by the Rove machine during the 2000 primaries. He was wounded and angry for a year or more, after which he went Stockholm Syndrome with the Bush administration. I don't know what the moral of the story is, except perhaps that Bush & Co. are better torturers than the North Vietnamese.

Maverick, my ass.

As it happens, I was recently referring in another thread to a book-length factor analysis of science fiction readers' preferences for various authors.

Greg, Massachusetts liberalism is of the noblesse oblige sort; Midwestern liberalism, when it emerges is usually populist.

I'm troubled by money dominating politics, but letting the federal government take over the pursestrings is not the answer. It allows people now in power to protect their allies and select their successors even moreso than they've been able to all along.

I wondered about something similar years ago while digging into some political science research. A couple of times I saw someone, often by how a variable was operationalized, take something that was a nominal value, ordinal at best, and treat it like a interval or ratio value. Once you managed to get numbers, you could do anything you wanted with it, no matter what the underlying original information was like. An arbitrary example could be calculating the standard deviation of the house numbers on your block. The operation can be carried out correctly, but does the number really mean anything useful?

(Math geek note: Yes I understand that there is some rather high powered criticism of Stevens' system of value typologies -- by Tukey, for example. But I think they still are useful, if not abused.)

As I said above, I can easily imagine that the conclusions drawn are correct. But this is a process where the researchers first "reasonably" score the senators, then use those numbers to score the bills, then turn around and use those numbers to re-score or adjust the scores of those same senators. How does their approach ensure that all of this is not simply dependent on the original "reasonable" (their term) scoring of the senators? If their approach does do that, it would help if someone could explain it to some of us denser types around here.

Go, Midwest! Senator Wellstone hogged the top of the liberal list, too. Though we've certainly been a contradictory lot. Remember who knocked out populist giant Robert La Follette, Jr. in the 1946 primary: Joseph McCarthy. And for Senator Harkin's home state to switch from blue to red in 2004 makes me slightly embarrassed to be from Iowa.

(And Senator Feinstein does indeed frequently sound like a moderate Republican, but Senator Boxer is pretty unequivocally a liberal.)

Just to clarify: "But this is a process where the researchers first "reasonably" score the senators, then use those numbers to score the bills, then turn around and use those numbers to re-score or adjust the scores of those same senators."

One of the things that blew me away about this approach was that they didn't start from a "reasonably" scoring. If you tunnel back to the longer older posting I wrote about this I explain that they started in a time when computers were very weak and they wanted to fit a model to the voting data. The simplest model they could imagine was every senator would be a point on a line and every vote would split the line at some point. They then attempted to compute a best fit for all the senators (assuming they don't move) and every vote. Amazingly it worked and it's output was the reductionist ranking of every senator and a position for each bill. Later they were able to try the same trick in with more dimensions.

It actually took them a while to figure out what qualitative measures the dimensions were reflecting since it's often hard to puzzle out why a bill is actually about a social issue. For example one of the votes shown in that drawing was about the time limits on the guest worker proposal, it was a pure social vote, but it's not obvious to me why.

They have data now for many legislative bodies over very long periods. The book on voteview is just mind boggling.

They can also show that we have grown very polarized over the last few decades as the Republican legislators (though not the public) have shifted to the right. Their new book on that is just depressing.

I don't think I understand these charts, but I did notice a couple things:

1. On each chart, the D's and R's are usually in two separate groups. Is that what gives the impression that there's a difference between Democrats and Republicans?

2. Even if the vote was close to unanimous, the D's and R's are still shown in two separate groups. Take a look at:

Number: 226 (Session 2)
Yeas-Nays: 96-1

Number: 230 (Session 2)
Yeas-Nays: 93-5

Why is that?

I can well believe that there is a difference in voting patterns between D's and R's, but the two groups in each chart don't appear to reflect the voting patterns for each question.

Sure, at least in some communities the number arrived at is a useful at least preliminary proxy for housing density against road density (how long are the blocks and how far apart are the house numbers) derived by a few key strokes from published data. Granted it's more useful for being quick and dirty from data already avaible in digitized form. Examples and details on request - nod to JVP.

For a first approximation - I may well be completely off base pending detailed analysis and an explanation of hypothesis, algorithm, model and process not completely in evidence (just maybe there is some confusion of hypothesis and model? or maybe economics and political science just uses the words differently?) - I see an interesting view of the data but as I read the hypothesis to be Republicans favor legislation that Republicans favor and Democrats favor legislation that Democrats favor I don't see that the graphics justify any further conclusion. That is I don't see more here than the tabular rankings or gradings of legislators in which a given special interest group grades or ranks legislators according to the given special interest. That is I suggest special interests could be divided Republican or Democrat by how the special interest ranks legislators - or legislators could be divided Republican or Democrat by how they are ranked by different special interests but the information is circular.

Very good pictures and in that sense useful just as a Power Point presentation may be more useful than plain text - there is a little bit of Temple Grandin in all of us and a picture is often worth many words.

Just the same on the axis I count most John McCain and Larry Craig are far apart - see also Craig from (formerly - the water is going to Ketchum/Sun Valley mansions instead of to farmers)) agricultural Idaho on immigration/guest workers or on my pet issues.

Laurence, no 'usually' about it, what you're seeing is a static feature of the graphs: look closely, it's the exact same two groups each time. So you can't infer from the unchanging position of those two groups that there's something fishy about the impression of difference.

D and R signify party; red and blue signify how they voted in that vote; the straight black line is a best straight-line fit drawn through the space; "errors" are reds on the blue side of the line, or blues on the red side of the line; the space is an unchanging backdrop against which to display the lines for each vote,

Each line is designed to minimize the errors in that space; the space is designed to minimize the sum of the errors incurred by all the lines; the existence of the clusters in that one space is the clue that there really is a difference between Rs and Ds. The vertical nature of the gap between them is the clue that that difference is about economic interest groups, not about Social Issues.

I'd be interested to see how many of the remaining errors can be eliminated if the black lines are permitted to be parabolas, or some other dog-legged function, and not constrained to be straight lines.

I just had the weirdest thought that this could somehow be applied to wikipedia.

You could use edit-histories as a source of yes/no "answers". putting text in is a "yes" to that text. taking text out is a "no" to that text. A single edit might have a number of yes/no answers built into it as text is removed and replaced with new text.

Within an article, this should be automatically extractable into some form of relative coordinate system. With the result being a graph showing the location of editors, the moderates towards the middle, and the extremists towards the edges.

Now if I only had an infinite amount of time and an infinite number of monkeys, I could code this up as a program and ship it.