Seth Masket (University of Denver), in the course of generous words of praise, writes that he fears that variation in agenda control across states could undermine the comparability of ideal point estimates that I used to discuss the ideology of Dede Scozzafava, Republican candidate in a special election for the US House in the 23rd District of New York.

As a bit of introduction, in some states, agenda control is tighter, meaning that parties exercise tight control over what bills get introduced for a vote. In other states, agenda control is looser. For example, Seth’s home state of Colorado adopted an initiative in 1988 called GAVEL (Give A Vote to Every Legislator) which prevents party leaderships from suppressing bills in the early stages of lawmaking.

How could this be a problem? Well, roll call-based measures of ideology, of which our paper is one, relies on the public votes that are allowed to come to the floor for consideration. Thus it could be the case that we have a very “selective” roll call record that suppresses the true range of variation in ideology simply because some bills (typically the minority party’s) never get a vote in some states, but not others.

While variation in agenda control is very interesting, investigating it before now has been very difficult, because we don’t have legislator-level ideal point estimates. So having this problem should be considered a luxury…

But more broadly, existing evidence on Congress leads me to doubt that agenda control is that big of a problem for estimating ideal points. Remember that this was a debate about NOMINATE for a while (eg, Snyder 1992 and Rosenthal 1992, plus the simulation evidence in McCarty, Poole, and Rosenthal’s 2006 book), and these concerns havn’t really stopped the ideal point project.

The party line is that since agenda control isn’t perfect, and there’s always error in legislative voting, there should be enough cutpoints to differentiate between legislators. See Poole and Rosenthal on this.

As an additional bit of evidence, the common movement of ideal points between US House and Senate–despite very different agenda-setting institutions–implies that these institutions need not undermine our estimates too badly. Polarization looks like it’s rising in both chambers almost identically. But agenda control is far tighter in the House.

Finally, the roll call-based scores are “normed” by the Votesmart NPAT survey. It should be the case that, even if agenda control compresses the range of ideal points in the state alone, they are decompressed when considered in tandem with an external issue preference survey.

Still, it would behoove us to study these institutions far more carefully. I’m glad for people to use the data to do so, once we get the paper published!