Tester's up by 5--no, wait a minute--Rehberg's up by 10, ARGH! Making Sense of Conflicting Polls

Within 48 hours, we have seen two different firms release conflicting polls in the Montana Senate race. On May 1, Public Policy Polling (PPP)--a Democratic polling company--released a poll indicating that Jon Tester maintained a five point lead over Congressman Rehberg, 48-43. The lead was just within the margin of error of 3.2%. Nate Silver at The New York Times has analyzed polling firms and their biases. He actually found that PPP's polls lean slightly toward Republicans.

On May 3, Rasmussen Reports--a polling firm that generally has a pro-Republican tilt in its polling--released another poll with completely different results. This poll shows a ten point lead for Congressman Rehberg--53-43--the largest lead we've seen in this race to date. This lead however is also just inside the margin of error, which was 5 percent in this poll.

How do we make sense of these conflicting polls?

Let's start with a couple of fundamental points. First, the PPP poll was conducted between April 26-29 and included 934 Montana voters. The Rasmussen poll was conducted on May 2 and included only 450 likely Montana voters. That's why we see the different margin of errors--the larger the sample size, the lower the margin of error.

Second, let's talk about the margin of error and what that means. In the PPP poll, Senator Tester's support could range between 51.2 and 44.8. Congressman Rehberg's support could be as high as 46.2 and as low as 39.8 percent. In the second poll by Rasmussen, Senator Tester and Congressman Rehberg could both be tied at 48 percent. In either case, the leads by both are within the margin of error--so the results are not quite as out of line as one might expect just by looking at the head to head matchups reported by the polling firms.

But let's dig a bit deeper. One of the hardest things to figure out in the polling world is who will actually show up to vote. Forecasting turnout is about as hard as forecasting the weather because there are so many variables at work and the instruments we use to measure intent are subject to social desirability biases. If you ask a person if they intend to vote, most likely will give you the socially desirable answer: "Sure, I plan to vote". The problem is about 80 to 85% of voters will answer yes--and we know that turnout generally hovers between 50 and 60 percent. In other words, a bunch of folks who say they will vote simply don't.

Pollsters have lots of ways to measure turnout, and the differences in measuring turnout can have consequences for the final polling results that are reported.

How does Rasmusen and PPP differ in their turnout screeing questions? According to an e-mail exchange I had this morning with Tom Jensen at PPP, his organization calls folks who have voted in one of the last three general elections.Rasmussen, however, polls "likely voters". What's a likely voter? Rasmussen asks several screening questions, including the respondent's voting history, their interest in the election, and their likely voting intentions. This is a much more vigorous screening process designed to weed out folks who may not actually show up on election day.

PPP's process likely yields a "liberal" definition of turnout and Rasmussen's a "conservative". I use those quotes deliberately. PPP process might include folks in the sample who are less committed to voting than the Rasmussen poll. Demographically, Democrats usually have the turnout deck stacked against them relative to Republicans. Folks who are poorer, less educated, and not white are less likely to vote than those who are richer, more educated, and white. In short, those who are more likely to vote for Democrats are also those who are less likely to vote.

In short, the difference in results MIGHT be a factor of how each polling firm choses to define a voter. And there is no one "right" way. A generous interpretation of these conflicting polls is the higher the turnout on election day, the better chance Senator Tester has at getting reelected. Lower turnout, on the other hand, will likely benefit Congressman Rehberg.

Final take away message: Read polls carefully, examine the methodology section thoroughly, and go beyond the first page of the press release if you really want to understand why polls conflict.

Other Important Notes:


Both Senator Tester and Crossroads GPS launched some new ads in the past week. Senator Tester's ad, a nice positive bio spot about is propensity to fly Montana meat to DC with him on the plane, is right below.



The Crossroads GPS ad is below. It covers no new ground, focusing on Tester's votes on healthcare, cap and trade, and the federal budget--again, mentioning that Senator Tester voted 97% of the time with President Obama (see my previous analysis on voting here).

Senator Tester's Ad: Not Terribly Early

In yesterday's post, I speculated about the timing of Senator Tester's first campaign ad. Was it too early? Too late?

My colleagues at the Wesleyan Media Project--run by Erika Fowler, Mike Franz, and Travis Ridout (all Wisconsin grads, I might add)--provided some data to help answer that question. In 2010, the first election of the post-Citizens United era, four senators running for reelection faced competitive races: Senators Feingold, Reid, Murray, and Bennet. Feingold lost reelection, and the other three won with less than 53% of the vote. Here's when each launched their first ad:

Bennet: March 17

Feingold: April 27

Reid: April 30

Murray: July 28

Tester's first ad launched on March 26. I'd say given that he's a first-term senator running for reelection, his situation is most analogous to Bennet's--who was appointed to a vacant seat and running his first campaign. Murray and Feingold were both elected in 1992, Reid was elected in 1986--so well-established reputations. Four data points are not much to draw any firm conclusions, but it would seem that Tester's ad launch is not particularly early given the competitive nature of the seat and the fact that he's running for his second term. It seems just right.

Tester Up First: Tester Campaign Launches its First Ad

This morning, Montana's airwaves--which have already been inundated by advertising sponsored by outside groups variously attacking Senator Tester or Congressman Rehberg--received yet another campaign ad. This one, however, is different. First, it's actually sponsored by one of the candidates: Senator Tester. Second, it is a positive ad. Watch it below.




A couple of quick points. First, campaigns generally begin with positive, bio spots introducing (or re-introducing as is the case here) the candidate. After some time making people feel good about the candidate, campaigns move into the argument phase, where the candidate makes arguments about why they are the best choice and the opponent is the worst choice. Finally, in the closing weeks, we'll probably see some talking head ads with the candidate making their final pitch and asking citizens for their vote. As expected, the Tester campaign is proceeding as one might expect and anticipate.

Second, according to Mike Dennison's article in the Billings Gazette this morning (read here), the buy is $60,000. One of my sources tells me the buy is actually $53,000--and not a statewide buy at 1,000 Gross Ratings Points as I initially wrote. This buy is only 500 GRP in Billings, and 250 in other markets. So, the buy is not as big and extensive as I had been led to believe. Of course, one might argue that to have an effect--given the Internet and other ways to push ads--that the initial buy might not have to be as great.

Third, the positive ad is long on biography and short on issues. Again, not unusual. That's not what this ad is really aiming to do. It is designed to make Montanans remember why they like Tester: He's a farmer who works his own fields, he's friendly, and he could be your neighbor. Negative ads, for all their alleged faults, are the ads which focus more on issue based concerns. The ad is not devoid of issue content, but it is general and vague.

Finally, it is hard to say whether this buy is early or not. Given the changes in campaign finance and the fact that outside groups have been spending money since April of last year--and much of that has been negative--I'm led to believe that it is surprising that ads haven't gone up BEFORE now. Citizens United did change the nature of the campaign game, so many of us are learning as we go. The next question is: How long will we stay in the first advertising phase before moving into the argument phase?

The Montana Senate Race and Why Cutting Congressional Pay May Not be the Best Idea

In a campaign that very much centers on representational style and trust, the issue of congressional pay—not surprisingly—has emerged in the Tester-Rehberg Battle Royale. Highlighting congressional pay, perks, and benefits as an issue is an easy way for a campaign opponent to cast aspersions on the motives of an incumbent, to demonstrate that they are out of touch with their constituents, and infer that they abuse their public office for the purpose of self-gain. Both sides are using the issue to score political points and to demonstrate their closeness to Montanans. Unfortunately, congressional pay is an issue that should not be used as a political football because of the potentially serious consequences for Congress institutionally.

Congressional pay has long been a touchy political issue for representatives and senators. First, members of Congress are loath to increase their pay because of the negative repercussions they may face from constituents. Political scientists Bianco, Spence, and Wilkerson wrote an article on the Compensation Act of 1816, which noted that electorally vulnerable members were less likely to vote for the act, and that supporters of the provision were less likely to run for reelection because of the act’s unpopularity (1996). The act raised the pay of members by thirty percent, but it was the first pay increase since Congress set its own pay in 1789.

The current law governing congressional pay, the Government Pay and Ethics Act of 1989, increased member pay by forty percent over three years (see Hall and Van Houweling 1995). Arguments were made at the time, notably by Ornstein (1989), that increasing member pay was necessary to attract talented and experienced individuals to the job. In addition, several reforms were put in place in the act to create barriers to special interest influence on members. Members would no longer be allowed to accept honoraria from individuals and groups (some members made more than $25,000 annually by giving speeches to trade organizations and the like). Members whose service began before 1980 would also no longer be allowed to convert campaign cash to personal use upon retirement after 1992. Finally, a waiting period was instituted after retirement before members could lobby government. The main point of the act was to raise barriers to special interest influence while paying members more to remove the temptation to look for income in addition to official congressional pay.

As far as I can determine, congressional pay was first raised as a possible campaign issue a year ago by Congressman Rehberg. During the budget debate, Rehberg introduced legislation entitled “No Pay Raise for Congress Until the Budget is Balanced Act”. A press release announcing the bill notes that “Rehberg has a long history of supporting legislation to prevent pay raises or to stop the practice of automatic pay raises in Congress.” The Democratic Party of Montana chided Rehberg, noting that he has voted to increase his own pay “five times” since he was elected to Congress. You can read the party’s press release here.

Just last week, Citizens for Strength and Security launched an ad attacking the Congressman on the issue of congressional pay. It charges that Rehberg is hypocritical for accepting pay increases when he pledged to not accept any pay increases when running for the Senate against Max Baucus in 1996. Read my previous blog and view the ad here. It is important to note that Congress automatically receives an annual pay increase in the form of a cost of living adjustment UNLESS Congress acts to refuse that increase. The votes used against Rehberg are votes he made against proposals that would either end this procedure or to reject the annual increase. Senator Tester introduced legislation in 2011 entitled The Congressional Pay Prevention Act that would repeal these annual increases altogether. Today, Congressman Rehberg’s campaign pushed back saying that Senator Tester voted against a Senate amendment yesterday that would have frozen congressional pay. The vote was on a Senate amendment proposed by Republican Pat Roberts of Kansas. The vote cited can be accessed here. Furthermore, the Rehberg campaign notes that the Congressman has co-sponsored legislation in the 108^th, 109^th, 111^th, and 112^th Congresses eliminating the automatic annual pay adjustments. Both sides have called each other hypocritical and attempted to claim the moral high ground on the issue.

There are very good reasons other than electoral considerations for Senator Tester and Congressman Rehberg to oppose increasing congressional pay. Senator Tester’s stance on the issue is based largely on equity and fairness concerns. Most Americans do not get automatic pay increases annually. Folks making minimum wage do not get a cost of living adjustment. Farmers, business owners, ranchers, and many others in the private sector not only haven’t received pay increases, but have either lost their jobs or have had to take serious pay cuts. All of this creates a sound argument for why Congress should forgo those automatic increases. Congressman Rehberg’s stance on the issue seems to be based on performance. Pay raises are given in the private sector for tangible improvements in performance and meeting particular targets, like sales goals. Rehberg’s legislation suggests that Congress is not doing its job (and many Americans would certainly agree), so just like workers in the private sector, they should not get any pay increase. Future pay increases should be tied to particular performance metrics—like achieving a balanced budget.

I do not take issue, fundamentally, with the position of either Senator Tester or Congressman Rehberg. Nor do I seek to impugn their motives for their stance on congressional pay. I do, however, think that it is worth considering the broader institutional consequences of attacking the pay of members of Congress. Cutting pay may score points with justifiably frustrated constituents and fit preexisting notions of what is fair and equitable, but the outcome has the real potential to undercut the functioning of Congress as an institution in the long-term. By removing the automatic pay increases, member pay will increasingly become less competitive with other possible career avenues for members currently serving and folks thinking of running for office. Public service will become less attractive and possibly more open to creative income generating opportunities that could compromise the ethical positions of members. Add to the mix the difficulty of passing another pay increase in the future to fix a pay system that has gotten dramatically out of whack--remember what happened in 1816? What makes the situation even more difficult is the fact that most of the public already believe that members are grossly overpaid while simultaneously underestimating the actual value of congressional salaries and perks. If we want to prevent corruption, undermine special interest influence on the legislative process, and attract the best minds to Congress, the last thing we want to do is to cut their pay, send them home, and make them part time (this is something that has become popular among some in the Tea Party and was floated most prominently by Texas Governor Rick Perry during his brief presidential campaign). My position is unpopular and likely to draw some comment, but this an example where perhaps doing what the people want isn’t in the best interest of the people’s house.

Denny Rehberg Subject to Outside Attack

Lots of attention has been paid by outside groups attacking Senator Tester in the Montana Senate Race. Well, now some groups are now going after Congressman Rehberg. Citizens for Strength and Security just dropped $400,000 on the ad below--attacking Congressman Rehberg for taking pay raises throughout his career.




I'll have more to say about this advertisement--and frequent attacks on pay increases--in another blog post coming soon....

The Montana Senate Race featured on NPR

Yesterday, a piece aired by Martin Kaste on the Rehberg-Tester race. You can listen to it here.

The 1982 Melcher campaign ad that I mentioned is on YouTube. You can view the whole ad here.

The Montana Senate Race: Who are these guys and what are their records?

Watching this Montana Senate race closely has proved to be immensely interesting, enjoyable, and fascinating. Readers of this blog will note that I have repeatedly stressed the notion of “one of us”—campaigns work hard to show how their candidate is connected to voters and how their opponent is not. This really gets at the heart of representative democracy: voters are more likely to trust decisions made by representatives who look, act, and think like them. To use the language of postmodernism, what voters do not want and fear is the “other”: the representative who is beholden to someone else and who will make decisions not in their best interest.

Part of proving or demonstrating “one of us” is to discuss legislative records: What have you done for me lately and how is it evidence that you are serving the best interests of the state? Senator Tester casts himself as an independent voice for an independent-minded state, while calling Congressman Rehberg a yes man for Republican Party bosses. Congressman Rehberg, alternatively, says he’s the true voice of the Montana voter and Senator Tester is merely another vote for President Obama and his agenda. Each side will use a vote or a series of votes to support their position: Senator Tester voted with President Obama X number of times, and Congressman Rehberg is too conservative for Montana because he voted for House Republican proposal X, Y, or Z.

How can the average voter make any sense of these claims? As with all things, it depends on how you look at the records of both candidates. I am going to present three ways of thinking about Senator Tester’s and Congressman Rehberg’s records. I am going to present some additional data that will help you contextualize those records. Using this information, I provide some brief commentary but will largely leave it to Montanans to draw their own conclusions about the claims of both campaigns about their own records and the records of their opponents.

In general, the claims made by the campaigns about themselves and their opponent revolve around three distinct concepts:

Party Unity: How often does the member vote with their party in Congress?

Ideology: How liberal or conservative is the member?

Presidential Support: How often does the member support the president’s legislative agenda?

There are a number of ways these concepts can be empirically measured. Here are some standard measures utilized by political scientists that we’ll examine:

Party Unity: There are many ways to measure party unity, but the most common measure is produced annually by Congressional Quarterly since the late 1950s. CQ defines a party unity vote as a majority of one party opposing the majority of the other party on a given vote. To be specific, if 51% of Republicans vote yea on a motion and 51% of Democrats vote Nay, that vote meets the minimum standard to be included as a party unity vote. A party unity score is the percentage of times a member votes with their party on all party unity votes cast in a given year. The percentage of party unity votes as a percentage of all votes cast in Congress has ebbed and flowed throughout the years—the mid-1970s were a low point, while the last twenty years has seen a rebound in the number of votes clearly dividing the parties. Party unity scores have also increased over the late 20^th and early 21^st centuries, meaning members of the House and the Senate have become more loyal to their parties and less likely to break partisan ranks. Averages today are typically in the high 80s to low 90s in both chambers, with the House usually exhibiting more party loyalty than the Senate.

Ideology: One way to measure how liberal or conservative a member is to use DW-NOMINATE scores. DW-NOMINATE is a measure of ideology created by Poole and Rosenthal in their seminal work, Congress: A Political-Economic History of Roll Call Voting. Essentially, DW-NOMINATE takes the all the votes cast in Congress over time and develops factors which explain the variation found in those votes. The first dimension score Poole and Rosenthal call ideology, and this measure places members of Congress on a left-right dimension. Positive scores approaching 1 are more conservative, while negative scores approaching -1 are more liberal. It is important to note that both parties have become increasingly polarized after 1975, which means the average Democrat has become more liberal and the average Republican more conservative in both the Senate and the House. The other important thing to note is that both parties have become more ideologically homogenous—meaning moderates have become increasingly rare. More information on NOMINATE and NOMINATE scores can be found at Keith Poole’s website here.

Presidential Support: Again, Congressional Quarterly has set the standard for developing a measure of Presidential Support. CQ determines whether the presidential administration has taken a clear and public position on legislation. If the administration has, the vote on taken on that legislation is included in the universe of votes utilized to calculate an individual legislator’s presidential support. Simply put, a presidential support score is the percentage of times a member of the House or the Senate has voted to support legislation on which the administration has taken a public position. Higher scores indicate more support for the administration, lower scores less support. Traditionally, administrations receive the bulk of their support from their parties in Congress—with support of the administration’s positions from the opposition party declining in recent decades. Congressional Democrats often support Democratic administrations, and Congressional Republicans support Republican administrations.

Below, I have created a series of graphs depicting the voting records of the Montana congressional delegation between 2001 and 2011. In each instance, I plot the party unity, ideology, and presidential support scores for Senators Tester, Baucus, Burns, and Congressman Rehberg. I also provide contextual information, namely the average scores for Democrats and Republicans in both chambers. Finally, I wrap up the discussion with some concluding thoughts and by examining key congressional votes to see if that information tells a story that is any different from these more general measures of party unity, ideology, and presidential support.

Party Unity

The first chart plots the party unity scores of Montana’s Senate delegation and compares it to the party means in the chamber. Again, a party unity score is the percentage of time the member votes with their party when the majority of each party is in opposition. The second chart plots Congressman Rehberg’s party unity score relative to his party’s average scores.

Figure 1: Party Unity in Montana's Senate Delegation, 107th-111th Congresses

Figure 2: Congressman Rehberg's Party Unity, 107th-111th Congresses

NOMINATE

How liberal or conservative is Montana’s Congressional Delegation? NOMINATE scores help sort this out. The first chart plots the first dimension DW-NOMINATE values of Montana’s Senate delegation, the second plots Congressman Rehberg relative to his peers in the House.

Figure 3: Montana's Senate Delegation and Ideology, 107th-111th Congresses



Figure 4: Congressman Rehberg's Ideology, 107th-111th Congresses

Presidential Support

How often did Presidents Bush and Obama receive support from Montana’s Congressional delegation? The final two charts show us. Again, the first chart is the CQ Presidential Support score plotted for Senators Burns, Baucus, and Tester and the second chart is Congressman Rehberg relative to his Republican peers.

Figure 5: Montana's Senate Delegation and Presidential Support, 107th-111th Congresses

Figure 6: Congressman Rehberg's Presidential Support, 107th-111th Congresses

Summary

In general, we learn the following from these measures of party unity, ideology, and presidential support:

1. Members of Congress generally support their parties on party unity votes.

2. Members of Congress generally support the administration when the administration shares their party affiliation.

3. Ideologically, Montana’s Democratic Senate delegation votes to the right of the average Senate Democrat. It is difficult to call either Senator Baucus or Tester a liberal—they are moderate or—perhaps more generously—conservative Democrats. Congressman Rehberg faithfully exists at about his party’s mean in the House—and quite possibly a tad to the left of the mean GOP member in the last Congress. Congressman Rehberg is conservative, but his conservatism is not extreme relative to his peers. He’s no Ron Paul or Randy Neugebauer, both representing Texas and two of the most conservative members of the Republican delegation in the House.

Three different ways to consider the legislative records of Montana’s congressional delegation. Overall, the picture is far more complex and nuanced than we might expect given the campaign rhetoric we hear on Montana’s airwaves, read in our inboxes, and a view on Facebook.