I saw on my Twitter feed that MSU-Billings has come out with it's latest Montana public opinion poll. You can read the article about the poll findings here and go look at the full report here. I take some issue with how this poll was reported and how some folks are talking about it in the Twitter-verse.
First, let's talk gay marriage. The poll results indicated that a plurality of Montanans favor gay marriage, with 46.6 in favor and 42 percent opposed. The problem with this statement is the margin of error in the poll is 5 percentage points. This means a majority of Montanans might actually support gay marriage (5+46.6 is 51.6 percent) OR less than a plurality support it (46.6-5 = 41.2). So, is not accurate to say that a plurality of Montanans support gay marriage. It would be better to say that opinion on gay marriage is mixed and that support for gay marriage is statistically tied with opposition to it.
Second, let's talk about job approval ratings. Montana Cowgirl, one of the Montana state politics blogs I follow, sent a Tweet saying that Steve Daines is the most unpopular politician in the state. Here are the job approval numbers in the poll:
Steve Bullock, Approval: 53 percent.
Jon Tester, Approval: 44.7 percent
Max Baucus, Approval: 44.1 percent
Steve Daines, Approval: 39 percent
Remember that margin of error? We can confidently say that Steve Bullock is more popular than Steve Daines because the Bullock's approval rating could be as low as 49 percent or as high as 58 percent. Bullock might be as popular as Tester and Baucus, and Steve Daines might be as popular as Tester and Baucus. Or Steve Daines might really be the most unpopular elected official (39-5 = 34). But we cannot confidently, based upon these results, say that Steve Daines is the most unpopular elected official with the large margin of error.
What is most interesting, however, the number of folks who are undecided about Daines and Bullock relative to the senators:
Baucus, undecided: 16.9
Tester, undecided: 15.9
Daines, undecided: 37
Bullock, undecided: 33
What this tells me, and I can stay this with statistical confidence, is that Steve Bullock and Steve Daines are the least KNOWN statewide elected officials. And, look at the disapproval numbers:
Bullock, dispprove: 14
Daines, disapprove: 24
Tester, disapprove: 39.4
Baucus, disapprove: 39
Looked at this way, I could say with confidence that more Montanans who can express an opinion on on Tester and Max Baucus neither clearly approve or disapprove of the job they are doing. I can also say, however, that among those who express an opinion on Daines' job that more approve of the job he's doing than disapprove (39-5 = 34, and 24+5=29). He and Bullock are the only ones according to these numbers who have a statistically discernible pro-job approval rating outside the margin of error.
Clearly, among those Montanans who express an opinion on Steve Bullock really like the job he's doing. I suspect that part of that has to do with a) Bullock's not associated with the mess and Washington and b) the legislature has gone home, so he's not associated with any divisive doings in Helena at the moment. Main lesson: It's good to be the Governor when Washington is blowing up.
All three of our federal officeholders likely have depressed job approval numbers because of the government shutdown and the debt crisis. What will be interesting to watch is whether Daines, in particular, can increase his approval ratings among those who are currently undecided about him.
Showing posts with label polling. Show all posts
Showing posts with label polling. Show all posts
Friday, October 25, 2013
Friday, May 4, 2012
Tester's up by 5--no, wait a minute--Rehberg's up by 10, ARGH! Making Sense of Conflicting Polls
Within 48 hours, we have seen two different firms release conflicting polls in the Montana Senate race. On May 1, Public Policy Polling (PPP)--a Democratic polling company--released a poll indicating that Jon Tester maintained a five point lead over Congressman Rehberg, 48-43. The lead was just within the margin of error of 3.2%. Nate Silver at The New York Times has analyzed polling firms and their biases. He actually found that PPP's polls lean slightly toward Republicans.
On May 3, Rasmussen Reports--a polling firm that generally has a pro-Republican tilt in its polling--released another poll with completely different results. This poll shows a ten point lead for Congressman Rehberg--53-43--the largest lead we've seen in this race to date. This lead however is also just inside the margin of error, which was 5 percent in this poll.
How do we make sense of these conflicting polls?
Let's start with a couple of fundamental points. First, the PPP poll was conducted between April 26-29 and included 934 Montana voters. The Rasmussen poll was conducted on May 2 and included only 450 likely Montana voters. That's why we see the different margin of errors--the larger the sample size, the lower the margin of error.
Second, let's talk about the margin of error and what that means. In the PPP poll, Senator Tester's support could range between 51.2 and 44.8. Congressman Rehberg's support could be as high as 46.2 and as low as 39.8 percent. In the second poll by Rasmussen, Senator Tester and Congressman Rehberg could both be tied at 48 percent. In either case, the leads by both are within the margin of error--so the results are not quite as out of line as one might expect just by looking at the head to head matchups reported by the polling firms.
But let's dig a bit deeper. One of the hardest things to figure out in the polling world is who will actually show up to vote. Forecasting turnout is about as hard as forecasting the weather because there are so many variables at work and the instruments we use to measure intent are subject to social desirability biases. If you ask a person if they intend to vote, most likely will give you the socially desirable answer: "Sure, I plan to vote". The problem is about 80 to 85% of voters will answer yes--and we know that turnout generally hovers between 50 and 60 percent. In other words, a bunch of folks who say they will vote simply don't.
Pollsters have lots of ways to measure turnout, and the differences in measuring turnout can have consequences for the final polling results that are reported.
How does Rasmusen and PPP differ in their turnout screeing questions? According to an e-mail exchange I had this morning with Tom Jensen at PPP, his organization calls folks who have voted in one of the last three general elections.Rasmussen, however, polls "likely voters". What's a likely voter? Rasmussen asks several screening questions, including the respondent's voting history, their interest in the election, and their likely voting intentions. This is a much more vigorous screening process designed to weed out folks who may not actually show up on election day.
PPP's process likely yields a "liberal" definition of turnout and Rasmussen's a "conservative". I use those quotes deliberately. PPP process might include folks in the sample who are less committed to voting than the Rasmussen poll. Demographically, Democrats usually have the turnout deck stacked against them relative to Republicans. Folks who are poorer, less educated, and not white are less likely to vote than those who are richer, more educated, and white. In short, those who are more likely to vote for Democrats are also those who are less likely to vote.
In short, the difference in results MIGHT be a factor of how each polling firm choses to define a voter. And there is no one "right" way. A generous interpretation of these conflicting polls is the higher the turnout on election day, the better chance Senator Tester has at getting reelected. Lower turnout, on the other hand, will likely benefit Congressman Rehberg.
Final take away message: Read polls carefully, examine the methodology section thoroughly, and go beyond the first page of the press release if you really want to understand why polls conflict.
Other Important Notes:
Both Senator Tester and Crossroads GPS launched some new ads in the past week. Senator Tester's ad, a nice positive bio spot about is propensity to fly Montana meat to DC with him on the plane, is right below.
The Crossroads GPS ad is below. It covers no new ground, focusing on Tester's votes on healthcare, cap and trade, and the federal budget--again, mentioning that Senator Tester voted 97% of the time with President Obama (see my previous analysis on voting here).
On May 3, Rasmussen Reports--a polling firm that generally has a pro-Republican tilt in its polling--released another poll with completely different results. This poll shows a ten point lead for Congressman Rehberg--53-43--the largest lead we've seen in this race to date. This lead however is also just inside the margin of error, which was 5 percent in this poll.
How do we make sense of these conflicting polls?
Let's start with a couple of fundamental points. First, the PPP poll was conducted between April 26-29 and included 934 Montana voters. The Rasmussen poll was conducted on May 2 and included only 450 likely Montana voters. That's why we see the different margin of errors--the larger the sample size, the lower the margin of error.
Second, let's talk about the margin of error and what that means. In the PPP poll, Senator Tester's support could range between 51.2 and 44.8. Congressman Rehberg's support could be as high as 46.2 and as low as 39.8 percent. In the second poll by Rasmussen, Senator Tester and Congressman Rehberg could both be tied at 48 percent. In either case, the leads by both are within the margin of error--so the results are not quite as out of line as one might expect just by looking at the head to head matchups reported by the polling firms.
But let's dig a bit deeper. One of the hardest things to figure out in the polling world is who will actually show up to vote. Forecasting turnout is about as hard as forecasting the weather because there are so many variables at work and the instruments we use to measure intent are subject to social desirability biases. If you ask a person if they intend to vote, most likely will give you the socially desirable answer: "Sure, I plan to vote". The problem is about 80 to 85% of voters will answer yes--and we know that turnout generally hovers between 50 and 60 percent. In other words, a bunch of folks who say they will vote simply don't.
Pollsters have lots of ways to measure turnout, and the differences in measuring turnout can have consequences for the final polling results that are reported.
How does Rasmusen and PPP differ in their turnout screeing questions? According to an e-mail exchange I had this morning with Tom Jensen at PPP, his organization calls folks who have voted in one of the last three general elections.Rasmussen, however, polls "likely voters". What's a likely voter? Rasmussen asks several screening questions, including the respondent's voting history, their interest in the election, and their likely voting intentions. This is a much more vigorous screening process designed to weed out folks who may not actually show up on election day.
PPP's process likely yields a "liberal" definition of turnout and Rasmussen's a "conservative". I use those quotes deliberately. PPP process might include folks in the sample who are less committed to voting than the Rasmussen poll. Demographically, Democrats usually have the turnout deck stacked against them relative to Republicans. Folks who are poorer, less educated, and not white are less likely to vote than those who are richer, more educated, and white. In short, those who are more likely to vote for Democrats are also those who are less likely to vote.
In short, the difference in results MIGHT be a factor of how each polling firm choses to define a voter. And there is no one "right" way. A generous interpretation of these conflicting polls is the higher the turnout on election day, the better chance Senator Tester has at getting reelected. Lower turnout, on the other hand, will likely benefit Congressman Rehberg.
Final take away message: Read polls carefully, examine the methodology section thoroughly, and go beyond the first page of the press release if you really want to understand why polls conflict.
Other Important Notes:
Both Senator Tester and Crossroads GPS launched some new ads in the past week. Senator Tester's ad, a nice positive bio spot about is propensity to fly Montana meat to DC with him on the plane, is right below.
The Crossroads GPS ad is below. It covers no new ground, focusing on Tester's votes on healthcare, cap and trade, and the federal budget--again, mentioning that Senator Tester voted 97% of the time with President Obama (see my previous analysis on voting here).
Tuesday, May 1, 2012
Newest Poll Showing Tester with the Lead
I'm still in the middle of finals week, and there's so much to post! Senator Tester has a new ad....GPS Crossroads has a new ad...and today, a poll showing Senator Tester with a five point lead. The lead is still just within the margin of error, but this is the first poll in the last--five I think--that shows Senator Tester with a lead. PPP did the poll, and you can read more about it here.
I'll post the ads and more analysis once my grading is done, when I will have more time to really sink my teeth into this race. Stay tuned and be patient, my friends.
I'll post the ads and more analysis once my grading is done, when I will have more time to really sink my teeth into this race. Stay tuned and be patient, my friends.
Thursday, February 23, 2012
New poll--but same story in the Montana Senate Race
Montana Republicans and the Rehberg campaign are touting a poll showing their man leading in the Montana Senate race. The poll was done by Rasmussen and shows Congressman Rehberg with 47% of the vote and Senator Tester with 44%. The poll was of 500 likely Montana voters conducted on February 22, 2012. The margin of error is 4.5 percent, so really, the race is too close to call. I generally do not like to say a candidate is in the lead unless the candidate has a lead outside the margin of error. You can read about the particulars here.
I should note that in 2010, Rasmussen's polling consistently showed a considerable Republican bias as noted by Nate Silver who blogs about survey methodology and statistics over at The New York Times. Read his analysis here. That said, I have less of a reason to believe that any bias--should it remain in Rasmussen's methodology during this election cycle--is reflective in THESE results given that they show us pretty much what we've seen since March of 2011: the race is...too close to call.
Thursday, December 1, 2011
New Poll in MT Senate Race
There's a new poll on the Montana Senate Race tonight by Public Policy Polling. Read the results here.
Four quick points about the poll:
1. The race has been dead even since the very beginning. There has been hardly any change in the numbers in this new poll.
2. This consistency is NOT surprising. You have two candidates who are well-known and are well-defined. It will be hard to move public opinion with campaign advertising in such an environment. In order to move voters in such a situation, absent either an external event that shakes up the race or a mistake by one of the candidates, lots of money will have to be spent on advertising to have any effect on voter opinion at all.
3. The fact that a significant amount of money has already been spent by outside organizations further underscores the second point. That said, money spent on advertising this past summer was not generally targeted at Montana voters (many of whom were spending their precious warm days outside and away from television), but at the political elite and the donor base. This advertising served to get the base excited and perhaps get them to give money. The organizations that did spend money on advertising were also sending signals to other political organizations about the importance of this race and the effort they plan to expend in the upcoming months on the Montana Senate race. This signaling can cause other organizations to take note and decide that the Montana Senate campaign is worth their time and attention as well.
4. Given that the fundamentals of this race have changed very little thus far, I expect--like most close congressional races--that this campaign will boil down to the enthusiasm of each candidate's base and the ground game. For all the money that is spent on advertising, more important and more significant are the resources and efforts spent on connecting with voters personally and through social media.
Four quick points about the poll:
1. The race has been dead even since the very beginning. There has been hardly any change in the numbers in this new poll.
2. This consistency is NOT surprising. You have two candidates who are well-known and are well-defined. It will be hard to move public opinion with campaign advertising in such an environment. In order to move voters in such a situation, absent either an external event that shakes up the race or a mistake by one of the candidates, lots of money will have to be spent on advertising to have any effect on voter opinion at all.
3. The fact that a significant amount of money has already been spent by outside organizations further underscores the second point. That said, money spent on advertising this past summer was not generally targeted at Montana voters (many of whom were spending their precious warm days outside and away from television), but at the political elite and the donor base. This advertising served to get the base excited and perhaps get them to give money. The organizations that did spend money on advertising were also sending signals to other political organizations about the importance of this race and the effort they plan to expend in the upcoming months on the Montana Senate race. This signaling can cause other organizations to take note and decide that the Montana Senate campaign is worth their time and attention as well.
4. Given that the fundamentals of this race have changed very little thus far, I expect--like most close congressional races--that this campaign will boil down to the enthusiasm of each candidate's base and the ground game. For all the money that is spent on advertising, more important and more significant are the resources and efforts spent on connecting with voters personally and through social media.
Friday, November 19, 2010
The Difference Between 1994 and 2010? Democrats Saw the Wave
An interesting piece on the 2010 midterms from MSNBC today on the debt Democrats took on to prevent even larger losses in the House. Some key points to consider:
In 1994, The Democrats were caught unawares when the GOP tidal wave hit. Now, because of the proliferation of polling, the Democrats not only knew they were in trouble, they had a good idea where and could shift resources appropriately. The same thing happened in 2006 when the Republicans lost their majority. In both cases, it seems the losses could have been much worse. Because of the ability to use polls to target with sophistication and partisan-motivated redistricting, large wave elections like '94 and '10 are much rarer today.
Second, the large debt combined with the loss of the House majority weakens considerable the Democrats going into the 2012 congressional election cycle. It's much hard to raise money when you aren't in the majority. Add to this the monumental drubbing the Democrats took in state gubernatorial and legislative races--which give the GOP a big leg up in redistricting, and forget about the Democrats winning back the House. Indeed, we'll probably see Republican gains in 2012.
In 1994, The Democrats were caught unawares when the GOP tidal wave hit. Now, because of the proliferation of polling, the Democrats not only knew they were in trouble, they had a good idea where and could shift resources appropriately. The same thing happened in 2006 when the Republicans lost their majority. In both cases, it seems the losses could have been much worse. Because of the ability to use polls to target with sophistication and partisan-motivated redistricting, large wave elections like '94 and '10 are much rarer today.
Second, the large debt combined with the loss of the House majority weakens considerable the Democrats going into the 2012 congressional election cycle. It's much hard to raise money when you aren't in the majority. Add to this the monumental drubbing the Democrats took in state gubernatorial and legislative races--which give the GOP a big leg up in redistricting, and forget about the Democrats winning back the House. Indeed, we'll probably see Republican gains in 2012.
Subscribe to:
Posts (Atom)
