Margin of Σrror

Margin of Σrror -

Has the Millennial generation ‘overwhelmed’ the electorate? | Harry J Enten

Though they’ve drawn comparisons with the Greatest Generation for their liberal leanings, young people haven’t taken over yet

The Millennials are the most Democratic cohort in a generation. Some believe attribute their liberal inclinations to the racially diverse demographics within their generation, and while that may be partially true, I prescribe the Millennials’ (born after 1980 through the mid-90s) Democratic leanings to the fact they grew up during a strong Clinton administration and a weak Bush one. The Millennials today have mostly replaced the very white, very Democratic voters of the Greatest Generation (who were born 1910-1927) who came of age during the weak years under Hoover and a Roosevelt administration so strong it won FDR a third term.

In both the 2004 and 2012 election, almost all age cohorts voted the same relative to other cohorts, and the Millennials were as Democratic relative to the nation in 2004 and 2012. The Greatest Generation was too small a percentage of the electorate in 2012 to collect poll data, but they voted as Democratic as the Millennnials did in 2004.

Proving the cohort point further, the then +60-year-old Greatest Generation has been the most Democratic cohort all the way back in 1988, when age cohorts didn’t differ all that much in how they voted – despite even some members of the Lost Generation still voting. The Greatest Generation crowd was the most Democratic in the 1992 election, as well.

Even the most ardent critics of the cohort theory will admit that a person’s views of the presidential administration he or she grew up with will shape their political views going forward. The real question is whether or not the Millennials have a a large enough portion of the electorate to “overwhelm” the rest: would Bush have still won in 2004 with 2012 demographics, and would Obama have still won with 2004 demographics? With the release of the Census Bureau’s Current Population Survey (CPS) voter supplement, we can find out.

The CPS is a geographically defined sample that seeks to fully represents each type of voter. Respondents are asked simple questions such as race, registration, and whether or not they voted. It is generally seen as a more comprehensive survey than network exit polls to determine the exact composition of the electorate.

There are, however, reasons to be cautious about making too much of the CPS report. The CPS doesn’t have a perfect response rate, and as my friend Sean Trende points out, there are more people who claimed to the CPS to have voted than actually did. The CPS tabulators assume that those who didn’t respond didn’t vote, which is likely not true, but there’s no perfect way to account for the discrepancies. Preliminary examination of board of election data from different states shows that fewer African Americans, for instance, voted than the numbers that the CPS finds.

Either way, the CPS is a very solid starting point, and we can still take a preliminary step in answering whether or not Obama would have won without the Millennials. We can determine this by multiplying the percent of what Obama won among Millennials by the percentage they made up of the electorate. In other words, the percentage of vote the Millennials contributed to Obama’s margin of victory. Then we compare this with the percentage of the vote the Greatest Generation contributed in prior years, and we’ll have a solid answer.

The Millennials now make up 18% of the electorate, per the CPS. That’s less than the exit poll data reports, but exits have been known to count too many young voters. Meanwhile, the CPS data is backed quite well by Pew Research, which most would agree is one of the finest pollsters out there, if not the best.

Those born between 1910 and 1927 were just 2% of the 2012 electorate. In 1996, the election before the Millennials began voting, the Greatest made up 15% of everyone who cast a ballot. By the time 2004 rolled around, Millennials were 8% of the electorate, while the Greatest was down to 7%. Thus, as a percentage of the total electorate, there was a 5pt gain in the Democratic coalition of the Greatest Generation and Millennials from 1996 to 2012, and from 2004 to 2012.

Given Obama’s +20pt win among Millennials, what percentage of the vote is that 5pt difference worth? With the 2004 electorate, Obama would have won by 2.6pt instead of 3.85pt last year. Meanwhile, if the 2004 electorate had looked like the 2012 one, George W Bush would have won by 1.2pt instead of 2.45pt. So yes, the Democratic candidate would have done slightly better with the demographic boost, but neither election would have turned out any differently.

Further comparisons to 1996 and 2004 undersell the Greatest Generation’s impact. The Greatest were 17% of the electorate in 1992, 21% in 1988, 24% in 1984, and 27% in 1980, but have dropped steeply as more of their members pass away. The Millennials, meanwhile, are still far away from 24%, let alone 27%. If other age cohorts had voted the same relative to the national vote, Obama might have actually won by more in prior years.

To me, the evidence does not suggest the Millennials have “overwhelmed” the Greatest. They are, if anything, a new “Greatest Generation” both in terms of voting patterns and, to a lesser extent, size – though we can still expect the Millennials to grow somewhat as a percentage of the electorate, since people are more likely to vote as they get older. The question going forward is whether Generation Z (born in the mid-90s and later) will follow the voting patterns of the Millennials.

His small re-election margin and his projected historic ratings mean that the Obama administration has largely been seen as mediocre, which in turn suggests that the next age cohort will walk the middle of the road. The polling data agrees that Generation Z will be less Democratic, than the Millenials, but if one believes the racial diversity theory, then the next generation should actually be more liberal, as it will have fewer white voters.

We don’t know whether the age cohort or racial diversity theory will end up being more correct in the long run. If it’s the diversity one, Republicans are in a lot of trouble. If it’s the age cohort theory, then it will be politics as usual. I’d bet on age.

guardian.co.uk © 2013 Guardian News and Media Limited or its affiliated companies. All rights reserved. | Use of this content is subject to our Terms & Conditions | More Feeds

The IRS and AP controversies spell trouble for Obama and Democrats | Harry J Enten

Trust in government is a key factor in how people vote in midterm elections. All of that is heavily at stake now for 2014

The Justice Department has “secretly obtained” two months of conversations between Associated Press (AP) officials in a move called “unprecedented”. The Internal Revenue Service (IRS) Cincinnati office singled out new conservative groups for extra scrutiny over the past couple of years. One of these controversies alone would have caused a headache for the Obama administration, but the two of them together could spell big trouble for the Democrats in the 2014 midterm elections.

Historically speaking, trust in government has been tied very closely to how people view the state of the economy. When consumer sentiment is up, trust in government goes up. When consumer confidence goes down, trust in government goes down. Pew Research has a very nice chart that illustrates this relationship.

You can see how the two lines generally flow together. This especially the case after 1970 – before which time trust in government was higher than it’s been over the past 40 years. Right now consumer confidence is 76.4. That’s down from earlier this year, but it’s up significantly since 2010.

Trust in government isn’t, however, always linked to the consumer sentiment. After the Watergate scandal, trust in government remained in a relatively low stable position through 1977, even as the economy improved. Trust in government fell in response to the House banking scandal in 1992 and Whitewater controversy of 1993 and 1994, as consumer confidence climbed. Finally, the economy was improving, but trust in government fell off its post-9/11 highs through the early part of the first decade of this century as Americans angered over the Iraq War and Hurricane Katrina that pounded the Gulf Coast.

Put another way, scandal can negatively impact how much Americans trust. It has to be a big scandal though. Benghazi, for instance, is likely not going to do it. Most Americans aren’t paying attention to it, and as many Americans think the Republicans have gone too far as handled it appropriately.

The tax scandal, however, can play that role. My friends at NBC’s First Read note

“The IRS story packs a bigger political punch… [and] will trigger new congressional hearings and new questions for the president and his team. More significantly, the IRS news is a political gift to a Republican Party whose base was strained on immigration.”

The idea that the IRS would go after conservative groups, who hate the IRS, specifically seems to make a lot more sense than a president not wanting to create a foreign policy crisis in which be could benefit from a rally around the flag effect.

The obtaining of AP records likewise probably makes more sense in the voters’ eyes. Obama has been critiqued for not doing enough press conferences or interviews with White House reporters. As one Democratic strategist put it, the “AP phone records thing just sealed the deal for what the newest narrative around Obama administration is going to be”.

Indeed, these stories are coming at the perfect time for peak scandal coverages. Brendan Nyhan notes that scandals more often happen when the president is detested by members of the other party, as Obama is. Likewise, they are more likely to become big news when there aren’t other news stories like the Boston bombings. Finally, scandals are more likely to take place in the beginning of the second term.

Therefore, the question is whether declining trust in the government has historically played a major factor in midterm elections. It turns out that it does. When trust in government falls, the party in the White House tends to do worse in midterm elections.

This 2010 graph from Pew Research illustrates the point well.

Trust fell dramatically in the lead up to the 1974 midterms thanks to Watergate, and the Republicans lost nearly 50 seats. Trust absolutely tumbled in the lead-up to the 1994 midterms, and Bill Clinton’s Democratic Party lost over 50 seats. Democrats took back the house in 2006 as Americans trust in the Bush administration dropped. And although it isn’t on the bottom chart, Americans trust in government, as seen in the top chart, was bad in 2010. Republicans gained 63 house seats.

On the reverse, there was minimal change in the composition in 1986, 1990, 1998, or 2002 when trust in government was relatively high.

So what does this mean for 2014? There are reasons to believe that Democrats shouldn’t see major losses. The economy is doing better, and an incumbent president’s party rarely loses big twice in midterms. Still, if trust falls, it may trump these structural factors. The thing to watch over the next days, weeks, and months is how big the scandals become. If they become big news, and that seems quite possible, Obama’s Democratic party may be heading for major losses in 2014.

guardian.co.uk © 2013 Guardian News and Media Limited or its affiliated companies. All rights reserved. | Use of this content is subject to our Terms & Conditions | More Feeds

Polls at this point in the Virginia governor’s race tell us very little | Harry J Enten

Virginia’s governor race is being billed as a test case for 2016, but you can’t read much into polling data this early

Virginia’s 2013 gubernatorial election may not have particularly appealing candidates, but the race has taken on extra meaning because of the state’s swing status in presidential elections. The race pits a so-far unlikable Democrat, Terry McAuliffe (aka T-Mac), against a exceptionally conservative Republican, attorney general Ken Cuccinelli (aka Cuch). Are swing state Virginians willing to elect a person, Cuccinelli, who wants to defund Planned Parenthood and has labeled the Environmental Protection Agency an “agency of mass destruction”? McAuliffe, on the other hand, was a long-time Clinton fundraiser and adviser, and his run is seen as a test of the Clinton name and organization for Hillary’s possible run in 2016.

From two polls produced this week we can make similar conclusions with different results. An NBC/Marist poll has T-Mac up by 2pt among registered voters and down by 3pt among likely voters. A Washington Post survey has Cuch up 5pt among registered voters and 10pt among likely voters. You’ll note that the overall results are different, but that the gap between the registered and likely voters is the same.

Right now, the truth is probably somewhere in the middle. Quinnpiac has polled registered voters three times since January, and each of their results fell in between the registered voter numbers from Marist and the Post. Likewise, a Christopher Newport University survey from January had a similar 4pt gap between registered voters and likely voters. Off-year elections tend to feature more Republican friendly electorates, given that African Americans and young people are a lower percentage of the electorate than in presidential year elections.

But does this current polling actually tell us anything about what will happen on election day? Not really. At this point in 2009, all but one scientific poll had eventual winner and current Republican Governor Bob McDonnell winning by 9pt or less. A few polls even had Democrat Creigh Deeds ahead by mid-June, but most had McDonnell ahead by about 5pt. With a few exceptions – notably from SurveyUSA, which consistently showed double-digit McDonnell leads by early August – most surveys continued to show a single-digit McDonnell lead through September. McDonnell won by 17pt.

You think that’s unusual? Go back to the 2005. Through this point in the campaign, not a single poll had the eventual winner, Democrat Tim Kaine, ahead. A few polls had him down as much as 10pt, while the majority had him behind by mid-to-high single digits. Republican Jerry Kilgore trailed in only one public poll, all the way through early October! Tim Kaine, of course, went on to win by 6pt.

Overall, the polling at this point and through September has been off by about 10-15pt the last two elections. Given that Cuccinelli has a 4pt lead in the HuffPollster aggregate, either candidate could win. Perhaps Cuccinelli will win by a margin greater than 4pt, maybe less, maybe exactly that. Or maybe T-Mac will come out on top.

You might be tempted to look at the candidates’ favorable ratings, but I wouldn’t make too big a deal of those either, as they merely reflect the ballot standing of each man at this point. The important point is that both feature positive net favorables. Given that over 30% of voters have no opinion of Cuch, and over 40% have no opinion of T-Mac, the favorable ratings of these candidates will change. In 2005, Jerry Kilgore had a higher net favorable than Tim Kaine, which could be seen in the ballot test. He ended the election with a lower net favorable rating and lost. In 2009, Bob McDonnell had a net favorable rating equal to Creigh Deeds in early polling, which was reflected by a tight race in the ballot test. McDonnell ended up with a much higher net favorable rating, and indeed, he crushed Deeds.

What about the difference between the likely and registered voter results? The number one polling lesson from 2012 was that when likely and registered voters disagree in elections with high voter turnout, you should go with the registered voter results. Obviously, an off-year affair is not a high turnout election. Still, I was curious to see if over the past two cycles, pollsters have tightened the electorate, keeping in mind that only a few pollsters release results among both registered and likely voters, and most don’t.

Among those who report results from different screens, it’s not really clear that pollsters who use a likely voter screen are getting more accurate results than those that only polled registered voters. A 2009 October Virginia Commonwealth University poll had McDonnell winning by 18pt among likely voters and 16pt among registered voters. Neither was more accurate than the other. A 2009 October Roanoke College poll had McDonnell ahead by 17pt among likely voters and by 19pt among who said they were certain to vote (who we’ll call “definite” for the sake of clarity). In this instance, the looser screen ended up being closer to the final result.

In 2005, tightening the electorate did pollsters no favors either. An October Diageo/Hotline poll found Kaine winning by 2pt among registered voters, 1pt among likely voters, and down by 2pt among definite voters. Kaine, as mentioned, won by 6pt. An October Roanoke College poll had Kaine winning by 8pt among probable voters and 10pt among definite voters. Again the looser screen performed better than the more selective one.

What about earlier in the campaign? In 2009, an August Washington Post poll had McDonnell up by 15pt among likely voters and by 7pt among registered voters. Clearly, the likely voter result was closer to the truth – in this case, at least. In 2005, the opposite occurred. An early September Washington Post poll had Kilgore up by 7pt among likely voters and by 4pt among registered voters. Earlier in the campaign, SurveyUSA showed a dramatic rise in Kilgore’s lead from 4pt to 10pt between May and June when switching between registered and likely voters. Other polling companies showed a consistent race, which means that the far less accurate June result was almost certainly because of a change in the electorate being polled.

Thus, I’m not really sure there’s much to take from the recent polling data on the Virginia gubernatorial race. Yes, Ken Cuccinelli likely has a small lead at the moment, but that edge probably means very little. The difference between the registered and likely voter screen may be instructive, but we can’t yet be sure when to trust it. Tighter voter screens have not proven to necessarily be more accurate in either the 2005 or 2009 Virginia gubernatorial elections.

We can only sit back and allow the campaign to unfold.

• A previous version of this article incorrectly stated the leader of The Washington Post’s survey of the Virginia gubernatorial race and has been corrected accordingly

guardian.co.uk © 2013 Guardian News and Media Limited or its affiliated companies. All rights reserved. | Use of this content is subject to our Terms & Conditions | More Feeds

After Mark Sanford’s win, embarrassed pollsters go back to the drawing board | Harry J Enten

South Carolina’s special election made fools out of a few pollsters, but we’re better off with flawed data than none at all

Mark Sanford is heading off to Washington DC, while pollsters are heading back to their workshops. His victory by 9pt over Elizabeth Colbert Busch in South Carolina’s first district special election was surprisingly large. The last two public polls from Public Policy Polling (PPP) and Red Racing Horses (RRH) had Sanford winning by 1pt and a tie, respectively. PPP published a poll just two weeks before the election that had Colbert Busch winning by 9pt.

All three reports have joined the top 10 least accurate polls within two weeks of a special election, since 2004. PPP’s first poll was especially bad. It had an error of 18pt, which makes it the second least accurate poll taken two weeks before a special election since 2004. As my friend Mark Blumenthal points out, this first PPP survey had far too many African Americans as a percentage of the electorate. I don’t doubt that some white voters, a mostly Republican demographic in South Carolina’s first district, were disenchanted with Sanford by allegations that he violated the terms of his divorce, but the difference in the percentage of black voters was too great. Colbert Busch never had a lead of 9pt. One might wonder whether she even had a lead.

These errors might make people think twice about trusting PPP and RRH. After all, many major news organizations won’t cite PPP because it uses interactive voice response (IVR) technology instead of live interviewers, and because it doesn’t call cell phones. RRH doesn’t use live interviewers or call cells either, and it certainly doesn’t have a long track record; it’s apparently run by people who have no real background in polling. It wasn’t a surprise, therefore, that the RRH poll had women as too great a percentage of the electorate, at 60% versus the about 55% it should have been.

The truth, however, that PPP’s and RRH’s final polls seem to have been more accurate than the private (or internal) polls which are surveys produced by the parties and candidates. Most, though not all, use live interviewers and call mobile phones, and often, unreleased internal polls are more accurate than your average public poll. Most of the private polls for this race actually showed Colbert Busch holding a small lead.

All of this is to say that all the polling stunk it up in South Carolina’s first. Republican turnout wasn’t depressed as most thought it would be, and Republican voters did ultimately pull the lever for Sanford. Most of the undecided voters were Republicans, and there’s a reason PPP started to see that more white voters would vote than prior surveys indicated.

That’s why I will continue to pay attention to PPP and RRH in the future. Yes, PPP having Colbert Busch up nine was an embarrassment, but no one did better than PPP’s final or RRH’s only poll. The fact that even private pollsters fumbled so badly suggests that nobody who used better techniques could have been more accurate. Polling special elections just isn’t easy, as there really isn’t a baseline to understand who will turn out to vote and who will stay home.

Besides, the polling was useful, though imperfect. We knew that Sanford wouldn’t come close to replicating Mitt Romney’s 18pt win in the district, for instance. Even if the overall result was off, we learned some nuances of county polling, where the differences of support for each candidate were greater than expected. Thanks to RRH’s survey, we had a better idea on how counties would vote relative to each other, compared to the old method of just applying a uniform swing off the 2012 results. That doesn’t mean RRH is a great pollster, or even a particularly competent pollster, but it does suggest that almost any poll data can often be better than just going off the “fundamentals”.

It’s no surprise that you’re going to continue see websites like HuffPollster and Real Clear Politics report on surveys like PPP and RRH, which don’t meet the highest standards in the world. Polling data, even just okay data, can tell us a lot. In this case, the “flawed” public data was as good as the private data, and it was better than not looking at any polling at all.

guardian.co.uk © 2013 Guardian News and Media Limited or its affiliated companies. All rights reserved. | Use of this content is subject to our Terms & Conditions | More Feeds

Yes, Mark Sanford really has a chance to beat Elizabeth Colbert Busch | Harry J Enten

South Carolina’s special election may turn into a referendum on President Obama, which is bad news for Colbert Busch

Republican Mark Sanford has closed the gap with Democrat Elizabeth Colbert Busch in the South Carolina first district special election to take place on Tuesday. The latest poll from Public Policy Polling (PPP) has Sanford jumping into to a 1 pt lead 47% to 46% after being down in the same survey 9 pts just two weeks ago. A poll from Red Racing Horses (RRH) has Colbert Busch and Sanford tied at 46%.

A Sanford win in the final two weeks after being down 9 pts would be unusual. As I noted last week, only a little less than 15% of special election polls taken in the final two weeks of a campaign over the past decade have had a 9 pts or greater error margin. Even in the quick changing special Massachusetts senate election of 2010, Scott Brown gained only 4 pts in the final 10 days.

It’s not as if voters all of a sudden like Sanford, the former South Carolina Governor who is most well known nationally for cheating on his wife. Sanford’s less liked than Colbert Busch with a 43% favorable rating compared to 50%, respectively, per PPP. They just dislike President Obama. By a 4 pt margin, though, voters have a higher opinion of Sanford than President Obama. That reflects a district that voted for Republican Mitt Romney by 18 pts.

Therefore it shouldn’t be surprising that Sanford’s comeback is entirely built upon newfound Republican support likely gained by nationalizing the race. Some Republican voters are deciding that they would vote instead of staying home in disgust of Sanford’s affair and divorce. The electorate PPP now projects voted for Mitt Romney over President Obama by 13 pts versus a projected electorate of just a 5 pt edge for Romney in their last poll. Sanford has also expanded his lead among Romney voters from 49 pts two weeks ago to 61 pts now.

The good news for Colbert Busch is that most of Sanford’s comeback occurred about a week ago. You’ll note that RRH conducted their poll in the beginning to middle part of last week, while PPP’s was done over the weekend. Despite the differences in timing, the results are almost identical. That tends to indicate that Sanford has likely leveled off. The result is a race that is at this point simply too close to call with neither candidate having too much momentum.

So just how long will we have to wait for results on Tuesday Night? We should know by 9:30pm, if the primary for this election is any guide. The polls close at 7pm EST. During the primary this year, it took about 45 minutes (7:45pm EST) after the polls closed for results to start being reported. By 8pm, about 5% of precincts had posted their results. By 8:35pm, 50% of precincts were in. By 9:20pm, we had results for all but 1% of precincts.

Where are each of the candidates supposed to do best? The district is made up of five counties: Beaufort, Berkeley, Charleston, Colleston and Dorchester. In the 2012 house election for this seat, Republican Tim Scott got between 60% and 65% in all the counties except for Colleston. Colleston, however, makes up less than 1% of the district’s population.

The RRH poll forecasts that we should see greater differences between the counties in this election. In their poll, which had a tie, Colbert-Busch led by 4 pts in Charleston and 13 pts in Beaufort. Sanford grabbed a lead of 13 pts in Dorchester and 20 pts in Berkeley.

Usually this county breakdown would be bad news for Colbert Busch given the populations each of these counties make up in the district. The issue for Sanford is that some voters the more culturally conservative areas in Dorchester and Berkeley seem to still be staying home. That’s why we’re expecting to see an electorate that voted for Romney by 13 pts, not 18 pts. If these counties vote their population weight, Sanford likely wins by 3 to 4 pts.

Thus, the key for Republican Mark Sanford winning is either high turnout or over-performing the expected county breakdown. Whether or not this occurs is what will determine either Sanford or Democrat Elizabeth Colbert Busch wins tomorrow.

guardian.co.uk © 2013 Guardian News and Media Limited or its affiliated companies. All rights reserved. | Use of this content is subject to our Terms & Conditions | More Feeds

No Patriot Act II: Americans choose civil liberties over security laws | Harry J Enten

Unlike 9/11, the Boston attack will not lead to new anti-terror law. But Democrats are now less civil libertarian than Republicans

Terrorist attacks offer lawmakers an ability to react. After 9/11, the American government decided to go to war in Afghanistan and to enact new laws aimed at curbing future attacks. The Patriot Act, for instance, has been regarded by some as a necessary step for safety and by others as an infringement on civil liberties.

Following the Boston Marathon attack, we’ve heard Republicans Lindsey Graham and John McCain, among others, push for Dzhokhar Tsarnaev to be handled in a way that many believe would be a violation of his civil liberties. So, has the Boston bombing opened up an avenue for lawmakers to pursue controversial new anti-terrorism measures that may limit civil liberties?

Almost certainly not. The latest CNN/Time/ORC poll finds that 49% of Americans are not willing to give up civil liberties in order curb terrorism, while only 40% are. In fact, 61% of Americans are more fearful that the government will overreact to the Boston bombing, compared to 31% who are worried that the government won’t act strongly enough.

Other polls confirm these findings. Just after the attacks, Fox News found that 43% of Americans were willing to give up “some personal freedom” to reduce the threat of an attack, while 45% were not. A Washington Post poll, from before the bombers were caught, reported that only 41% of Americans were most worried that the government wouldn’t go far enough because of constitutional concerns. Almost half of Americans, 48%, were worried the government would go too far and compromise constitutional rights.

The reaction to Boston has been monumentally different to the polling results after 9/11. Immediately following the attacks on the WTC, 66% of Americans were willing to give up “civil liberties” to stop terrorism – 26pt higher than today. And 39% of Americans were concerned that strong laws wouldn’t be enacted, while 34% were more concerned about restricting civil liberties. That 4pt lead for enacting stronger laws is now a 30pt lead in favor of protecting civil liberties, per the ORC poll. After 9/11, 71% of Americans were willing to give up “personal freedom” to reduce the threat of a terrorist attack per Fox – 28pt higher than today.

Indeed, the party breakdown of new polling means that Graham and McCain have even less chance of getting their way. Democrats at large – who are unlikely to agree with hawkish senators – are now more willing to give up personal freedoms than Republicans. In the CNN/Time/ORC survey, 51% of Democrats were were willing to give up some civil liberties to curb terrorism, while only 41% of Republicans were. Fox found an identical 51% of Democrats were willing to give up “personal freedom”, against just 43% of Republicans. The Washington Post poll found the same 8pt spread between Democrats and Republicans on the question of whether the government might compromise constitutional rights.

Republicans, it seems, have become the standard-bearers of civil liberties due to two factors: who’s in the White House and shifting currents inside each party.

The executive branch, the government’s chief, is currently a Democrat – one who many Republicans believe, for instance, is out to take their guns. After 9/11, a Republican president held office, which likely accounts for the parties switching positions. We already know that a respondent or a politician will often oppose an issue or policy just because of who’s in charge.

Second, the Republican party is increasingly becoming the party of Rand Paul and civil libertarians. You would expect exactly these respondents to be against an intrusion on civil liberties. Many Paulites tend to call themselves independents, which would also explain why, in the CNN/Time/ORC and Fox News, independents were the least likely to give up personal freedoms, at 32% and 29%, respectively.

This puts hawkish Republicans like Graham and McCain in an awkward position within their own party. If there were a Republican in the White House, I think more Republicans would be willing to sacrifice civil liberties to prevent terrorism. At the same time, though, the Republican party simply is in a different place than it was a decade ago.

Overall, the chances of any major, hawkish changes in terrorist policy are significantly hampered by public opinion. Americans did not react to the Boston bombings with anything near the willingness to sacrifice civil liberties they showed after 9/11. That Republicans – usually hawkish on national security issues – are wary of giving power to the Democratic-run executive branch only further weakens the chances that any new law might pass.

guardian.co.uk © 2013 Guardian News and Media Limited or its affiliated companies. All rights reserved. | Use of this content is subject to our Terms & Conditions | More Feeds

No, Joe: even without the 2008 crash, McCain would not have beaten Obama | Harry J Enten

Vice-President Biden was probably just being kind – but the idea that Senator McCain might have won the 2008 election is bunk

US Vice-President Joe Biden says a lot of funny things. This weekend, the ever-hilarious veep said that John McCain “probably” would have defeated Barack Obama in the 2008 election – were it not for the global financial crisis following the Lehman Brothers bankruptcy on 15 September 2008. I disagree wholeheartedly with Biden’s assertion: Barack Obama would have been in a strong position to win 2008 even without the financial collapse.

Let’s start off with Obama’s most basic advantage at the time: the GOP had held the White House for the eight years before he ran. From 1952 through to 2004, there have been six instances of a party holding the White House for more than a term. In the subsequent election – after eight years with the president on their side – that party won only one of those those six elections. Whether the incumbent party held the White House for more than four years explains about 30% of the difference in vote margins in elections for over half a century, heading right into 2008.

These odds were made worse by an economy that was already lousy months before the crash. When averaging across multiple segments of the economy, growth was negative by early July. The only other year growth was negative across this many sectors was in 1980, per Nate Silver’s economic index. In that election, Jimmy Carter became the only president since the start of the 20th century to lose after taking the White House from the rival party – and he got blown out by 9pt.

This 2008 recession, combined with an increasingly disliked war in Iraq, caused President George W Bush’s approvals to plunge to almost 30% by 1 September 2008 – the lowest early September approval for any sitting president before a major election, going back all the way since modern scientific polling began in the 1930s. The only president who comes close is Truman, who had approval ratings around 32% and 33% going into the 1946 midterms and 1952 presidential election. His party lost control of both chambers of Congress and then the presidency.

Indeed, the fundamentals strongly indicate that Obama should have beaten McCain even before the financial collapse. Don’t believe me? Look at the models advanced by political scientists in August of 2008. Using a combination of economic measures, Bush’s approval ratings, Obama-McCain poll data, and a host of other factors, six of nine models written up in the journal of the American Political Science Association had Obama beating McCain. The combination models had Obama winning 80% of the time, by an average of 4pt.

Focusing only on the polling from Obama v McCain leads us to the same conclusion. From the time Obama clinched the Democratic nomination, in early June, through the announcement of his running mate, in late August, Pollster.com recorded 103 national polls. McCain led in six of them. Three of the polls showed a tie. That means that 91% of the polls conducted during the summer had Obama beating McCain, by an average of a slightly less than 4pt, and with the median of Obama winning by 3pt.

McCain did take the lead after the Republican National Convention, by an advantage of a little less than 2pt. A smart convention plan, along with the unexpected emergence of Sarah Palin, provided a temporary boost. But looking at the data, the boost was clearly not going to last. The Pollster.com aggregate (being its most sensitive in order to catch any micro-trends) already had Obama regaining the lead by 14pt in September.

The research seems to concur with a naive reading of the polls. Looking at wave studies on respondents’ choices before and after the collapse, two different studies both agree that Obama would have won without the collapse. Sunshine Hillygus and Michael Henderson (pdf) found that the collapse gave Obama a single point more. Richard Johnston, Emily Thorson, and Andrew Gooch (pdf) put the gain at 3pt. Neither gain was “decisive” in determining the winner the election (and my thanks to Nadia Hassan for sending me links to these studies).

Thus, I’m fairly confident that Obama would have won the 2008 election without the financial disaster of September 2008. Would he have won by less? Maybe, though not by much. The economy, already limping, and negative views about Iraq drove opinion on Bush down to record lows; McCain, as the candidate of Bush’s party, was hardpressed to overcome these obstacles, which gave him a deficit in the polls long before the Lehman’s collapse. McCain would almost certainly have lost to Obama even if the economy had not buckled towards the end of the election season.

guardian.co.uk © 2013 Guardian News and Media Limited or its affiliated companies. All rights reserved. | Use of this content is subject to our Terms & Conditions | More Feeds

It’ll take more than a presidential library to make George W Bush popular | Harry J Enten

Don’t be fooled by a new poll showing Americans think fondly of ‘Dubya’. Of recent presidents, only Richard Nixon was less liked

The George W Bush presidential library is opening this week. Not surprisingly, fans of the former president are out in full force trying to help with the rehabilitation’s of Bush’s image.

Bush, of course, ended his presidency with an approval rating around 30%. This previous low, combined with the library opening, has helped give an impression of elevation to a new ABC/Washington Post poll that put his retrospective job approval rating at 47%.

The fact that President Obama’s approval rating in the current HuffPollster aggregate is a similar 48% only helps with the intrigue. The Washington Free Beacon wrote an article titled “Dubya’s Approval Matches Obama’s”. Add on the fact that Obama’s approval rating and Bush’s retrospective approval on the economy approval rating are about the same, and you got the makings of a great press narrative.

Let me be the one who tries to nip this story in the bud. I have no clue what type of person George W Bush is, or how history will view him in the future. What I do know is that the 47% retrospective approval rating should be put in context.

First, retrospective approval ratings should almost never be compared to current job approval ratings. Humans have a tendency to remember their elected officials more fondly than they did when they left office. Back in 2010, Gallup asked Americans what their retrospective approval rating was for Presidents John F Kennedy through George W Bush. In every instance except for one, the retrospective approval was higher than the final approval was when they left office.

Most Republicans, for instance, love to make fun of Jimmy Carter. Carter was the only president of the 20th century to lose re-election after replacing a president of a different party. He left office with a 34% job approval rating. His retrospective job approval rating in the 2010 Gallup poll jumped by 18pt.

Second, Bush’s retrospective approval is the second worst among presidents in the last 50 years ago. To save you doing the math, Carter’s 52% approval rating is higher than Bush’s 47%. Only the Watergate-tainted Richard Nixon recorded a lower retrospective approval than Bush.

Most presidents have retrospective approval ratings above 60%. All but Nixon and Bush have +10 or better retrospective net approval ratings. Thus, not only did Bush tie for the second worst final approval rating while in office, but he is also has the second worst retrospective job approval rating.

Third, the Gallup data should make clear that George W Bush hasn’t seen much recovery over the past three years in his retrospective approval rating. The 47% in the latest Washington Post poll is the same as the 47% that Gallup found in 2010. The 50% disapproval now is nearly identical to the 51% three years ago. Yes, people view him more rosily now than they did during his presidency, but that effect has been baked in for a while now. He’s not getting more loved as time goes by.

Finally, the retrospective approval rating probably gives a false sense of how Americans view Bush now. Remember that a retrospective approval is exactly that – retrospective. The better way to view how Bush stands with the American public is his current favorable ratings. In the past two years, there have been five polls conducted that have asked about Bush’s favorable rating with the American public.

Bush currently holds an average -5pt net favorable rating with the American public. President Obama’s favorable rating is almost exactly the reverse of that, at +7pt in the HuffPollster aggregate. Indeed, even the losing Republican nominee Mitt Romney ended the 2012 campaign with a higher net favorable than Bush, at -3pt.

It’s no wonder that President Bush continues to hurt the Republican party. Mitt Romney tried his hardest to tie what many saw as a lackluster economy in 2012 to President Obama. The problem was that most Americans still blamed Bush over Obama. In the network exit polls, 53% said the economic problems were more Bush’s than Obama’s fault. Only 38% of Americans disagreed.

So, George W Bush may be more fondly thought of now than he was when he left office, yet this is to be expected – and discounted. Almost all presidents see a boost after they leave office. Bush is still quite unpopular compared with other former presidents, and his current favorable rating is far worse than President Obama’s. No library opening is changing that. Republicans would be wise to stay away from embracing George W Bush for the time being.

• This article has been amended to reflect updated polling data as of 25 April 2013.

guardian.co.uk © 2013 Guardian News and Media Limited or its affiliated companies. All rights reserved. | Use of this content is subject to our Terms & Conditions | More Feeds

Sanford v Colbert Busch: what are the odds for South Carolina’s first debate? | Harry J Enten

Debating a cardboard cutout has not helped the former governor against his opponent, and the polls show it

Republican Mark Sanford needs a breakthrough moment in tonight’s debate for South Carolina’s congressional special election, or else his comeback effort to win on 7 May is likely to fall short against Democrat Elizabeth Colbert Busch.

Sanford hopes to return to public office as a congressman, after resigning his in the wake of an extramarital affair for which he resigned from the governor’s mansion and left his wife. After winning the Republican nomination for the special election in South Carolina’s first district, Sanford has faced a barrage of bad press suggesting that he can do no good, even including the news that he violated the terms of his divorce with his former wife.

His Democratic opponent, businesswoman Elizabeth Colbert Busch, has her own claim to fame. Her brother, Stephen Colbert, has hosted numerous fundraisers on behalf of his sibling, helping her stay on the air in a district that went for Mitt Romney by over 15pt. She has also received support from a number of outside organizations, like the Democratic Congressional Campaign Committee.

The National Republican Congressional Committee, meanwhile, has abandoned Sanford. They refuse to air advertisements on his behalf, leaving the airways to his opponents. Sanford himself has little money left, and has been forced to rely on tactics such as a “debate” last week – with a cardboard cutout of Nancy Pelosi. He couldn’t afford podiums. Sanford used music stands instead. As a smart analyst told me, “It just feels like he’s going down.”

The one poll we have from the race shows Sanford trailing by 9pt, 50% to 41%. The poll, conducted last week by Public Policy Polling (PPP), shows an electorate that has Democrats as a greater proportion of the vote compared to PPP’s prior survey, which Colbert Busch led by 2pt. The new poll’s respondents voted for Romney over Obama by only a 5pt margin, compared to the 18pt margin Romney by which won the district last fall. PPP attributes this difference to low Republican enthusiasm, itself stemming from the most recent charges against Sanford.

Not everyone is buying PPP’s survey, though. Cook Political Report’s Dave Wasserman, for instance, thinks that Colbert Busch leads, but by a margin less than 9pt. My own math indicates that when you adjust PPP’s projected electorate to match its prior survey, Colbert Busch is ahead by a little over 3pt.

So just what are the chances that PPP’s poll is off by 9pt? Not very likely.

Over the past ten years, there have been 30 non-candidate/party sponsored polls conducted in the final two weeks before 18 off-cycle (i.e. not on election day during a midterm or presidential year) special congressional elections. Overall, these polls have been mostly accurate; 27 out of 30 times, the candidate leading in the poll won most of the votes in the special election. (Note: I only look at those elections where at least one Democrat and Republican are running. You can see the raw data here.)

Two of the three polls after in which the leading candidate didn’t win the election were for the 2009 New York 23rd special election, a unique race, in that there were three competitive candidates: Democrat Bill Owens, Republican Dede Scozzafava and Conservative Party candidate Doug Hoffman. In the week prior to the election, the Republican candidate dropped out, remained on the ballot, and endorsed the Democrat.

Not surprisingly, the 23rd special had PPP’s largest absolute error; they predicted Conservative Hoffman to win by 17pt. He lost by 2pt to the Democrat.

In 26 of the 30 special election surveys, the polling error was less than 9pt – Colbert Busch’s lead. The polls’ average error was only 4pt, and the median even lower, at 3pt.

These margins leave little room for Sanford. He would need an error of at least 9pt, which has happened only 13% of the time. He would also need that error to be in his favor, which means he’s looking at a 7% likelihood. That’s not outside the realm of possibility, but it’s a long shot. My guess is internal polling shows the same results, which is why the Cook Political Report and Rothenberg Political Report have put their finger on the scale for Colbert Busch.

The best comparison to this scenario would be Louisiana’s sixth special election in 2008. Louisiana’s sixth was like South Carolina’s first in that a Republican represented the district for over 30 years, and both districts were over 15pt more Republican on the presidential level than the nation as a whole in the prior presidential election. Like Sanford, Republican Woody Jenkins had run for statewide multiple times. Jenkins also had his own ethics problems: a connection with former KKK wizard and Louisiana politician David Duke.

The final poll before the Louisiana sixth special had Democrat Don Cazayoux ahead by 9pt, just like the recent poll that shows Democrat Colbert Busch leading. Jenkins did better than his numbers suggested. He didn’t lose by 9pt, but he did lose – by 3pt. Even though the poll overestimated Jenkins’ deficit, Cazayoux’s lead was too great to overcome.

Like in Louisiana’s sixth, there is a decent chance that the upstart Democrat’s lead is less than 9pt. It’s quite possible Republican voters will come around to Sanford on election day.

The chance, however, that the PPP poll is so off that Mark Sanford defeats Elizabeth Colbert Busch? Less than 15%, unless something happens very quickly.

guardian.co.uk © 2013 Guardian News and Media Limited or its affiliated companies. All rights reserved. | Use of this content is subject to our Terms & Conditions | More Feeds

Immigration reform won’t deliver a Latino voter ‘bonanza’ to Democrats | Harry J Enten

To assume that 11 million undocumented immigrants potentially eligible to vote will change US political arithmetic is erroneous

I have been quite skeptical about the ability of the Republican party to win over Latino voters. As others and I have noted, Latinos don’t vote Democratic just because of immigration policy. They vote Democratic because they are more ideologically “in sync” with the Democratic party. That’s why I’ve felt that going along with immigration reform was unlikely to net Republicans many Latino voters.

That said, I can’t agree either with the math in a Politico article titled “Immigration reform could be a bonanza for Democrats”. The article starts off promisingly enough with the premise that if immigration reform passed and undocumented immigrants became citizens, Latinos would start voting Democratic in even larger numbers. I can go along with this because the main reason anyone votes for or against a political party is for its economic platform, and 81% of first generation Latino immigrants say they want a “bigger government with more services”, compared to only 48% of Americans overall.

Politico then uses the commonly quoted figure of 11 million undocumented immigrants and claims that there would, therefore, be “up to” 11 million undocumented immigrants up for grabs if they all became citizens. If these 11 million then voted along the lines of the Latinos who cast a ballot in 2012, Obama would have won the national vote by 7pt instead of 4pt. He could have carried Arizona and even Texas, which were each won comfortably by Republican Mitt Romney.

The problem I have, though, is why would anyone use the 11 million figure for reference.

First off, 1 million undocumented immigrants are under the age of 18. I don’t care what your immigration status is, you can’t vote in United States presidential elections if you are under the age of 18.

Second, of the 10 million adults, 19% aren’t actually Latino; 11% are Asian. Asian voting patterns tend to be less stable than Latinos. In the past election, Asians went for President Obama by about 45pt. Twenty years ago, they went for Republican George HW Bush.

Without more research (and there is surprisingly little of it), it’s unclear to know how undocumented Asian voting patterns would change given immigration reform. Also, keep in mind that about 60% of these Asian immigrants are in California and Washington State alone – so they’re not exactly going to be a game-changer in the electoral college.

Third, I tend to doubt that all 8 million adult Latino undocumented immigrants would go for citizenship. A Latino Decisions poll says 87% of them would, but I’m skeptical. Only 60% of all legal immigrants actually apply for citizenship. Since the Immigration Reform and Control Act of 1986, only about 40% of the newly eligible immigrants by so-said act became citizens. It would be reasonable to expect a similar percentage this time.

Only 36% of Mexican immigrants who are eligible to become citizens have gone through the process. Mexicans make up 72% of all undocumented Latino immigrants. Six in ten of the other 28% of eligible Latino immigrants have become citizens.

All together, if undocumented Latino immigrants become citizens at the rate that eligible Latino immigrants do, then we’re looking at 3.5 million new voting-age citizens. That 3.5 million is a far cry from the 11 million we first talking about.

Finally, just how many of these 3.5 million undocumented Latino immigrants can be expected to vote in the presidential election? Per the standard census Current Population Survey (CPS), only 49.9% of all voting age Latino American citizens cast a ballot in 2008. (Note, there is no report available for 2012 as of this point.) Based on pre-election surveys and work by Michael McDonald, there is reason to believe that percentage may have dropped further in 2012.

All told, it would seem that only about 1.7 million new Latino voters would be added if undocumented immigrants were granted citizenship. Nationally, this would be a net of about 775,000 votes. This would increase Obama’s vote margin, but not to 7pt; it would only go up to about 4.4pt – in other words, half a point from where it actually was in November 2012. Even adding in new Asian voters, who vote at a lower rate than even Latinos, and other undocumented immigrants (and controlling for the percentage who apply for citizenship, percentage of citizens who vote, and the percentage who voted for Obama), the margin probably only goes up to, at most, 4.6pt.

The amount this would shift individual states in elections is debatable. Take Nevada, where, at last count, there were 190,000 undocumented immigrants – the highest percentage of any state population. Most of them are Latino. Apply the same math we did above, Obama would have gained about 17,000 votes. It would have increased his state margin of victory by 1.4pt. That’s not nothing, but we’re talking about the state with the largest percentage undocumented immigrants.

Most states aren’t close to Nevada’s undocumented immigrant population, while the ones that are simply aren’t competitive at the president level: Arizona, California, and New Jersey. No state in 2012 would have had a different outcome if undocumented immigrants were given the right to vote.

The truth of the matter is that passing immigration reform won’t be a votes “bonanza” for the Democratic party because of potentially or newly enfranchised undocumented immigrants. That doesn’t mean passing immigration reform will help the Republican party among Latinos; the GOP should probably still be worrying about its Latino voter appeal. But it’s not facing a landslide from a new citizen electorate.

guardian.co.uk © 2013 Guardian News and Media Limited or its affiliated companies. All rights reserved. | Use of this content is subject to our Terms & Conditions | More Feeds