Margin of Σrror

Has the Millennial generation ‘overwhelmed’ the electorate? | Harry J Enten

Harry J. Enten — Thu, 16 May 2013 16:16:10 +0000

Though they've drawn comparisons with the Greatest Generation for their liberal leanings, young people haven't taken over yet

The Millennials are the most Democratic cohort in a generation. Some believe attribute their liberal inclinations to the racially diverse demographics within their generation, and while that may be partially true, I prescribe the Millennials' (born after 1980 through the mid-90s) Democratic leanings to the fact they grew up during a strong Clinton administration and a weak Bush one. The Millennials today have mostly replaced the very white, very Democratic voters of the Greatest Generation (who were born 1910-1927) who came of age during the weak years under Hoover and a Roosevelt administration so strong it won FDR a third term.

In both the 2004 and 2012 election, almost all age cohorts voted the same relative to other cohorts, and the Millennials were as Democratic relative to the nation in 2004 and 2012. The Greatest Generation was too small a percentage of the electorate in 2012 to collect poll data, but they voted as Democratic as the Millennnials did in 2004.

Proving the cohort point further, the then +60-year-old Greatest Generation has been the most Democratic cohort all the way back in 1988, when age cohorts didn't differ all that much in how they voted – despite even some members of the Lost Generation still voting. The Greatest Generation crowd was the most Democratic in the 1992 election, as well.

Even the most ardent critics of the cohort theory will admit that a person's views of the presidential administration he or she grew up with will shape their political views going forward. The real question is whether or not the Millennials have a a large enough portion of the electorate to "overwhelm" the rest: would Bush have still won in 2004 with 2012 demographics, and would Obama have still won with 2004 demographics? With the release of the Census Bureau's Current Population Survey (CPS) voter supplement, we can find out.

The CPS is a geographically defined sample that seeks to fully represents each type of voter. Respondents are asked simple questions such as race, registration, and whether or not they voted. It is generally seen as a more comprehensive survey than network exit polls to determine the exact composition of the electorate.

There are, however, reasons to be cautious about making too much of the CPS report. The CPS doesn't have a perfect response rate, and as my friend Sean Trende points out, there are more people who claimed to the CPS to have voted than actually did. The CPS tabulators assume that those who didn't respond didn't vote, which is likely not true, but there's no perfect way to account for the discrepancies. Preliminary examination of board of election data from different states shows that fewer African Americans, for instance, voted than the numbers that the CPS finds.

Either way, the CPS is a very solid starting point, and we can still take a preliminary step in answering whether or not Obama would have won without the Millennials. We can determine this by multiplying the percent of what Obama won among Millennials by the percentage they made up of the electorate. In other words, the percentage of vote the Millennials contributed to Obama's margin of victory. Then we compare this with the percentage of the vote the Greatest Generation contributed in prior years, and we'll have a solid answer.

The Millennials now make up 18% of the electorate, per the CPS. That's less than the exit poll data reports, but exits have been known to count too many young voters. Meanwhile, the CPS data is backed quite well by Pew Research, which most would agree is one of the finest pollsters out there, if not the best.

Those born between 1910 and 1927 were just 2% of the 2012 electorate. In 1996, the election before the Millennials began voting, the Greatest made up 15% of everyone who cast a ballot. By the time 2004 rolled around, Millennials were 8% of the electorate, while the Greatest was down to 7%. Thus, as a percentage of the total electorate, there was a 5pt gain in the Democratic coalition of the Greatest Generation and Millennials from 1996 to 2012, and from 2004 to 2012.

Given Obama's +20pt win among Millennials, what percentage of the vote is that 5pt difference worth? With the 2004 electorate, Obama would have won by 2.6pt instead of 3.85pt last year. Meanwhile, if the 2004 electorate had looked like the 2012 one, George W Bush would have won by 1.2pt instead of 2.45pt. So yes, the Democratic candidate would have done slightly better with the demographic boost, but neither election would have turned out any differently.

Further comparisons to 1996 and 2004 undersell the Greatest Generation's impact. The Greatest were 17% of the electorate in 1992, 21% in 1988, 24% in 1984, and 27% in 1980, but have dropped steeply as more of their members pass away. The Millennials, meanwhile, are still far away from 24%, let alone 27%. If other age cohorts had voted the same relative to the national vote, Obama might have actually won by more in prior years.

To me, the evidence does not suggest the Millennials have "overwhelmed" the Greatest. They are, if anything, a new "Greatest Generation" both in terms of voting patterns and, to a lesser extent, size – though we can still expect the Millennials to grow somewhat as a percentage of the electorate, since people are more likely to vote as they get older. The question going forward is whether Generation Z (born in the mid-90s and later) will follow the voting patterns of the Millennials.

His small re-election margin and his projected historic ratings mean that the Obama administration has largely been seen as mediocre, which in turn suggests that the next age cohort will walk the middle of the road. The polling data agrees that Generation Z will be less Democratic, than the Millenials, but if one believes the racial diversity theory, then the next generation should actually be more liberal, as it will have fewer white voters.

We don't know whether the age cohort or racial diversity theory will end up being more correct in the long run. If it's the diversity one, Republicans are in a lot of trouble. If it's the age cohort theory, then it will be politics as usual. I'd bet on age.

Harry J Enten

The IRS and AP controversies spell trouble for Obama and Democrats | Harry J Enten

Harry J. Enten — Mon, 13 May 2013 22:23:58 +0000

Trust in government is a key factor in how people vote in midterm elections. All of that is heavily at stake now for 2014

The Justice Department has "secretly obtained" two months of conversations between Associated Press (AP) officials in a move called "unprecedented". The Internal Revenue Service (IRS) Cincinnati office singled out new conservative groups for extra scrutiny over the past couple of years. One of these controversies alone would have caused a headache for the Obama administration, but the two of them together could spell big trouble for the Democrats in the 2014 midterm elections.

Historically speaking, trust in government has been tied very closely to how people view the state of the economy. When consumer sentiment is up, trust in government goes up. When consumer confidence goes down, trust in government goes down. Pew Research has a very nice chart that illustrates this relationship.

You can see how the two lines generally flow together. This especially the case after 1970 – before which time trust in government was higher than it's been over the past 40 years. Right now consumer confidence is 76.4. That's down from earlier this year, but it's up significantly since 2010.

Trust in government isn't, however, always linked to the consumer sentiment. After the Watergate scandal, trust in government remained in a relatively low stable position through 1977, even as the economy improved. Trust in government fell in response to the House banking scandal in 1992 and Whitewater controversy of 1993 and 1994, as consumer confidence climbed. Finally, the economy was improving, but trust in government fell off its post-9/11 highs through the early part of the first decade of this century as Americans angered over the Iraq War and Hurricane Katrina that pounded the Gulf Coast.

Put another way, scandal can negatively impact how much Americans trust. It has to be a big scandal though. Benghazi, for instance, is likely not going to do it. Most Americans aren't paying attention to it, and as many Americans think the Republicans have gone too far as handled it appropriately.

The tax scandal, however, can play that role. My friends at NBC's First Read note

"The IRS story packs a bigger political punch... [and] will trigger new congressional hearings and new questions for the president and his team. More significantly, the IRS news is a political gift to a Republican Party whose base was strained on immigration."

The idea that the IRS would go after conservative groups, who hate the IRS, specifically seems to make a lot more sense than a president not wanting to create a foreign policy crisis in which be could benefit from a rally around the flag effect.

The obtaining of AP records likewise probably makes more sense in the voters' eyes. Obama has been critiqued for not doing enough press conferences or interviews with White House reporters. As one Democratic strategist put it, the "AP phone records thing just sealed the deal for what the newest narrative around Obama administration is going to be".

Indeed, these stories are coming at the perfect time for peak scandal coverages. Brendan Nyhan notes that scandals more often happen when the president is detested by members of the other party, as Obama is. Likewise, they are more likely to become big news when there aren't other news stories like the Boston bombings. Finally, scandals are more likely to take place in the beginning of the second term.

Therefore, the question is whether declining trust in the government has historically played a major factor in midterm elections. It turns out that it does. When trust in government falls, the party in the White House tends to do worse in midterm elections.

This 2010 graph from Pew Research illustrates the point well.

Trust fell dramatically in the lead up to the 1974 midterms thanks to Watergate, and the Republicans lost nearly 50 seats. Trust absolutely tumbled in the lead-up to the 1994 midterms, and Bill Clinton's Democratic Party lost over 50 seats. Democrats took back the house in 2006 as Americans trust in the Bush administration dropped. And although it isn't on the bottom chart, Americans trust in government, as seen in the top chart, was bad in 2010. Republicans gained 63 house seats.

On the reverse, there was minimal change in the composition in 1986, 1990, 1998, or 2002 when trust in government was relatively high.

So what does this mean for 2014? There are reasons to believe that Democrats shouldn't see major losses. The economy is doing better, and an incumbent president's party rarely loses big twice in midterms. Still, if trust falls, it may trump these structural factors. The thing to watch over the next days, weeks, and months is how big the scandals become. If they become big news, and that seems quite possible, Obama's Democratic party may be heading for major losses in 2014.

Harry J Enten

Polls at this point in the Virginia governor’s race tell us very little | Harry J Enten

Harry J. Enten — Fri, 10 May 2013 16:05:32 +0000

Virginia's governor race is being billed as a test case for 2016, but you can't read much into polling data this early

Virginia's 2013 gubernatorial election may not have particularly appealing candidates, but the race has taken on extra meaning because of the state's swing status in presidential elections. The race pits a so-far unlikable Democrat, Terry McAuliffe (aka T-Mac), against a exceptionally conservative Republican, attorney general Ken Cuccinelli (aka Cuch). Are swing state Virginians willing to elect a person, Cuccinelli, who wants to defund Planned Parenthood and has labeled the Environmental Protection Agency an "agency of mass destruction"? McAuliffe, on the other hand, was a long-time Clinton fundraiser and adviser, and his run is seen as a test of the Clinton name and organization for Hillary's possible run in 2016.

From two polls produced this week we can make similar conclusions with different results. An NBC/Marist poll has T-Mac up by 2pt among registered voters and down by 3pt among likely voters. A Washington Post survey has Cuch up 5pt among registered voters and 10pt among likely voters. You'll note that the overall results are different, but that the gap between the registered and likely voters is the same.

Right now, the truth is probably somewhere in the middle. Quinnpiac has polled registered voters three times since January, and each of their results fell in between the registered voter numbers from Marist and the Post. Likewise, a Christopher Newport University survey from January had a similar 4pt gap between registered voters and likely voters. Off-year elections tend to feature more Republican friendly electorates, given that African Americans and young people are a lower percentage of the electorate than in presidential year elections.

But does this current polling actually tell us anything about what will happen on election day? Not really. At this point in 2009, all but one scientific poll had eventual winner and current Republican Governor Bob McDonnell winning by 9pt or less. A few polls even had Democrat Creigh Deeds ahead by mid-June, but most had McDonnell ahead by about 5pt. With a few exceptions – notably from SurveyUSA, which consistently showed double-digit McDonnell leads by early August – most surveys continued to show a single-digit McDonnell lead through September. McDonnell won by 17pt.

You think that's unusual? Go back to the 2005. Through this point in the campaign, not a single poll had the eventual winner, Democrat Tim Kaine, ahead. A few polls had him down as much as 10pt, while the majority had him behind by mid-to-high single digits. Republican Jerry Kilgore trailed in only one public poll, all the way through early October! Tim Kaine, of course, went on to win by 6pt.

Overall, the polling at this point and through September has been off by about 10-15pt the last two elections. Given that Cuccinelli has a 4pt lead in the HuffPollster aggregate, either candidate could win. Perhaps Cuccinelli will win by a margin greater than 4pt, maybe less, maybe exactly that. Or maybe T-Mac will come out on top.

You might be tempted to look at the candidates' favorable ratings, but I wouldn't make too big a deal of those either, as they merely reflect the ballot standing of each man at this point. The important point is that both feature positive net favorables. Given that over 30% of voters have no opinion of Cuch, and over 40% have no opinion of T-Mac, the favorable ratings of these candidates will change. In 2005, Jerry Kilgore had a higher net favorable than Tim Kaine, which could be seen in the ballot test. He ended the election with a lower net favorable rating and lost. In 2009, Bob McDonnell had a net favorable rating equal to Creigh Deeds in early polling, which was reflected by a tight race in the ballot test. McDonnell ended up with a much higher net favorable rating, and indeed, he crushed Deeds.

What about the difference between the likely and registered voter results? The number one polling lesson from 2012 was that when likely and registered voters disagree in elections with high voter turnout, you should go with the registered voter results. Obviously, an off-year affair is not a high turnout election. Still, I was curious to see if over the past two cycles, pollsters have tightened the electorate, keeping in mind that only a few pollsters release results among both registered and likely voters, and most don't.

Among those who report results from different screens, it's not really clear that pollsters who use a likely voter screen are getting more accurate results than those that only polled registered voters. A 2009 October Virginia Commonwealth University poll had McDonnell winning by 18pt among likely voters and 16pt among registered voters. Neither was more accurate than the other. A 2009 October Roanoke College poll had McDonnell ahead by 17pt among likely voters and by 19pt among who said they were certain to vote (who we'll call "definite" for the sake of clarity). In this instance, the looser screen ended up being closer to the final result.

In 2005, tightening the electorate did pollsters no favors either. An October Diageo/Hotline poll found Kaine winning by 2pt among registered voters, 1pt among likely voters, and down by 2pt among definite voters. Kaine, as mentioned, won by 6pt. An October Roanoke College poll had Kaine winning by 8pt among probable voters and 10pt among definite voters. Again the looser screen performed better than the more selective one.

What about earlier in the campaign? In 2009, an August Washington Post poll had McDonnell up by 15pt among likely voters and by 7pt among registered voters. Clearly, the likely voter result was closer to the truth – in this case, at least. In 2005, the opposite occurred. An early September Washington Post poll had Kilgore up by 7pt among likely voters and by 4pt among registered voters. Earlier in the campaign, SurveyUSA showed a dramatic rise in Kilgore's lead from 4pt to 10pt between May and June when switching between registered and likely voters. Other polling companies showed a consistent race, which means that the far less accurate June result was almost certainly because of a change in the electorate being polled.

Thus, I'm not really sure there's much to take from the recent polling data on the Virginia gubernatorial race. Yes, Ken Cuccinelli likely has a small lead at the moment, but that edge probably means very little. The difference between the registered and likely voter screen may be instructive, but we can't yet be sure when to trust it. Tighter voter screens have not proven to necessarily be more accurate in either the 2005 or 2009 Virginia gubernatorial elections.

We can only sit back and allow the campaign to unfold.

• A previous version of this article incorrectly stated the leader of The Washington Post's survey of the Virginia gubernatorial race and has been corrected accordingly

Harry J Enten

After Mark Sanford’s win, embarrassed pollsters go back to the drawing board | Harry J Enten

Harry J. Enten — Wed, 08 May 2013 14:01:39 +0000

South Carolina's special election made fools out of a few pollsters, but we're better off with flawed data than none at all

Mark Sanford is heading off to Washington DC, while pollsters are heading back to their workshops. His victory by 9pt over Elizabeth Colbert Busch in South Carolina's first district special election was surprisingly large. The last two public polls from Public Policy Polling (PPP) and Red Racing Horses (RRH) had Sanford winning by 1pt and a tie, respectively. PPP published a poll just two weeks before the election that had Colbert Busch winning by 9pt.

All three reports have joined the top 10 least accurate polls within two weeks of a special election, since 2004. PPP's first poll was especially bad. It had an error of 18pt, which makes it the second least accurate poll taken two weeks before a special election since 2004. As my friend Mark Blumenthal points out, this first PPP survey had far too many African Americans as a percentage of the electorate. I don't doubt that some white voters, a mostly Republican demographic in South Carolina's first district, were disenchanted with Sanford by allegations that he violated the terms of his divorce, but the difference in the percentage of black voters was too great. Colbert Busch never had a lead of 9pt. One might wonder whether she even had a lead.

These errors might make people think twice about trusting PPP and RRH. After all, many major news organizations won't cite PPP because it uses interactive voice response (IVR) technology instead of live interviewers, and because it doesn't call cell phones. RRH doesn't use live interviewers or call cells either, and it certainly doesn't have a long track record; it's apparently run by people who have no real background in polling. It wasn't a surprise, therefore, that the RRH poll had women as too great a percentage of the electorate, at 60% versus the about 55% it should have been.

The truth, however, that PPP's and RRH's final polls seem to have been more accurate than the private (or internal) polls which are surveys produced by the parties and candidates. Most, though not all, use live interviewers and call mobile phones, and often, unreleased internal polls are more accurate than your average public poll. Most of the private polls for this race actually showed Colbert Busch holding a small lead.

All of this is to say that all the polling stunk it up in South Carolina's first. Republican turnout wasn't depressed as most thought it would be, and Republican voters did ultimately pull the lever for Sanford. Most of the undecided voters were Republicans, and there's a reason PPP started to see that more white voters would vote than prior surveys indicated.

That's why I will continue to pay attention to PPP and RRH in the future. Yes, PPP having Colbert Busch up nine was an embarrassment, but no one did better than PPP's final or RRH's only poll. The fact that even private pollsters fumbled so badly suggests that nobody who used better techniques could have been more accurate. Polling special elections just isn't easy, as there really isn't a baseline to understand who will turn out to vote and who will stay home.

Besides, the polling was useful, though imperfect. We knew that Sanford wouldn't come close to replicating Mitt Romney's 18pt win in the district, for instance. Even if the overall result was off, we learned some nuances of county polling, where the differences of support for each candidate were greater than expected. Thanks to RRH's survey, we had a better idea on how counties would vote relative to each other, compared to the old method of just applying a uniform swing off the 2012 results. That doesn't mean RRH is a great pollster, or even a particularly competent pollster, but it does suggest that almost any poll data can often be better than just going off the "fundamentals".

It's no surprise that you're going to continue see websites like HuffPollster and Real Clear Politics report on surveys like PPP and RRH, which don't meet the highest standards in the world. Polling data, even just okay data, can tell us a lot. In this case, the "flawed" public data was as good as the private data, and it was better than not looking at any polling at all.

Harry J Enten

Why Chris Christie’s weight loss could actually work against him | Harry Enten

Harry J. Enten — Tue, 07 May 2013 16:35:42 +0000

Polls have shown surprisingly that when it comes to male candidates, voters look more favorably on overweight aspirants

Sometimes, Washington DC seems like Hollywood for nerds. No clearer was that the case than the media's reaction to the news that New Jersey governor Chris Christie submitted himself to lapband weight-loss surgery. Never mind that Christie said he was simply losing weight to be healthy for the sake of his family. The first question the press, as they would in Hollywood, asked is how being slimmer would impact Christie's future career. So do Christie's chance of becoming president improve because he is carrying fewer pounds?

Some of us may be too young to remember that the press wondered if Christie was "too fat" to be governor from New Jersey. In fact, it was the subject of a Newsweek article and Democrat Jon Corzine's attacks in 2009. Back then, 78% of voters said that Christie's weight made no difference in their vote. You might think that the voters were lying until you realize that Christie became arguably the most conservative Republican governor from New Jersey in modern history. He did so while winning a larger percentage of the vote than any Republican candidate for governor in New Jersey in 24 years.

Recent polling backs up the older polling. A Quinnipiac poll taken in March 2013 of New Jersey voters found that only 21% of voters had any reservations about a generic candidate's weight. A September 2012 Quinnpiac poll showed that 84% of New Jersey voters said it wouldn't make a difference in their vote if a generic candidate was overweight. In New York, the percentage rose to 88%, so it's not just a home state bias. Even after an asthma attack in 2011, only 18% of New Jersey voters said they were worried about Christie's weight. Keep in mind that all of this New Jersey polling took place when Christie had a far lower approval rating than he sports now.

There are also signs that attacks on Christie's weight backfire. I'm not just talking about the fact that Jon Corzine lost in 2009. I'm talking about an October 2011 Quinnipiac poll that showed that 71% of New Jersey voters said that jokes about Christie's weight were in bad taste. This polling was consistent across political parties. Moreover, 79% of women, who Republicans have a problem with, thought that the jokes were in bad taste. Most were voters willing to go farther than just "bad taste": 72% agreed with Christie that political commentators who brought up his weight were "ignorant".

Some might say that the weight issue would be different in other states. I don't buy it. New Jersey and New York are two of the slimmest states in the nation with obesity rates of less than 25%. If weight were an issue, we'd expect to be in these states. I would think it would be far less of an issue in the battleground states of the midwest given that 25% to, in some cases, over 30% of the population is overweight in these states. In a Republican primary, Christie's weight shouldn't deter him from winning southern states given that the majority have obesity rates over 30%.

Indeed, some of the livelier southern politicians of our day were at least at some point overweight while in office. Anyone remember Bill Clinton's weight problem? His McDonald's excursions didn't stop him from becoming president. Haley Barbour was a well-liked two-term Republican governor of Mississippi. Mike Huckabee did slim down for health reasons before his presidential run, yet was popular as an overweight governor. Newt Gingrich's health would likely benefit from losing a few pounds, but I think most would agree he didn't lose in 2012 because of weight.

Examples and polling aside, you might think that these politicians' weight did hurt them. That is, they would have been even more popular if they were slimmer. Given the polling, you'd have to believe the voters were unwilling to admit that weight kept them from pulling the lever for these overweight candidates.

The good news is that we actually have scientific research that seeks to control for this potential social desirability bias. A 2010 University of Missouri-Kansas City study looked at how people reacted to pictures of a potential candidate of normal weight. This control group was compared to other respondents who were shown pictures of the same candidate, except the candidate's picture was morphed to be obese. Everything else about the candidate including political affiliation, views, and background remained the same.

And did the candidate's weight make a difference? If the candidate was female, extra weight was a small negative. That concurs with a survey this year by Lake Research that found the mere mention of a woman candidate's physical appearance hurt them. This was especially the case when the appearance coverage was unflattering.

For men, however, the 2010 University of Missouri study found that being obese was not a negative. It was actually a large positive! Respondents were over 20 points more likely to have a warmer feeling towards the same male candidate if he were obese than if were skinny. The obese candidate was 10% better liked than if he were skinny. The obese candidate was also thought to be more intelligent than the skinny one.

These positives for males make sense if you think about it. Society trains us to think this way. To bring it back to Hollywood, look at the Nutty Professor with Eddie Murphy. Murphy's character Professor Klump was obese. He was, however, smart as a whip and very likable. Klump's skinny equivalent Buddy Love was not nearly as smart and was a jerk. At the end of both Nutty Professor 1 and 2, the audience is rooting for Klump to beat his arch nemesis, and he does. Meanwhile, Klump doesn't get together with an overweight woman. Instead, he ends up with characters played by the very attractive Jada Pinkett Smith in the first movie and Janet Jackson in the second one. These characters happen to be very kind and intelligent too.

When you take all the evidence into account, Chris Christie doesn't need to lose weight to become president. Voters say they don't care about weight, and their actions back them up. The research says that Christie's electoral prospects might be better off keeping on the pounds. For now, can't we just be happy that Christie wants to be a healthier individual?

Harry J Enten

Yes, Mark Sanford really has a chance to beat Elizabeth Colbert Busch | Harry J Enten

Harry J. Enten — Mon, 06 May 2013 16:40:50 +0000

South Carolina's special election may turn into a referendum on President Obama, which is bad news for Colbert Busch

Republican Mark Sanford has closed the gap with Democrat Elizabeth Colbert Busch in the South Carolina first district special election to take place on Tuesday. The latest poll from Public Policy Polling (PPP) has Sanford jumping into to a 1 pt lead 47% to 46% after being down in the same survey 9 pts just two weeks ago. A poll from Red Racing Horses (RRH) has Colbert Busch and Sanford tied at 46%.

A Sanford win in the final two weeks after being down 9 pts would be unusual. As I noted last week, only a little less than 15% of special election polls taken in the final two weeks of a campaign over the past decade have had a 9 pts or greater error margin. Even in the quick changing special Massachusetts senate election of 2010, Scott Brown gained only 4 pts in the final 10 days.

It's not as if voters all of a sudden like Sanford, the former South Carolina Governor who is most well known nationally for cheating on his wife. Sanford's less liked than Colbert Busch with a 43% favorable rating compared to 50%, respectively, per PPP. They just dislike President Obama. By a 4 pt margin, though, voters have a higher opinion of Sanford than President Obama. That reflects a district that voted for Republican Mitt Romney by 18 pts.

Therefore it shouldn't be surprising that Sanford's comeback is entirely built upon newfound Republican support likely gained by nationalizing the race. Some Republican voters are deciding that they would vote instead of staying home in disgust of Sanford's affair and divorce. The electorate PPP now projects voted for Mitt Romney over President Obama by 13 pts versus a projected electorate of just a 5 pt edge for Romney in their last poll. Sanford has also expanded his lead among Romney voters from 49 pts two weeks ago to 61 pts now.

The good news for Colbert Busch is that most of Sanford's comeback occurred about a week ago. You'll note that RRH conducted their poll in the beginning to middle part of last week, while PPP's was done over the weekend. Despite the differences in timing, the results are almost identical. That tends to indicate that Sanford has likely leveled off. The result is a race that is at this point simply too close to call with neither candidate having too much momentum.

So just how long will we have to wait for results on Tuesday Night? We should know by 9:30pm, if the primary for this election is any guide. The polls close at 7pm EST. During the primary this year, it took about 45 minutes (7:45pm EST) after the polls closed for results to start being reported. By 8pm, about 5% of precincts had posted their results. By 8:35pm, 50% of precincts were in. By 9:20pm, we had results for all but 1% of precincts.

Where are each of the candidates supposed to do best? The district is made up of five counties: Beaufort, Berkeley, Charleston, Colleston and Dorchester. In the 2012 house election for this seat, Republican Tim Scott got between 60% and 65% in all the counties except for Colleston. Colleston, however, makes up less than 1% of the district's population.

The RRH poll forecasts that we should see greater differences between the counties in this election. In their poll, which had a tie, Colbert-Busch led by 4 pts in Charleston and 13 pts in Beaufort. Sanford grabbed a lead of 13 pts in Dorchester and 20 pts in Berkeley.

Usually this county breakdown would be bad news for Colbert Busch given the populations each of these counties make up in the district. The issue for Sanford is that some voters the more culturally conservative areas in Dorchester and Berkeley seem to still be staying home. That's why we're expecting to see an electorate that voted for Romney by 13 pts, not 18 pts. If these counties vote their population weight, Sanford likely wins by 3 to 4 pts.

Thus, the key for Republican Mark Sanford winning is either high turnout or over-performing the expected county breakdown. Whether or not this occurs is what will determine either Sanford or Democrat Elizabeth Colbert Busch wins tomorrow.

Harry J Enten

How good is ADP at forecasting the monthly BLS jobs reports? | Harry J Enten

Harry J. Enten — Sat, 04 May 2013 13:00:02 +0000

With official jobs data seen as a key economic metric, no wonder other agencies second-guess them. But 'guess' is about right

The government reported 165,000 new jobs created according April's nonfarm payroll numbers (176,000 in total in the private sector) – a pleasant surprise to most economists, who were anticipating fewer. Part of the reason that expectations were off was because the Automatic Data Processing (ADP) jobs report predicted that only 119,000 would be created, an apparent error of 57,000. Why is this discrepancy a big deal?

Jobs reports used to be exciting only for economists and stockbrokers, but since the election season, every political junkie and their dog seems to have taken an interest. People recognize that the economy plays a vital role in deciding votes; these reports, therefore, offer a vital clue to predicting the politicians' election chances. So, now we have both the economic and political class yearning for 8.30am on the first Friday of the month, all to learn about the jobs numbers.

But as in so many arenas in America, people can't wait to see what happens. They race to get the answer as quickly as they can, picking up on whatever clues they deem fit. Enter the ADP jobs report, a jobs survey released two days before the official Bureau of Labor Statistics (BLS) government report. Many use the ADP to predict the BLS, but past ADP surveys have sometimes been far off actual BLS results. As Steven Russolillo noted in September, "some months, it's spot on; others it's wildly off base."

The ADP, hoping to make its data more accurate, made some major changes for its October 2012 report. That month, the ADP started using ADP payroll data, BLS employment data, and the Philadelphia Federal Reserve's Aruoba-Diebold-Scott Business Conditions Index. As a result, ADP surveyed 62,000 more clients than previously, 2 million more employees, and two more company-size classes and industries. They brought on Moody's Analytics to replace Macroeconomic Advisers for processing data. To put it mildly, these are not small changes.

Have the adjustments brought ADP any closer to solving the monthly jobs mystery?

To answer this question, I've compared ADP forecasts of the past seven months with the same seven-month period last year, and looked at the ADP's accuracy in predicting final BLS numbers. The BLS produces an initial, second, and final report as it calculates more data, and the data between reports can differ greatly. The ADP wants to land as close as possible to the BLS's final report, though most attention is usually paid to the initial report.

The past seven months have seen an average difference between the ADP and initial BLS report of 42,286 jobs. (You can see all the data here.) Some months, such as October and November 2012, had errors of under 30,000 jobs, while March and April 2013 saw errors of 57,000 or greater. No month had an initial error of less than 26,000; the error in margin ends up within +/-19,000 of 45,000.

Compared to the same time last year, the average error has, in fact, diminished. Last year, ADP was off by an average error of 55,429 jobs, which is 13,143 jobs greater than their more recent average. This difference, however, is not statistically significant, due to a small sample size (seven observations) and the fact that the old ADP results could sometimes be very accurate.

Last year, three months under the old methods had errors of 17,000 or less, compared to the initial BLS report – far more accurate than any month per the new ADP. The problem for the old ADP was that four months last year had errors of 66,000 or greater, which less accurate than all seven months of the new ADP.

In that light, the new ADP does look better than the old. When it comes to their forecasts and the initial jobs report, we still haven't seen an error so wrong it makes your eyes pop out. Of course, we haven't seen stunning accuracy either.

The BLS's final jobs report, however – what ADP should supposedly be best at predicting – apparently confounds ADP. We see zero consistency in their results. Four out of six final reports (or second report for March 2013, since we don't have the final one yet) have had errors of 28,000 or less. Two final reports, December 2012 and March 2013, have been within 4,000. November 2012 and February 2013, though, have seen errors of over 120,000 jobs! The old ADP, by comparison, had its biggest miss last year, in January 2012, at 107,000 jobs.

The average error of the new ADP on final BLS reports has been 51,167 jobs, which is actually worse than the ADP's error on initial BLS reports. It's better than the average error of 58,333 from last year, but it's not better by a statistically significant amount.

When you put it all together, I can't really say that ADP has done better with its new methodology than it did with its old. There are some signs that the changes have made it more accurate – perhaps those huge misses of November 2012 and February 2013 will turn out to be anomalies – but we'll need a larger sample size to know for sure. But at this point, it looks as likely as ever that the ADP numbers will be way off-the-mark measured against the BLS's final reports.

The smart bet right now? Have a little patience and wait for the actual government statistics.

Harry J Enten

No Patriot Act II: Americans choose civil liberties over security laws | Harry J Enten

Harry J. Enten — Fri, 03 May 2013 14:30:01 +0000

Unlike 9/11, the Boston attack will not lead to new anti-terror law. But Democrats are now less civil libertarian than Republicans

Terrorist attacks offer lawmakers an ability to react. After 9/11, the American government decided to go to war in Afghanistan and to enact new laws aimed at curbing future attacks. The Patriot Act, for instance, has been regarded by some as a necessary step for safety and by others as an infringement on civil liberties.

Following the Boston Marathon attack, we've heard Republicans Lindsey Graham and John McCain, among others, push for Dzhokhar Tsarnaev to be handled in a way that many believe would be a violation of his civil liberties. So, has the Boston bombing opened up an avenue for lawmakers to pursue controversial new anti-terrorism measures that may limit civil liberties?

Almost certainly not. The latest CNN/Time/ORC poll finds that 49% of Americans are not willing to give up civil liberties in order curb terrorism, while only 40% are. In fact, 61% of Americans are more fearful that the government will overreact to the Boston bombing, compared to 31% who are worried that the government won't act strongly enough.

Other polls confirm these findings. Just after the attacks, Fox News found that 43% of Americans were willing to give up "some personal freedom" to reduce the threat of an attack, while 45% were not. A Washington Post poll, from before the bombers were caught, reported that only 41% of Americans were most worried that the government wouldn't go far enough because of constitutional concerns. Almost half of Americans, 48%, were worried the government would go too far and compromise constitutional rights.

The reaction to Boston has been monumentally different to the polling results after 9/11. Immediately following the attacks on the WTC, 66% of Americans were willing to give up "civil liberties" to stop terrorism – 26pt higher than today. And 39% of Americans were concerned that strong laws wouldn't be enacted, while 34% were more concerned about restricting civil liberties. That 4pt lead for enacting stronger laws is now a 30pt lead in favor of protecting civil liberties, per the ORC poll. After 9/11, 71% of Americans were willing to give up "personal freedom" to reduce the threat of a terrorist attack per Fox – 28pt higher than today.

Indeed, the party breakdown of new polling means that Graham and McCain have even less chance of getting their way. Democrats at large – who are unlikely to agree with hawkish senators – are now more willing to give up personal freedoms than Republicans. In the CNN/Time/ORC survey, 51% of Democrats were were willing to give up some civil liberties to curb terrorism, while only 41% of Republicans were. Fox found an identical 51% of Democrats were willing to give up "personal freedom", against just 43% of Republicans. The Washington Post poll found the same 8pt spread between Democrats and Republicans on the question of whether the government might compromise constitutional rights.

Republicans, it seems, have become the standard-bearers of civil liberties due to two factors: who's in the White House and shifting currents inside each party.

The executive branch, the government's chief, is currently a Democrat – one who many Republicans believe, for instance, is out to take their guns. After 9/11, a Republican president held office, which likely accounts for the parties switching positions. We already know that a respondent or a politician will often oppose an issue or policy just because of who's in charge.

Second, the Republican party is increasingly becoming the party of Rand Paul and civil libertarians. You would expect exactly these respondents to be against an intrusion on civil liberties. Many Paulites tend to call themselves independents, which would also explain why, in the CNN/Time/ORC and Fox News, independents were the least likely to give up personal freedoms, at 32% and 29%, respectively.

This puts hawkish Republicans like Graham and McCain in an awkward position within their own party. If there were a Republican in the White House, I think more Republicans would be willing to sacrifice civil liberties to prevent terrorism. At the same time, though, the Republican party simply is in a different place than it was a decade ago.

Overall, the chances of any major, hawkish changes in terrorist policy are significantly hampered by public opinion. Americans did not react to the Boston bombings with anything near the willingness to sacrifice civil liberties they showed after 9/11. That Republicans – usually hawkish on national security issues – are wary of giving power to the Democratic-run executive branch only further weakens the chances that any new law might pass.

Harry J Enten

No, Joe: even without the 2008 crash, McCain would not have beaten Obama | Harry J Enten

Harry J. Enten — Tue, 30 Apr 2013 21:41:57 +0000

Vice-President Biden was probably just being kind – but the idea that Senator McCain might have won the 2008 election is bunk

US Vice-President Joe Biden says a lot of funny things. This weekend, the ever-hilarious veep said that John McCain "probably" would have defeated Barack Obama in the 2008 election – were it not for the global financial crisis following the Lehman Brothers bankruptcy on 15 September 2008. I disagree wholeheartedly with Biden's assertion: Barack Obama would have been in a strong position to win 2008 even without the financial collapse.

Let's start off with Obama's most basic advantage at the time: the GOP had held the White House for the eight years before he ran. From 1952 through to 2004, there have been six instances of a party holding the White House for more than a term. In the subsequent election – after eight years with the president on their side – that party won only one of those those six elections. Whether the incumbent party held the White House for more than four years explains about 30% of the difference in vote margins in elections for over half a century, heading right into 2008.

These odds were made worse by an economy that was already lousy months before the crash. When averaging across multiple segments of the economy, growth was negative by early July. The only other year growth was negative across this many sectors was in 1980, per Nate Silver's economic index. In that election, Jimmy Carter became the only president since the start of the 20th century to lose after taking the White House from the rival party – and he got blown out by 9pt.

This 2008 recession, combined with an increasingly disliked war in Iraq, caused President George W Bush's approvals to plunge to almost 30% by 1 September 2008 – the lowest early September approval for any sitting president before a major election, going back all the way since modern scientific polling began in the 1930s. The only president who comes close is Truman, who had approval ratings around 32% and 33% going into the 1946 midterms and 1952 presidential election. His party lost control of both chambers of Congress and then the presidency.

Indeed, the fundamentals strongly indicate that Obama should have beaten McCain even before the financial collapse. Don't believe me? Look at the models advanced by political scientists in August of 2008. Using a combination of economic measures, Bush's approval ratings, Obama-McCain poll data, and a host of other factors, six of nine models written up in the journal of the American Political Science Association had Obama beating McCain. The combination models had Obama winning 80% of the time, by an average of 4pt.

Focusing only on the polling from Obama v McCain leads us to the same conclusion. From the time Obama clinched the Democratic nomination, in early June, through the announcement of his running mate, in late August, Pollster.com recorded 103 national polls. McCain led in six of them. Three of the polls showed a tie. That means that 91% of the polls conducted during the summer had Obama beating McCain, by an average of a slightly less than 4pt, and with the median of Obama winning by 3pt.

McCain did take the lead after the Republican National Convention, by an advantage of a little less than 2pt. A smart convention plan, along with the unexpected emergence of Sarah Palin, provided a temporary boost. But looking at the data, the boost was clearly not going to last. The Pollster.com aggregate (being its most sensitive in order to catch any micro-trends) already had Obama regaining the lead by 14pt in September.

The research seems to concur with a naive reading of the polls. Looking at wave studies on respondents' choices before and after the collapse, two different studies both agree that Obama would have won without the collapse. Sunshine Hillygus and Michael Henderson (pdf) found that the collapse gave Obama a single point more. Richard Johnston, Emily Thorson, and Andrew Gooch (pdf) put the gain at 3pt. Neither gain was "decisive" in determining the winner the election (and my thanks to Nadia Hassan for sending me links to these studies).

Thus, I'm fairly confident that Obama would have won the 2008 election without the financial disaster of September 2008. Would he have won by less? Maybe, though not by much. The economy, already limping, and negative views about Iraq drove opinion on Bush down to record lows; McCain, as the candidate of Bush's party, was hardpressed to overcome these obstacles, which gave him a deficit in the polls long before the Lehman's collapse. McCain would almost certainly have lost to Obama even if the economy had not buckled towards the end of the election season.

Harry J Enten

It’ll take more than a presidential library to make George W Bush popular | Harry J Enten

Harry J. Enten — Tue, 30 Apr 2013 13:02:32 +0000

Don't be fooled by a new poll showing Americans think fondly of 'Dubya'. Of recent presidents, only Richard Nixon was less liked

The George W Bush presidential library is opening this week. Not surprisingly, fans of the former president are out in full force trying to help with the rehabilitation's of Bush's image.

Bush, of course, ended his presidency with an approval rating around 30%. This previous low, combined with the library opening, has helped give an impression of elevation to a new ABC/Washington Post poll that put his retrospective job approval rating at 47%.

The fact that President Obama's approval rating in the current HuffPollster aggregate is a similar 48% only helps with the intrigue. The Washington Free Beacon wrote an article titled "Dubya's Approval Matches Obama's". Add on the fact that Obama's approval rating and Bush's retrospective approval on the economy approval rating are about the same, and you got the makings of a great press narrative.

Let me be the one who tries to nip this story in the bud. I have no clue what type of person George W Bush is, or how history will view him in the future. What I do know is that the 47% retrospective approval rating should be put in context.

First, retrospective approval ratings should almost never be compared to current job approval ratings. Humans have a tendency to remember their elected officials more fondly than they did when they left office. Back in 2010, Gallup asked Americans what their retrospective approval rating was for Presidents John F Kennedy through George W Bush. In every instance except for one, the retrospective approval was higher than the final approval was when they left office.

Most Republicans, for instance, love to make fun of Jimmy Carter. Carter was the only president of the 20th century to lose re-election after replacing a president of a different party. He left office with a 34% job approval rating. His retrospective job approval rating in the 2010 Gallup poll jumped by 18pt.

Second, Bush's retrospective approval is the second worst among presidents in the last 50 years ago. To save you doing the math, Carter's 52% approval rating is higher than Bush's 47%. Only the Watergate-tainted Richard Nixon recorded a lower retrospective approval than Bush.

Most presidents have retrospective approval ratings above 60%. All but Nixon and Bush have +10 or better retrospective net approval ratings. Thus, not only did Bush tie for the second worst final approval rating while in office, but he is also has the second worst retrospective job approval rating.

Third, the Gallup data should make clear that George W Bush hasn't seen much recovery over the past three years in his retrospective approval rating. The 47% in the latest Washington Post poll is the same as the 47% that Gallup found in 2010. The 50% disapproval now is nearly identical to the 51% three years ago. Yes, people view him more rosily now than they did during his presidency, but that effect has been baked in for a while now. He's not getting more loved as time goes by.

Finally, the retrospective approval rating probably gives a false sense of how Americans view Bush now. Remember that a retrospective approval is exactly that – retrospective. The better way to view how Bush stands with the American public is his current favorable ratings. In the past two years, there have been five polls conducted that have asked about Bush's favorable rating with the American public.

Bush currently holds an average -5pt net favorable rating with the American public. President Obama's favorable rating is almost exactly the reverse of that, at +7pt in the HuffPollster aggregate. Indeed, even the losing Republican nominee Mitt Romney ended the 2012 campaign with a higher net favorable than Bush, at -3pt.

It's no wonder that President Bush continues to hurt the Republican party. Mitt Romney tried his hardest to tie what many saw as a lackluster economy in 2012 to President Obama. The problem was that most Americans still blamed Bush over Obama. In the network exit polls, 53% said the economic problems were more Bush's than Obama's fault. Only 38% of Americans disagreed.

So, George W Bush may be more fondly thought of now than he was when he left office, yet this is to be expected – and discounted. Almost all presidents see a boost after they leave office. Bush is still quite unpopular compared with other former presidents, and his current favorable rating is far worse than President Obama's. No library opening is changing that. Republicans would be wise to stay away from embracing George W Bush for the time being.

• This article has been amended to reflect updated polling data as of 25 April 2013.

Harry J Enten