Polls at this point in the Virginia governor’s race tell us very little | Harry J Enten

Virginia’s governor race is being billed as a test case for 2016, but you can’t read much into polling data this early

Virginia’s 2013 gubernatorial election may not have particularly appealing candidates, but the race has taken on extra meaning because of the state’s swing status in presidential elections. The race pits a so-far unlikable Democrat, Terry McAuliffe (aka T-Mac), against a exceptionally conservative Republican, attorney general Ken Cuccinelli (aka Cuch). Are swing state Virginians willing to elect a person, Cuccinelli, who wants to defund Planned Parenthood and has labeled the Environmental Protection Agency an “agency of mass destruction”? McAuliffe, on the other hand, was a long-time Clinton fundraiser and adviser, and his run is seen as a test of the Clinton name and organization for Hillary’s possible run in 2016.

From two polls produced this week we can make similar conclusions with different results. An NBC/Marist poll has T-Mac up by 2pt among registered voters and down by 3pt among likely voters. A Washington Post survey has Cuch up 5pt among registered voters and 10pt among likely voters. You’ll note that the overall results are different, but that the gap between the registered and likely voters is the same.

Right now, the truth is probably somewhere in the middle. Quinnpiac has polled registered voters three times since January, and each of their results fell in between the registered voter numbers from Marist and the Post. Likewise, a Christopher Newport University survey from January had a similar 4pt gap between registered voters and likely voters. Off-year elections tend to feature more Republican friendly electorates, given that African Americans and young people are a lower percentage of the electorate than in presidential year elections.

But does this current polling actually tell us anything about what will happen on election day? Not really. At this point in 2009, all but one scientific poll had eventual winner and current Republican Governor Bob McDonnell winning by 9pt or less. A few polls even had Democrat Creigh Deeds ahead by mid-June, but most had McDonnell ahead by about 5pt. With a few exceptions – notably from SurveyUSA, which consistently showed double-digit McDonnell leads by early August – most surveys continued to show a single-digit McDonnell lead through September. McDonnell won by 17pt.

You think that’s unusual? Go back to the 2005. Through this point in the campaign, not a single poll had the eventual winner, Democrat Tim Kaine, ahead. A few polls had him down as much as 10pt, while the majority had him behind by mid-to-high single digits. Republican Jerry Kilgore trailed in only one public poll, all the way through early October! Tim Kaine, of course, went on to win by 6pt.

Overall, the polling at this point and through September has been off by about 10-15pt the last two elections. Given that Cuccinelli has a 4pt lead in the HuffPollster aggregate, either candidate could win. Perhaps Cuccinelli will win by a margin greater than 4pt, maybe less, maybe exactly that. Or maybe T-Mac will come out on top.

You might be tempted to look at the candidates’ favorable ratings, but I wouldn’t make too big a deal of those either, as they merely reflect the ballot standing of each man at this point. The important point is that both feature positive net favorables. Given that over 30% of voters have no opinion of Cuch, and over 40% have no opinion of T-Mac, the favorable ratings of these candidates will change. In 2005, Jerry Kilgore had a higher net favorable than Tim Kaine, which could be seen in the ballot test. He ended the election with a lower net favorable rating and lost. In 2009, Bob McDonnell had a net favorable rating equal to Creigh Deeds in early polling, which was reflected by a tight race in the ballot test. McDonnell ended up with a much higher net favorable rating, and indeed, he crushed Deeds.

What about the difference between the likely and registered voter results? The number one polling lesson from 2012 was that when likely and registered voters disagree in elections with high voter turnout, you should go with the registered voter results. Obviously, an off-year affair is not a high turnout election. Still, I was curious to see if over the past two cycles, pollsters have tightened the electorate, keeping in mind that only a few pollsters release results among both registered and likely voters, and most don’t.

Among those who report results from different screens, it’s not really clear that pollsters who use a likely voter screen are getting more accurate results than those that only polled registered voters. A 2009 October Virginia Commonwealth University poll had McDonnell winning by 18pt among likely voters and 16pt among registered voters. Neither was more accurate than the other. A 2009 October Roanoke College poll had McDonnell ahead by 17pt among likely voters and by 19pt among who said they were certain to vote (who we’ll call “definite” for the sake of clarity). In this instance, the looser screen ended up being closer to the final result.

In 2005, tightening the electorate did pollsters no favors either. An October Diageo/Hotline poll found Kaine winning by 2pt among registered voters, 1pt among likely voters, and down by 2pt among definite voters. Kaine, as mentioned, won by 6pt. An October Roanoke College poll had Kaine winning by 8pt among probable voters and 10pt among definite voters. Again the looser screen performed better than the more selective one.

What about earlier in the campaign? In 2009, an August Washington Post poll had McDonnell up by 15pt among likely voters and by 7pt among registered voters. Clearly, the likely voter result was closer to the truth – in this case, at least. In 2005, the opposite occurred. An early September Washington Post poll had Kilgore up by 7pt among likely voters and by 4pt among registered voters. Earlier in the campaign, SurveyUSA showed a dramatic rise in Kilgore’s lead from 4pt to 10pt between May and June when switching between registered and likely voters. Other polling companies showed a consistent race, which means that the far less accurate June result was almost certainly because of a change in the electorate being polled.

Thus, I’m not really sure there’s much to take from the recent polling data on the Virginia gubernatorial race. Yes, Ken Cuccinelli likely has a small lead at the moment, but that edge probably means very little. The difference between the registered and likely voter screen may be instructive, but we can’t yet be sure when to trust it. Tighter voter screens have not proven to necessarily be more accurate in either the 2005 or 2009 Virginia gubernatorial elections.

We can only sit back and allow the campaign to unfold.

• A previous version of this article incorrectly stated the leader of The Washington Post’s survey of the Virginia gubernatorial race and has been corrected accordingly

Harry J Enten

After Mark Sanford’s win, embarrassed pollsters go back to the drawing board | Harry J Enten

South Carolina’s special election made fools out of a few pollsters, but we’re better off with flawed data than none at all

Mark Sanford is heading off to Washington DC, while pollsters are heading back to their workshops. His victory by 9pt over Elizabeth Colbert Busch in South Carolina’s first district special election was surprisingly large. The last two public polls from Public Policy Polling (PPP) and Red Racing Horses (RRH) had Sanford winning by 1pt and a tie, respectively. PPP published a poll just two weeks before the election that had Colbert Busch winning by 9pt.

All three reports have joined the top 10 least accurate polls within two weeks of a special election, since 2004. PPP’s first poll was especially bad. It had an error of 18pt, which makes it the second least accurate poll taken two weeks before a special election since 2004. As my friend Mark Blumenthal points out, this first PPP survey had far too many African Americans as a percentage of the electorate. I don’t doubt that some white voters, a mostly Republican demographic in South Carolina’s first district, were disenchanted with Sanford by allegations that he violated the terms of his divorce, but the difference in the percentage of black voters was too great. Colbert Busch never had a lead of 9pt. One might wonder whether she even had a lead.

These errors might make people think twice about trusting PPP and RRH. After all, many major news organizations won’t cite PPP because it uses interactive voice response (IVR) technology instead of live interviewers, and because it doesn’t call cell phones. RRH doesn’t use live interviewers or call cells either, and it certainly doesn’t have a long track record; it’s apparently run by people who have no real background in polling. It wasn’t a surprise, therefore, that the RRH poll had women as too great a percentage of the electorate, at 60% versus the about 55% it should have been.

The truth, however, that PPP’s and RRH’s final polls seem to have been more accurate than the private (or internal) polls which are surveys produced by the parties and candidates. Most, though not all, use live interviewers and call mobile phones, and often, unreleased internal polls are more accurate than your average public poll. Most of the private polls for this race actually showed Colbert Busch holding a small lead.

All of this is to say that all the polling stunk it up in South Carolina’s first. Republican turnout wasn’t depressed as most thought it would be, and Republican voters did ultimately pull the lever for Sanford. Most of the undecided voters were Republicans, and there’s a reason PPP started to see that more white voters would vote than prior surveys indicated.

That’s why I will continue to pay attention to PPP and RRH in the future. Yes, PPP having Colbert Busch up nine was an embarrassment, but no one did better than PPP’s final or RRH’s only poll. The fact that even private pollsters fumbled so badly suggests that nobody who used better techniques could have been more accurate. Polling special elections just isn’t easy, as there really isn’t a baseline to understand who will turn out to vote and who will stay home.

Besides, the polling was useful, though imperfect. We knew that Sanford wouldn’t come close to replicating Mitt Romney’s 18pt win in the district, for instance. Even if the overall result was off, we learned some nuances of county polling, where the differences of support for each candidate were greater than expected. Thanks to RRH’s survey, we had a better idea on how counties would vote relative to each other, compared to the old method of just applying a uniform swing off the 2012 results. That doesn’t mean RRH is a great pollster, or even a particularly competent pollster, but it does suggest that almost any poll data can often be better than just going off the “fundamentals”.

It’s no surprise that you’re going to continue see websites like HuffPollster and Real Clear Politics report on surveys like PPP and RRH, which don’t meet the highest standards in the world. Polling data, even just okay data, can tell us a lot. In this case, the “flawed” public data was as good as the private data, and it was better than not looking at any polling at all.

Harry J Enten

How good is ADP at forecasting the monthly BLS jobs reports? | Harry J Enten

With official jobs data seen as a key economic metric, no wonder other agencies second-guess them. But ‘guess’ is about right

The government reported 165,000 new jobs created according April’s nonfarm payroll numbers (176,000 in total in the private sector) – a pleasant surprise to most economists, who were anticipating fewer. Part of the reason that expectations were off was because the Automatic Data Processing (ADP) jobs report predicted that only 119,000 would be created, an apparent error of 57,000. Why is this discrepancy a big deal?

Jobs reports used to be exciting only for economists and stockbrokers, but since the election season, every political junkie and their dog seems to have taken an interest. People recognize that the economy plays a vital role in deciding votes; these reports, therefore, offer a vital clue to predicting the politicians’ election chances. So, now we have both the economic and political class yearning for 8.30am on the first Friday of the month, all to learn about the jobs numbers.

But as in so many arenas in America, people can’t wait to see what happens. They race to get the answer as quickly as they can, picking up on whatever clues they deem fit. Enter the ADP jobs report, a jobs survey released two days before the official Bureau of Labor Statistics (BLS) government report. Many use the ADP to predict the BLS, but past ADP surveys have sometimes been far off actual BLS results. As Steven Russolillo noted in September, “some months, it’s spot on; others it’s wildly off base.”

The ADP, hoping to make its data more accurate, made some major changes for its October 2012 report. That month, the ADP started using ADP payroll data, BLS employment data, and the Philadelphia Federal Reserve’s Aruoba-Diebold-Scott Business Conditions Index. As a result, ADP surveyed 62,000 more clients than previously, 2 million more employees, and two more company-size classes and industries. They brought on Moody’s Analytics to replace Macroeconomic Advisers for processing data. To put it mildly, these are not small changes.

Have the adjustments brought ADP any closer to solving the monthly jobs mystery?

To answer this question, I’ve compared ADP forecasts of the past seven months with the same seven-month period last year, and looked at the ADP’s accuracy in predicting final BLS numbers. The BLS produces an initial, second, and final report as it calculates more data, and the data between reports can differ greatly. The ADP wants to land as close as possible to the BLS’s final report, though most attention is usually paid to the initial report.

The past seven months have seen an average difference between the ADP and initial BLS report of 42,286 jobs. (You can see all the data here.) Some months, such as October and November 2012, had errors of under 30,000 jobs, while March and April 2013 saw errors of 57,000 or greater. No month had an initial error of less than 26,000; the error in margin ends up within +/-19,000 of 45,000.

Compared to the same time last year, the average error has, in fact, diminished. Last year, ADP was off by an average error of 55,429 jobs, which is 13,143 jobs greater than their more recent average. This difference, however, is not statistically significant, due to a small sample size (seven observations) and the fact that the old ADP results could sometimes be very accurate.

Last year, three months under the old methods had errors of 17,000 or less, compared to the initial BLS report – far more accurate than any month per the new ADP. The problem for the old ADP was that four months last year had errors of 66,000 or greater, which less accurate than all seven months of the new ADP.

In that light, the new ADP does look better than the old. When it comes to their forecasts and the initial jobs report, we still haven’t seen an error so wrong it makes your eyes pop out. Of course, we haven’t seen stunning accuracy either.

The BLS’s final jobs report, however – what ADP should supposedly be best at predicting – apparently confounds ADP. We see zero consistency in their results. Four out of six final reports (or second report for March 2013, since we don’t have the final one yet) have had errors of 28,000 or less. Two final reports, December 2012 and March 2013, have been within 4,000. November 2012 and February 2013, though, have seen errors of over 120,000 jobs! The old ADP, by comparison, had its biggest miss last year, in January 2012, at 107,000 jobs.

The average error of the new ADP on final BLS reports has been 51,167 jobs, which is actually worse than the ADP’s error on initial BLS reports. It’s better than the average error of 58,333 from last year, but it’s not better by a statistically significant amount.

When you put it all together, I can’t really say that ADP has done better with its new methodology than it did with its old. There are some signs that the changes have made it more accurate – perhaps those huge misses of November 2012 and February 2013 will turn out to be anomalies – but we’ll need a larger sample size to know for sure. But at this point, it looks as likely as ever that the ADP numbers will be way off-the-mark measured against the BLS’s final reports.

The smart bet right now? Have a little patience and wait for the actual government statistics.

Harry J Enten

No Patriot Act II: Americans choose civil liberties over security laws | Harry J Enten

Unlike 9/11, the Boston attack will not lead to new anti-terror law. But Democrats are now less civil libertarian than Republicans

Terrorist attacks offer lawmakers an ability to react. After 9/11, the American government decided to go to war in Afghanistan and to enact new laws aimed at curbing future attacks. The Patriot Act, for instance, has been regarded by some as a necessary step for safety and by others as an infringement on civil liberties.

Following the Boston Marathon attack, we’ve heard Republicans Lindsey Graham and John McCain, among others, push for Dzhokhar Tsarnaev to be handled in a way that many believe would be a violation of his civil liberties. So, has the Boston bombing opened up an avenue for lawmakers to pursue controversial new anti-terrorism measures that may limit civil liberties?

Almost certainly not. The latest CNN/Time/ORC poll finds that 49% of Americans are not willing to give up civil liberties in order curb terrorism, while only 40% are. In fact, 61% of Americans are more fearful that the government will overreact to the Boston bombing, compared to 31% who are worried that the government won’t act strongly enough.

Other polls confirm these findings. Just after the attacks, Fox News found that 43% of Americans were willing to give up “some personal freedom” to reduce the threat of an attack, while 45% were not. A Washington Post poll, from before the bombers were caught, reported that only 41% of Americans were most worried that the government wouldn’t go far enough because of constitutional concerns. Almost half of Americans, 48%, were worried the government would go too far and compromise constitutional rights.

The reaction to Boston has been monumentally different to the polling results after 9/11. Immediately following the attacks on the WTC, 66% of Americans were willing to give up “civil liberties” to stop terrorism – 26pt higher than today. And 39% of Americans were concerned that strong laws wouldn’t be enacted, while 34% were more concerned about restricting civil liberties. That 4pt lead for enacting stronger laws is now a 30pt lead in favor of protecting civil liberties, per the ORC poll. After 9/11, 71% of Americans were willing to give up “personal freedom” to reduce the threat of a terrorist attack per Fox – 28pt higher than today.

Indeed, the party breakdown of new polling means that Graham and McCain have even less chance of getting their way. Democrats at large – who are unlikely to agree with hawkish senators – are now more willing to give up personal freedoms than Republicans. In the CNN/Time/ORC survey, 51% of Democrats were were willing to give up some civil liberties to curb terrorism, while only 41% of Republicans were. Fox found an identical 51% of Democrats were willing to give up “personal freedom”, against just 43% of Republicans. The Washington Post poll found the same 8pt spread between Democrats and Republicans on the question of whether the government might compromise constitutional rights.

Republicans, it seems, have become the standard-bearers of civil liberties due to two factors: who’s in the White House and shifting currents inside each party.

The executive branch, the government’s chief, is currently a Democrat – one who many Republicans believe, for instance, is out to take their guns. After 9/11, a Republican president held office, which likely accounts for the parties switching positions. We already know that a respondent or a politician will often oppose an issue or policy just because of who’s in charge.

Second, the Republican party is increasingly becoming the party of Rand Paul and civil libertarians. You would expect exactly these respondents to be against an intrusion on civil liberties. Many Paulites tend to call themselves independents, which would also explain why, in the CNN/Time/ORC and Fox News, independents were the least likely to give up personal freedoms, at 32% and 29%, respectively.

This puts hawkish Republicans like Graham and McCain in an awkward position within their own party. If there were a Republican in the White House, I think more Republicans would be willing to sacrifice civil liberties to prevent terrorism. At the same time, though, the Republican party simply is in a different place than it was a decade ago.

Overall, the chances of any major, hawkish changes in terrorist policy are significantly hampered by public opinion. Americans did not react to the Boston bombings with anything near the willingness to sacrifice civil liberties they showed after 9/11. That Republicans – usually hawkish on national security issues – are wary of giving power to the Democratic-run executive branch only further weakens the chances that any new law might pass.

Harry J Enten

No, Joe: even without the 2008 crash, McCain would not have beaten Obama | Harry J Enten

Vice-President Biden was probably just being kind – but the idea that Senator McCain might have won the 2008 election is bunk

US Vice-President Joe Biden says a lot of funny things. This weekend, the ever-hilarious veep said that John McCain “probably” would have defeated Barack Obama in the 2008 election – were it not for the global financial crisis following the Lehman Brothers bankruptcy on 15 September 2008. I disagree wholeheartedly with Biden’s assertion: Barack Obama would have been in a strong position to win 2008 even without the financial collapse.

Let’s start off with Obama’s most basic advantage at the time: the GOP had held the White House for the eight years before he ran. From 1952 through to 2004, there have been six instances of a party holding the White House for more than a term. In the subsequent election – after eight years with the president on their side – that party won only one of those those six elections. Whether the incumbent party held the White House for more than four years explains about 30% of the difference in vote margins in elections for over half a century, heading right into 2008.

These odds were made worse by an economy that was already lousy months before the crash. When averaging across multiple segments of the economy, growth was negative by early July. The only other year growth was negative across this many sectors was in 1980, per Nate Silver’s economic index. In that election, Jimmy Carter became the only president since the start of the 20th century to lose after taking the White House from the rival party – and he got blown out by 9pt.

This 2008 recession, combined with an increasingly disliked war in Iraq, caused President George W Bush’s approvals to plunge to almost 30% by 1 September 2008 – the lowest early September approval for any sitting president before a major election, going back all the way since modern scientific polling began in the 1930s. The only president who comes close is Truman, who had approval ratings around 32% and 33% going into the 1946 midterms and 1952 presidential election. His party lost control of both chambers of Congress and then the presidency.

Indeed, the fundamentals strongly indicate that Obama should have beaten McCain even before the financial collapse. Don’t believe me? Look at the models advanced by political scientists in August of 2008. Using a combination of economic measures, Bush’s approval ratings, Obama-McCain poll data, and a host of other factors, six of nine models written up in the journal of the American Political Science Association had Obama beating McCain. The combination models had Obama winning 80% of the time, by an average of 4pt.

Focusing only on the polling from Obama v McCain leads us to the same conclusion. From the time Obama clinched the Democratic nomination, in early June, through the announcement of his running mate, in late August, Pollster.com recorded 103 national polls. McCain led in six of them. Three of the polls showed a tie. That means that 91% of the polls conducted during the summer had Obama beating McCain, by an average of a slightly less than 4pt, and with the median of Obama winning by 3pt.

McCain did take the lead after the Republican National Convention, by an advantage of a little less than 2pt. A smart convention plan, along with the unexpected emergence of Sarah Palin, provided a temporary boost. But looking at the data, the boost was clearly not going to last. The Pollster.com aggregate (being its most sensitive in order to catch any micro-trends) already had Obama regaining the lead by 14pt in September.

The research seems to concur with a naive reading of the polls. Looking at wave studies on respondents’ choices before and after the collapse, two different studies both agree that Obama would have won without the collapse. Sunshine Hillygus and Michael Henderson (pdf) found that the collapse gave Obama a single point more. Richard Johnston, Emily Thorson, and Andrew Gooch (pdf) put the gain at 3pt. Neither gain was “decisive” in determining the winner the election (and my thanks to Nadia Hassan for sending me links to these studies).

Thus, I’m fairly confident that Obama would have won the 2008 election without the financial disaster of September 2008. Would he have won by less? Maybe, though not by much. The economy, already limping, and negative views about Iraq drove opinion on Bush down to record lows; McCain, as the candidate of Bush’s party, was hardpressed to overcome these obstacles, which gave him a deficit in the polls long before the Lehman’s collapse. McCain would almost certainly have lost to Obama even if the economy had not buckled towards the end of the election season.

Harry J Enten

It’ll take more than a presidential library to make George W Bush popular | Harry J Enten

Don’t be fooled by a new poll showing Americans think fondly of ‘Dubya’. Of recent presidents, only Richard Nixon was less liked

The George W Bush presidential library is opening this week. Not surprisingly, fans of the former president are out in full force trying to help with the rehabilitation’s of Bush’s image.

Bush, of course, ended his presidency with an approval rating around 30%. This previous low, combined with the library opening, has helped give an impression of elevation to a new ABC/Washington Post poll that put his retrospective job approval rating at 47%.

The fact that President Obama’s approval rating in the current HuffPollster aggregate is a similar 48% only helps with the intrigue. The Washington Free Beacon wrote an article titled “Dubya’s Approval Matches Obama’s”. Add on the fact that Obama’s approval rating and Bush’s retrospective approval on the economy approval rating are about the same, and you got the makings of a great press narrative.

Let me be the one who tries to nip this story in the bud. I have no clue what type of person George W Bush is, or how history will view him in the future. What I do know is that the 47% retrospective approval rating should be put in context.

First, retrospective approval ratings should almost never be compared to current job approval ratings. Humans have a tendency to remember their elected officials more fondly than they did when they left office. Back in 2010, Gallup asked Americans what their retrospective approval rating was for Presidents John F Kennedy through George W Bush. In every instance except for one, the retrospective approval was higher than the final approval was when they left office.

Most Republicans, for instance, love to make fun of Jimmy Carter. Carter was the only president of the 20th century to lose re-election after replacing a president of a different party. He left office with a 34% job approval rating. His retrospective job approval rating in the 2010 Gallup poll jumped by 18pt.

Second, Bush’s retrospective approval is the second worst among presidents in the last 50 years ago. To save you doing the math, Carter’s 52% approval rating is higher than Bush’s 47%. Only the Watergate-tainted Richard Nixon recorded a lower retrospective approval than Bush.

Most presidents have retrospective approval ratings above 60%. All but Nixon and Bush have +10 or better retrospective net approval ratings. Thus, not only did Bush tie for the second worst final approval rating while in office, but he is also has the second worst retrospective job approval rating.

Third, the Gallup data should make clear that George W Bush hasn’t seen much recovery over the past three years in his retrospective approval rating. The 47% in the latest Washington Post poll is the same as the 47% that Gallup found in 2010. The 50% disapproval now is nearly identical to the 51% three years ago. Yes, people view him more rosily now than they did during his presidency, but that effect has been baked in for a while now. He’s not getting more loved as time goes by.

Finally, the retrospective approval rating probably gives a false sense of how Americans view Bush now. Remember that a retrospective approval is exactly that – retrospective. The better way to view how Bush stands with the American public is his current favorable ratings. In the past two years, there have been five polls conducted that have asked about Bush’s favorable rating with the American public.

Bush currently holds an average -5pt net favorable rating with the American public. President Obama’s favorable rating is almost exactly the reverse of that, at +7pt in the HuffPollster aggregate. Indeed, even the losing Republican nominee Mitt Romney ended the 2012 campaign with a higher net favorable than Bush, at -3pt.

It’s no wonder that President Bush continues to hurt the Republican party. Mitt Romney tried his hardest to tie what many saw as a lackluster economy in 2012 to President Obama. The problem was that most Americans still blamed Bush over Obama. In the network exit polls, 53% said the economic problems were more Bush’s than Obama’s fault. Only 38% of Americans disagreed.

So, George W Bush may be more fondly thought of now than he was when he left office, yet this is to be expected – and discounted. Almost all presidents see a boost after they leave office. Bush is still quite unpopular compared with other former presidents, and his current favorable rating is far worse than President Obama’s. No library opening is changing that. Republicans would be wise to stay away from embracing George W Bush for the time being.

• This article has been amended to reflect updated polling data as of 25 April 2013.

Harry J Enten

Sanford v Colbert Busch: what are the odds for South Carolina’s first debate? | Harry J Enten

Debating a cardboard cutout has not helped the former governor against his opponent, and the polls show it

Republican Mark Sanford needs a breakthrough moment in tonight’s debate for South Carolina’s congressional special election, or else his comeback effort to win on 7 May is likely to fall short against Democrat Elizabeth Colbert Busch.

Sanford hopes to return to public office as a congressman, after resigning his in the wake of an extramarital affair for which he resigned from the governor’s mansion and left his wife. After winning the Republican nomination for the special election in South Carolina’s first district, Sanford has faced a barrage of bad press suggesting that he can do no good, even including the news that he violated the terms of his divorce with his former wife.

His Democratic opponent, businesswoman Elizabeth Colbert Busch, has her own claim to fame. Her brother, Stephen Colbert, has hosted numerous fundraisers on behalf of his sibling, helping her stay on the air in a district that went for Mitt Romney by over 15pt. She has also received support from a number of outside organizations, like the Democratic Congressional Campaign Committee.

The National Republican Congressional Committee, meanwhile, has abandoned Sanford. They refuse to air advertisements on his behalf, leaving the airways to his opponents. Sanford himself has little money left, and has been forced to rely on tactics such as a “debate” last week – with a cardboard cutout of Nancy Pelosi. He couldn’t afford podiums. Sanford used music stands instead. As a smart analyst told me, “It just feels like he’s going down.”

The one poll we have from the race shows Sanford trailing by 9pt, 50% to 41%. The poll, conducted last week by Public Policy Polling (PPP), shows an electorate that has Democrats as a greater proportion of the vote compared to PPP’s prior survey, which Colbert Busch led by 2pt. The new poll’s respondents voted for Romney over Obama by only a 5pt margin, compared to the 18pt margin Romney by which won the district last fall. PPP attributes this difference to low Republican enthusiasm, itself stemming from the most recent charges against Sanford.

Not everyone is buying PPP’s survey, though. Cook Political Report’s Dave Wasserman, for instance, thinks that Colbert Busch leads, but by a margin less than 9pt. My own math indicates that when you adjust PPP’s projected electorate to match its prior survey, Colbert Busch is ahead by a little over 3pt.

So just what are the chances that PPP’s poll is off by 9pt? Not very likely.

Over the past ten years, there have been 30 non-candidate/party sponsored polls conducted in the final two weeks before 18 off-cycle (i.e. not on election day during a midterm or presidential year) special congressional elections. Overall, these polls have been mostly accurate; 27 out of 30 times, the candidate leading in the poll won most of the votes in the special election. (Note: I only look at those elections where at least one Democrat and Republican are running. You can see the raw data here.)

Two of the three polls after in which the leading candidate didn’t win the election were for the 2009 New York 23rd special election, a unique race, in that there were three competitive candidates: Democrat Bill Owens, Republican Dede Scozzafava and Conservative Party candidate Doug Hoffman. In the week prior to the election, the Republican candidate dropped out, remained on the ballot, and endorsed the Democrat.

Not surprisingly, the 23rd special had PPP’s largest absolute error; they predicted Conservative Hoffman to win by 17pt. He lost by 2pt to the Democrat.

In 26 of the 30 special election surveys, the polling error was less than 9pt – Colbert Busch’s lead. The polls’ average error was only 4pt, and the median even lower, at 3pt.

These margins leave little room for Sanford. He would need an error of at least 9pt, which has happened only 13% of the time. He would also need that error to be in his favor, which means he’s looking at a 7% likelihood. That’s not outside the realm of possibility, but it’s a long shot. My guess is internal polling shows the same results, which is why the Cook Political Report and Rothenberg Political Report have put their finger on the scale for Colbert Busch.

The best comparison to this scenario would be Louisiana’s sixth special election in 2008. Louisiana’s sixth was like South Carolina’s first in that a Republican represented the district for over 30 years, and both districts were over 15pt more Republican on the presidential level than the nation as a whole in the prior presidential election. Like Sanford, Republican Woody Jenkins had run for statewide multiple times. Jenkins also had his own ethics problems: a connection with former KKK wizard and Louisiana politician David Duke.

The final poll before the Louisiana sixth special had Democrat Don Cazayoux ahead by 9pt, just like the recent poll that shows Democrat Colbert Busch leading. Jenkins did better than his numbers suggested. He didn’t lose by 9pt, but he did lose – by 3pt. Even though the poll overestimated Jenkins’ deficit, Cazayoux’s lead was too great to overcome.

Like in Louisiana’s sixth, there is a decent chance that the upstart Democrat’s lead is less than 9pt. It’s quite possible Republican voters will come around to Sanford on election day.

The chance, however, that the PPP poll is so off that Mark Sanford defeats Elizabeth Colbert Busch? Less than 15%, unless something happens very quickly.

Harry J Enten

Immigration reform won’t deliver a Latino voter ‘bonanza’ to Democrats | Harry J Enten

To assume that 11 million undocumented immigrants potentially eligible to vote will change US political arithmetic is erroneous

I have been quite skeptical about the ability of the Republican party to win over Latino voters. As others and I have noted, Latinos don’t vote Democratic just because of immigration policy. They vote Democratic because they are more ideologically “in sync” with the Democratic party. That’s why I’ve felt that going along with immigration reform was unlikely to net Republicans many Latino voters.

That said, I can’t agree either with the math in a Politico article titled “Immigration reform could be a bonanza for Democrats”. The article starts off promisingly enough with the premise that if immigration reform passed and undocumented immigrants became citizens, Latinos would start voting Democratic in even larger numbers. I can go along with this because the main reason anyone votes for or against a political party is for its economic platform, and 81% of first generation Latino immigrants say they want a “bigger government with more services”, compared to only 48% of Americans overall.

Politico then uses the commonly quoted figure of 11 million undocumented immigrants and claims that there would, therefore, be “up to” 11 million undocumented immigrants up for grabs if they all became citizens. If these 11 million then voted along the lines of the Latinos who cast a ballot in 2012, Obama would have won the national vote by 7pt instead of 4pt. He could have carried Arizona and even Texas, which were each won comfortably by Republican Mitt Romney.

The problem I have, though, is why would anyone use the 11 million figure for reference.

First off, 1 million undocumented immigrants are under the age of 18. I don’t care what your immigration status is, you can’t vote in United States presidential elections if you are under the age of 18.

Second, of the 10 million adults, 19% aren’t actually Latino; 11% are Asian. Asian voting patterns tend to be less stable than Latinos. In the past election, Asians went for President Obama by about 45pt. Twenty years ago, they went for Republican George HW Bush.

Without more research (and there is surprisingly little of it), it’s unclear to know how undocumented Asian voting patterns would change given immigration reform. Also, keep in mind that about 60% of these Asian immigrants are in California and Washington State alone – so they’re not exactly going to be a game-changer in the electoral college.

Third, I tend to doubt that all 8 million adult Latino undocumented immigrants would go for citizenship. A Latino Decisions poll says 87% of them would, but I’m skeptical. Only 60% of all legal immigrants actually apply for citizenship. Since the Immigration Reform and Control Act of 1986, only about 40% of the newly eligible immigrants by so-said act became citizens. It would be reasonable to expect a similar percentage this time.

Only 36% of Mexican immigrants who are eligible to become citizens have gone through the process. Mexicans make up 72% of all undocumented Latino immigrants. Six in ten of the other 28% of eligible Latino immigrants have become citizens.

All together, if undocumented Latino immigrants become citizens at the rate that eligible Latino immigrants do, then we’re looking at 3.5 million new voting-age citizens. That 3.5 million is a far cry from the 11 million we first talking about.

Finally, just how many of these 3.5 million undocumented Latino immigrants can be expected to vote in the presidential election? Per the standard census Current Population Survey (CPS), only 49.9% of all voting age Latino American citizens cast a ballot in 2008. (Note, there is no report available for 2012 as of this point.) Based on pre-election surveys and work by Michael McDonald, there is reason to believe that percentage may have dropped further in 2012.

All told, it would seem that only about 1.7 million new Latino voters would be added if undocumented immigrants were granted citizenship. Nationally, this would be a net of about 775,000 votes. This would increase Obama’s vote margin, but not to 7pt; it would only go up to about 4.4pt – in other words, half a point from where it actually was in November 2012. Even adding in new Asian voters, who vote at a lower rate than even Latinos, and other undocumented immigrants (and controlling for the percentage who apply for citizenship, percentage of citizens who vote, and the percentage who voted for Obama), the margin probably only goes up to, at most, 4.6pt.

The amount this would shift individual states in elections is debatable. Take Nevada, where, at last count, there were 190,000 undocumented immigrants – the highest percentage of any state population. Most of them are Latino. Apply the same math we did above, Obama would have gained about 17,000 votes. It would have increased his state margin of victory by 1.4pt. That’s not nothing, but we’re talking about the state with the largest percentage undocumented immigrants.

Most states aren’t close to Nevada’s undocumented immigrant population, while the ones that are simply aren’t competitive at the president level: Arizona, California, and New Jersey. No state in 2012 would have had a different outcome if undocumented immigrants were given the right to vote.

The truth of the matter is that passing immigration reform won’t be a votes “bonanza” for the Democratic party because of potentially or newly enfranchised undocumented immigrants. That doesn’t mean passing immigration reform will help the Republican party among Latinos; the GOP should probably still be worrying about its Latino voter appeal. But it’s not facing a landslide from a new citizen electorate.

Harry J Enten

Can Weiner win the Democratic primary for New York mayor? | Harry J Enten

As the former congressman polls second behind Christine Quinn, Anthony Weiner looks set to come back from his sexting scandal

Anthony Weiner looks primed to shake up a dormant New York City mayoral race with his entry – should he get in the race officially. After appearing in print and on television, as well as putting out a policy notebook in the last week, the former New York congressman can now add “second-place poll finisher” to his resume.

Marist has published a survey of the Democratic primary race showing frontrunner Christine Quinn at 26%, Weiner at 15%, John Liu at 12%, Bill de Blasio at 11%, and Bill Thompson at 11%.

I take the poll as mostly good news for Weiner in his quest to take Gracie Mansion. Why?

1. Weiner has gone straight in at second spot

The fact that Weiner can shoot up to second place without even officially entering the race is a strong sign. What makes it even stronger is the fact that second place in a New York City mayoral primary is a ticket to the next round. When no candidate reaches 40%, the top two finishers go forward to a runoff. Quinn’s not anywhere close to 40%, while Weiner’s sitting pretty in second.

New York City runoffs can be odd ducks. You can end with a situation like the 1993 Comptroller primary where the incumbent Liz Holtzman finished second in the first round, then lost the second round with a lower percentage of the vote than she’d won in the first. You could have a 2001 mayoral primary scenario where Mark Green lost the first round by 5pt and won the second round by 2pt.

The point is that anything can happen in a runoff. The key is to live in the first round to fight another day.

2. Weiner has the potential to build a stronger position

The one thing I was struck by upon reading the Marist poll is that Weiner’s in a far stronger position than he was just two months ago. In Marist’s February poll, Weiner’s favorable rating among Democrats stood at 34%. That’s now up to 45%. His unfavorable rating is down 2pt, from 43% to 41%. Overall, that’s a 13pt gain in net favorability.

The percentage of Democrats who want him to run for mayor is also way up. The percentage of Democrats who wanted Weiner to run just after the sex scandal was only 27%. That’s up to 40% now. The percentage who don’t want him to run is down from 65% to 46%. Together, it’s a net 32pt turnaround for Weiner.

The point is that it would be a mistake to treat Weiner’s favorables now as stable. The more time that has passed since his sex scandal, the more people seem to be warming to him again. His recent press tour mostly garnered positive press; it certainly did not backfire – people were not turned off. With his wife Huma Abedin by his side, it’ll be awfully difficult for opponents to try and hang the Twitter-sexting scandal around his neck.

3. Quinn is an unusually weak frontrunner

Under attack for a record that many believe is not all that liberal, Quinn’s campaign has faltered. Quinn was at 37% two months ago; she’s now at 26% with Weiner in the field. It doesn’t take a genius to recognize that her numbers are going in the wrong direction. That’s good news for candidates not named Quinn.

The 26% is the lowest polling result for a Democratic frontrunner since 1977. Keep in mind that that field was evenly divided not because the candidates were weak, but because they were all so strong. It included women’s right leader Bella Abzug, future Governor Mario Cuomo, and Ed Koch, among others. This year, Quinn is at 26% even as the third-placed John Liu’s former campaign treasurer is on trial for campaign funding fraud.

Even the “meh” candidacy of Ruth Messinger in 1997 was in the upper 30s at this point. And Fernando Ferrer, who would go on to lose the general election by 20pt, was also polling in the upper 30s at in 2005.

For Quinn to have been running for many months, to have been on the cover of New York Magazine, and still to only be at 26% will be a dismayingly feeble result for her campaign managers. Her base, too, seems one that may be unstable.

Quinn is polling at 28% among black voters. The problem is that New York City politics does tend to be racially determined. Even in her subpar run of 2005, African-American candidate C Virginia Fields managed to win a plurality of the black vote. Right now, Bill Thompson, who was the Democratic candidate for mayor in 2009, is only at 14% among African Americans. I’d expect him to eat into Quinn’s percentage of black voters, or at least freeze her where she is.

4. Weiner v Quinn level-pegging in name recognition

The one piece of what might be seen as bad news for Weiner in the Marist poll is that with 85% name recognition, he’s still only polling at 15%. But I don’t see that as much of a problem: he wasn’t polling much higher before the sex scandal hit, but his numbers are in flux and have trended positive. Meanwhile, his frontrunner rival, Christine Quinn, has a similar level of name recognition, yet she’s slipped back to 26%.

Conclusion

The facts that he’s already in position to make the runoff as it stands, that his favorables are going in the right direction, and that the frontrunner is stalled, are all good signs. That doesn’t mean he will win. I am still partial to the possibility of Bill Thompson overhauling the others, given the math above.

At this point, all the candidates’ hopes are very much alive. But do I think Anthony Weiner can win this thing? Absolutely, he could.

Harry J Enten

Terrorism in the US: what is the real threat level? | Harry J Enten

In perspective, terror attacks are at a historic low. But the picture is more complex: major cities like Boston are still likely targets

Though we know nothing yet about who committed the bombings at the Boston Marathon, or why they did so, the assumption at this point is that these were acts of terror – and, as the president affirmed Tuesday, is being investigated as such.

The shadow cast by the 9/11 attack means that every such incident now tends to be seen as a new episode in a distinct and frightening era of terrorism in the mainland United States. But does this picture actually fit the historical record?

Let’s put Boston in the context of the history of terrorist attacks in the United States over the past 40 years. One graph that tries to do this, posted by the Washington Post on Tuesday morning, is this:

It’s taken from the National Consortium for the Study of Terrorism and Responses to Terrorism at the University of Maryland (pdf), and it plots the number of terrorist attacks in the 50 states. It’s difficult not to be immediately struck by the peak on the left hand side of the graph: no other year comes close to the near 500 attacks that occurred in 1970. In fact, only two other years even surpass 100 attacks, and both of those were also in the 1970s. But this graph doesn’t tell the whole story.

First, let’s start with the fact that most terrorist attacks occur in major metropolitan areas. Manhattan and Los Angeles are two of the most at-risk targets for major terrorist attacks throughout the period: 13.1% of all terrorist attacks from 1970 to 2008 occurred in Manhattan, New York alone; 6.0% were in Los Angeles County, California. Combined, that’s nearly a fifth of all terrorist attacks.

Other areas were less consistent at registering in the terrorism chart. San Francisco was a center of leftist terrorism in the 1970s, but has since cooled off. Maricopa County, Arizona has seen more attacks in the 2000s as “single-issue terrorism” (such as animal rights or hate crime) has become more dominant.

Second, over the last decade, very few terrorist attacks have occurred across the central swath of the country. When you look at a heat map of terrorist attacks over the last 40 years, you see terrorism occurring in many different states. That, however, shows where attacks have been most likely to occur by decade.

Only five of the terrorist attacks coded by type by University of Maryland occurred in the center of the country during the 2000s. Almost all the attacks that took place in that region were during the 1980s and 1990s, when rightwing and religious groups were the most likely to carry out attacks.

Third, terrorism attacks are most likely to be in places where crime is high. There’s not a perfect correspondence, to be sure; but the correlation between high crime rates and terrorist attacks is highly significant at 0.25.

Fourth, terrorist attacks tend to occur in areas that are most ethnically diverse. Even when taking into account population density, you are more likely to see a terrorist attack in an area where many languages are spoken. You are also more likely to see a terrorist attack in a city where people live with less residential stability. Poverty and inequality, however, are not a factor: you are more likely to see a terrorist attack in cities with a lower degree of concentrated disadvantage.

A key point, however, is that the percentage of foreign-born residents is not a significant predictor of terrorist attacks. The same goes for racial identity: a city with more black or Hispanic residents is no more likely to see a terrorist attack than a majority white city. Language likely differs from these factors because you usually see different languages in cities with foreign business interests or government interests, which is exactly where terrorist attacks are more likely to occur.

Fifth, the chance of surviving a terrorist attack is the same as it has always been. Deadly terrorist attacks may be down, but the the percentage of those attacks that are deadly is the same. In 1970, only about 5% of terrorist attacks were deadly. That spiked to 41% in 1973, but it has mostly hovered between zero and 20% in the years since. The two years with the biggest spikes were 2001 and 2006, when 25% of attacks were deadly.

Sadly, most of these factors tend to make the events that occurred in Boston predictable or likely, compared to other places. Boston is among the top ten major metropolitan areas, and it’s on the coast. It’s in the top third for United States cities by violent crime rate. A number of the places within the metropolitan area have high population densities, including the city of Boston itself. The city ranks highly for non-English speaking households (pdf). Finally, the attack had a fatal outcome.

About the only characteristic that perhaps doesn’t fit the conventional pattern is that it occurred at a sports event, rather than the target being a government building or business. Indeed, the targeting of ordinary civilian members of the public is the truly scary part of the Boston Marathon bombing: with the exception of the Atlanta Olympics bombing of 1996, the idea of attacking a sports event in the United States is novel and unusual.

In other ways, Boston is typical: from a statistical point of view, it’s not all that surprising that nearly 2% of all US terror attacks occurred in the Boston metropolitan area between 1970 and 2008. This week’s tragic incident sadly matches the pattern.

Harry J Enten

Margin of Σrror

Category Archives: World news

Polls at this point in the Virginia governor’s race tell us very little | Harry J Enten

After Mark Sanford’s win, embarrassed pollsters go back to the drawing board | Harry J Enten

How good is ADP at forecasting the monthly BLS jobs reports? | Harry J Enten

No Patriot Act II: Americans choose civil liberties over security laws | Harry J Enten

No, Joe: even without the 2008 crash, McCain would not have beaten Obama | Harry J Enten

It’ll take more than a presidential library to make George W Bush popular | Harry J Enten

Sanford v Colbert Busch: what are the odds for South Carolina’s first debate? | Harry J Enten

Immigration reform won’t deliver a Latino voter ‘bonanza’ to Democrats | Harry J Enten

Can Weiner win the Democratic primary for New York mayor? | Harry J Enten

Terrorism in the US: what is the real threat level? | Harry J Enten