Margin of Σrror

Margin of Σrror -

After Mark Sanford’s win, embarrassed pollsters go back to the drawing board | Harry J Enten

South Carolina’s special election made fools out of a few pollsters, but we’re better off with flawed data than none at all

Mark Sanford is heading off to Washington DC, while pollsters are heading back to their workshops. His victory by 9pt over Elizabeth Colbert Busch in South Carolina’s first district special election was surprisingly large. The last two public polls from Public Policy Polling (PPP) and Red Racing Horses (RRH) had Sanford winning by 1pt and a tie, respectively. PPP published a poll just two weeks before the election that had Colbert Busch winning by 9pt.

All three reports have joined the top 10 least accurate polls within two weeks of a special election, since 2004. PPP’s first poll was especially bad. It had an error of 18pt, which makes it the second least accurate poll taken two weeks before a special election since 2004. As my friend Mark Blumenthal points out, this first PPP survey had far too many African Americans as a percentage of the electorate. I don’t doubt that some white voters, a mostly Republican demographic in South Carolina’s first district, were disenchanted with Sanford by allegations that he violated the terms of his divorce, but the difference in the percentage of black voters was too great. Colbert Busch never had a lead of 9pt. One might wonder whether she even had a lead.

These errors might make people think twice about trusting PPP and RRH. After all, many major news organizations won’t cite PPP because it uses interactive voice response (IVR) technology instead of live interviewers, and because it doesn’t call cell phones. RRH doesn’t use live interviewers or call cells either, and it certainly doesn’t have a long track record; it’s apparently run by people who have no real background in polling. It wasn’t a surprise, therefore, that the RRH poll had women as too great a percentage of the electorate, at 60% versus the about 55% it should have been.

The truth, however, that PPP’s and RRH’s final polls seem to have been more accurate than the private (or internal) polls which are surveys produced by the parties and candidates. Most, though not all, use live interviewers and call mobile phones, and often, unreleased internal polls are more accurate than your average public poll. Most of the private polls for this race actually showed Colbert Busch holding a small lead.

All of this is to say that all the polling stunk it up in South Carolina’s first. Republican turnout wasn’t depressed as most thought it would be, and Republican voters did ultimately pull the lever for Sanford. Most of the undecided voters were Republicans, and there’s a reason PPP started to see that more white voters would vote than prior surveys indicated.

That’s why I will continue to pay attention to PPP and RRH in the future. Yes, PPP having Colbert Busch up nine was an embarrassment, but no one did better than PPP’s final or RRH’s only poll. The fact that even private pollsters fumbled so badly suggests that nobody who used better techniques could have been more accurate. Polling special elections just isn’t easy, as there really isn’t a baseline to understand who will turn out to vote and who will stay home.

Besides, the polling was useful, though imperfect. We knew that Sanford wouldn’t come close to replicating Mitt Romney’s 18pt win in the district, for instance. Even if the overall result was off, we learned some nuances of county polling, where the differences of support for each candidate were greater than expected. Thanks to RRH’s survey, we had a better idea on how counties would vote relative to each other, compared to the old method of just applying a uniform swing off the 2012 results. That doesn’t mean RRH is a great pollster, or even a particularly competent pollster, but it does suggest that almost any poll data can often be better than just going off the “fundamentals”.

It’s no surprise that you’re going to continue see websites like HuffPollster and Real Clear Politics report on surveys like PPP and RRH, which don’t meet the highest standards in the world. Polling data, even just okay data, can tell us a lot. In this case, the “flawed” public data was as good as the private data, and it was better than not looking at any polling at all.

guardian.co.uk © 2013 Guardian News and Media Limited or its affiliated companies. All rights reserved. | Use of this content is subject to our Terms & Conditions | More Feeds

Sanford v Colbert Busch: what are the odds for South Carolina’s first debate? | Harry J Enten

Debating a cardboard cutout has not helped the former governor against his opponent, and the polls show it

Republican Mark Sanford needs a breakthrough moment in tonight’s debate for South Carolina’s congressional special election, or else his comeback effort to win on 7 May is likely to fall short against Democrat Elizabeth Colbert Busch.

Sanford hopes to return to public office as a congressman, after resigning his in the wake of an extramarital affair for which he resigned from the governor’s mansion and left his wife. After winning the Republican nomination for the special election in South Carolina’s first district, Sanford has faced a barrage of bad press suggesting that he can do no good, even including the news that he violated the terms of his divorce with his former wife.

His Democratic opponent, businesswoman Elizabeth Colbert Busch, has her own claim to fame. Her brother, Stephen Colbert, has hosted numerous fundraisers on behalf of his sibling, helping her stay on the air in a district that went for Mitt Romney by over 15pt. She has also received support from a number of outside organizations, like the Democratic Congressional Campaign Committee.

The National Republican Congressional Committee, meanwhile, has abandoned Sanford. They refuse to air advertisements on his behalf, leaving the airways to his opponents. Sanford himself has little money left, and has been forced to rely on tactics such as a “debate” last week – with a cardboard cutout of Nancy Pelosi. He couldn’t afford podiums. Sanford used music stands instead. As a smart analyst told me, “It just feels like he’s going down.”

The one poll we have from the race shows Sanford trailing by 9pt, 50% to 41%. The poll, conducted last week by Public Policy Polling (PPP), shows an electorate that has Democrats as a greater proportion of the vote compared to PPP’s prior survey, which Colbert Busch led by 2pt. The new poll’s respondents voted for Romney over Obama by only a 5pt margin, compared to the 18pt margin Romney by which won the district last fall. PPP attributes this difference to low Republican enthusiasm, itself stemming from the most recent charges against Sanford.

Not everyone is buying PPP’s survey, though. Cook Political Report’s Dave Wasserman, for instance, thinks that Colbert Busch leads, but by a margin less than 9pt. My own math indicates that when you adjust PPP’s projected electorate to match its prior survey, Colbert Busch is ahead by a little over 3pt.

So just what are the chances that PPP’s poll is off by 9pt? Not very likely.

Over the past ten years, there have been 30 non-candidate/party sponsored polls conducted in the final two weeks before 18 off-cycle (i.e. not on election day during a midterm or presidential year) special congressional elections. Overall, these polls have been mostly accurate; 27 out of 30 times, the candidate leading in the poll won most of the votes in the special election. (Note: I only look at those elections where at least one Democrat and Republican are running. You can see the raw data here.)

Two of the three polls after in which the leading candidate didn’t win the election were for the 2009 New York 23rd special election, a unique race, in that there were three competitive candidates: Democrat Bill Owens, Republican Dede Scozzafava and Conservative Party candidate Doug Hoffman. In the week prior to the election, the Republican candidate dropped out, remained on the ballot, and endorsed the Democrat.

Not surprisingly, the 23rd special had PPP’s largest absolute error; they predicted Conservative Hoffman to win by 17pt. He lost by 2pt to the Democrat.

In 26 of the 30 special election surveys, the polling error was less than 9pt – Colbert Busch’s lead. The polls’ average error was only 4pt, and the median even lower, at 3pt.

These margins leave little room for Sanford. He would need an error of at least 9pt, which has happened only 13% of the time. He would also need that error to be in his favor, which means he’s looking at a 7% likelihood. That’s not outside the realm of possibility, but it’s a long shot. My guess is internal polling shows the same results, which is why the Cook Political Report and Rothenberg Political Report have put their finger on the scale for Colbert Busch.

The best comparison to this scenario would be Louisiana’s sixth special election in 2008. Louisiana’s sixth was like South Carolina’s first in that a Republican represented the district for over 30 years, and both districts were over 15pt more Republican on the presidential level than the nation as a whole in the prior presidential election. Like Sanford, Republican Woody Jenkins had run for statewide multiple times. Jenkins also had his own ethics problems: a connection with former KKK wizard and Louisiana politician David Duke.

The final poll before the Louisiana sixth special had Democrat Don Cazayoux ahead by 9pt, just like the recent poll that shows Democrat Colbert Busch leading. Jenkins did better than his numbers suggested. He didn’t lose by 9pt, but he did lose – by 3pt. Even though the poll overestimated Jenkins’ deficit, Cazayoux’s lead was too great to overcome.

Like in Louisiana’s sixth, there is a decent chance that the upstart Democrat’s lead is less than 9pt. It’s quite possible Republican voters will come around to Sanford on election day.

The chance, however, that the PPP poll is so off that Mark Sanford defeats Elizabeth Colbert Busch? Less than 15%, unless something happens very quickly.

guardian.co.uk © 2013 Guardian News and Media Limited or its affiliated companies. All rights reserved. | Use of this content is subject to our Terms & Conditions | More Feeds

Bill and Claire’s Unconstitutional Adventure

“No law, varying the compensation for the services of the Senators and Representatives, shall take effect, until an election of Representatives shall have intervened.”

-The 27th Amendment to the U.S. Constitution (1992)

Senators Bill Nelson (D-FL) and Claire McCaskill (D-MO) introduced a bill today that would force members of Congress to take pay cuts equal to the cuts affecting other government agencies under sequestration. This proposal is likely to be hugely popular with the American people. What’s the problem with this plan? It’s clearly unconstitutional, as well as simply being a bad idea.

The 27th Amendment to the U.S. Constitution (posted above) makes it unconstitutional for congressional pay to be changed until an election has taken place. While ratified in 1992, this amendment was originally proposed as part of the Bill of Rights and was supported by James Madison.

Madison supported this amendment because he did not want members to vote themselves a huge pay raise before the voters were allowed a chance to register their approval or disapproval of Congress. To quote Madison, “there is a seeming impropriety in leaving any set of men without control to put their hand into the public coffers, to take out money to put in their pockets.”

This amendment also protects members of Congress who express minority opinions on legislation or nominations. Imagine this scenario: The majority in Congress wants to pass a bill that it has the votes for, but it wants to look bipartisan in passing the bill. The speaker goes to the minority leader and says, “Have your members vote yes on my bill or we will vote to cut the pay of minority party members by half.” Seem implausible? Maybe, but the 27th Amendment is an important protection against the tyranny of the majority.

In addition to being unconstitutional, cutting the pay of members of Congress is a misplaced (albeit popular) reform. While the $174,000 salary that members of Congress receive is certainly a decent salary, it is not excessive if one considers the other potential job options for members of Congress. Who can forget the million dollars that former-Senator Jim DeMint (R-SC) has been offered to run the Heritage Foundation? Dozens of other examples exist of members getting huge pay raises to work as lobbyists or other positions in the private sector after leaving Congress.

As the old saying goes, “You get what you pay for.” If you offer second rate pay to members of Congress, you will get second rate members. (Yes, even worse than now!) In other words, there needs to be a competitive salary for members of Congress in order to maximize the probability of attracting high quality individuals to the job.

Furthermore, if the pay of members of Congress is cut too much, then serving in Congress will become even more difficult for middle and lower income Americans than it is now. Members of Congress have to maintain two residences and must have an extensive professional wardrobe (among other living expenses beyond that of the average American). This isn’t a problem if you are one of the 50 richest members of Congress, but if you are a clean energy expert/advocate,  high school teacher, or farmer/gospel music singer then a drastic pay cut might make it financially difficult to serve in Congress. At the very least, such an individual would be unable to build a financial nest egg in case of an election loss, which would disincentivize them from running in the first place.

All in all, a substantial pay cut for members of Congress would serve to make the membership of Congress even less economically representative of the country as a whole than it is now. While some people might be willing to take significant financial sacrifices to serve in Congress, enlightened statesmen will not always be at the helm.

In conclusion, even if you don’t agree that Mr. Nelson and Ms. McCaskill’s proposal is a bad idea, it is clear that it is unconstitutional. And when laws are legitimately unconstitutional, the Supreme Court has a way of striking that whole thing down.

[Note: The title of the article is a pun on the movie“Bill and Ted’s Excellent Adventure.” The final line of this piece is a (hopefully satirical) reference to Rep. Todd Akin’s disastrous comments about “legitimate rape” in the 2012 election.]

Americans want gun control, but not badly enough | Harry J Enten

Gun rights advocates know that, despite Newtown, the US public is only lukewarm about gun laws – and is cooling all the time

After a month of reported congressional movement on gun control, negotiations have apparently hit a snag. Democratic senators have decided to break up proposals into different packages – such as Senator Dianne Feinstein’s push for an assault weapons ban – instead of presenting one “Obama gun package”. And, unsurprisingly, Republicans and Democrats can’t seem to agree on what parts should make up a background check bill.

On the surface, this seems like Washington dysfunction at its worst, especially since the percentage of Americans who want tougher gun control has stayed at its post-Newtown high: a majority still wants a ban on assault weapons, although legislation on that has pretty nearly no chance of passing through Congress. Over 80% of Americans do agree on universal background checks, including a majority of Republicans.

But a deeper look at the numbers suggests that gun rights advocates may be playing a stronger hand than at first glance.

1. Most Americans don’t see gun control as the most significant way to prevent mass shootings

Per a Public Religion Research Institute poll, only 25% of Americans believe that stricter gun control laws and enforcement would be the key to preventing massacres. That was second to mental health screenings, at 30%, and just ahead of moral and religious teaching, at 20%.

Even when we expand the issue out to allow for multiple answers, as CBS News did, only 21% think that stricter gun control would prevent gun violence by much. Almost half, 46%, think mental health screening would help a lot, while 36% think armed guards in public places would be most useful.

2. Guns as a whole are not at the forefront of issues for most Americans

Only 4% of Americans listed guns as the most important problem facing the country in the latest CBS News poll. Instead, over 50% chose the economy, jobs or the budget deficit. That matches other recent polling, and the recent focus on the sequestration illustrates this data.

You might say, “Of course, the economy is the No 1 issue for Americans – how could gun control come close?” And I’d agree: if gun control were really at the top of the heap, I’d expect it to be polling higher. During the healthcare debate of 2009-10, for instance, healthcare regularly broke the 20% barrier in polls on the most important issue in the US.

Now, it’s possible for Americans to care about more than one issue at once, but it’s fairly clear that gun control can get lost in our current mess of unemployment, budget cuts, and a stalling legislature. But gun control tends to be tied with healthcare and immigration as the most important issue, at all of 5%. Right now, healthcare isn’t even a national issue so much as a state one, in parts of the country.

3. Most Americans don’t feel gun legislation needs to be passed this year

This doesn’t come as a shocker given my last point, yet gun rights advocates have to like this number: only 46% of Americans in the latest Pew Research poll believe it is essential to pass gun legislation this year. That number includes only 42% of independents, and in fact, only 71% of Democrats who think that gun safety legislation is essential this year.

This is a key point because Republicans might fear being seen, once again, as “too rigid” and the “party of no”, as many Americans feel they are. But they can rest a little easier when it comes to guns. The American public seems to be saying that there’s nothing wrong with a delay.

4. Public opinion on gun control will eventually run out

If new gun laws aren’t passed this year, then they likely won’t be passed at all. Past history indicates that the current tide of opinion in favor of gun control will ebb over the course of the year. After Columbine, the only event in recent history with a comparable increase in favor of gun control, the high-water mark dropped after a year.

The reasons are twofold. First, the movement in favor of gun control has been driven mostly by media coverage. The media has yet to abandon stories about gun control, but time and a business imperative will eventually take their toll. Newer and more compelling news stories will fill the headlines, and most people will follow where the news coverage leads.

Second, the general movement over the past two decades has been against greater gun control. Take a look at the image above and take away the Sandy Hook spike. Prior to the Newtown shooting, the percentage of Americans in favor of stricter gun control had dropped below 50% – the first time that had ever happened.

Thus, from a game theory standpoint, I’m not exactly sure congressional members who want minimal gun control should rush into any deal. They have the numbers to stop any gun control measures in the Senate by filibuster, and they have a majority in the House. As importantly, if they look at the polling, they’ll know that they won’t face much of a penalty from the public: America isn’t exactly clamoring for tighter gun control and believes that other steps would do more to curb gun violence.

This is not to say a bill on background checks won’t pass through Congress. It’s just that the current hold-up should surprise no one; and despite the weight of current opinion, the pressure to make a law on guns probably won’t increase.

guardian.co.uk © 2013 Guardian News and Media Limited or its affiliated companies. All rights reserved. | Use of this content is subject to our Terms & Conditions | More Feeds

The weighting game: why Rasmussen has Democrats winning 2014 midterms | Harry J Enten

To mend its 2012 reputation for leaning Republican, Rasmussen tweaked its methodology. Now the pollster has overcompensated

Throughout the 2012 election season, it became a commonplace that Rasmussen Reports had a consistent pro-GOP bias. Well, get ready to be surprised: Rasmussen now leans more Democratic than the average pollster.

This from the pollster that projected Mitt Romney to win the popular vote and did not call an Obama victory in six of the states he actually carried. Clearly, Rasmussen needed to change, and it did. But rather than solve its problems, it has ended up overcompensating by mistakenly weighting its surveys to the national exit poll demographics.

What do I mean? Let’s take a look at some of Rasmussen’s new demographic weighting.

Prior to the 2012 election, whites made up a little over 73% of the electorate in Rasmussen world. Now, non-Hispanic whites make up 72% of Rasmussen’s sample; blacks make up 13% and Latinos are at 10% – just like the exits.

So much for demographics. The biggest noticeable change, however, is Rasmussen’s party identification.

Rasmussen is one of the few pollsters to actively weight by party identification. It interviewed thousands upon thousands of voters, and then utilized a “dynamic weighting” scheme to get the correct percentage of Democrats, Republicans, and independents in the lead-up to the 2012 election. It ended up with a three-month rolling average of 4.2% more self-identified Republicans than Democrats.

The exit polls, though, had Democrats with a 6pt party identification advantage. Sure enough, Rasmussen now weights its polling to 38% Democratic and 32% Republican – the same exact spread as the exit polls gave.

Here are my qualms about Rasmussen’s methodology. First, I dislike weighting polls by party identification. Party ID is an attitude that isn’t consistent from election to election. Given Rasmussen’s major problems in correctly tracking party identification, I have zero faith it will be able to track any deviation from the 2012 exit polls.

Second, I’ve never heard of any reliable pollster weighting their polls to match the exits. The exit pollsters certainly do not. Exit polls are great surveys with error, just like any other poll.

Any pollster using random digit dialing, like Rasmussen, should start off with the census as a baseline for weighting. You should call cellphones or substitute with an internet sub-sample; Rasmussen tried the latter and seemingly failed. If you wish to see a smaller subsection of the population, like registered voters or likely voters, you ask a short battery of simple questions to determine who is a registered or a likely voter.

Third, exit polls often differ from most pre-election polls on demographics. As I discussed earlier this cycle, exit polls tend to have more minorities and young voters than other surveys. Democracy Corps, 2012′s most successful national telephone pollster, is a case in point.

Corps had non-Hispanic whites at 74% of the electorate – 2pt higher than the exits, which matches the average of national pre-election pollsters. Democracy Corps had 18-29 year-olds at 16% of the electorate (3pt lower than the exit polls), and people 65 and older at 18% (2pt higher than the exit polls).

And what about the infamous party identification? Democracy Corps had Democrats with only a 3pt edge over Republicans, 35% to 32%, respectively. If Democracy Corps had re-weighted its final poll using the 2012 exit polls, it would have found a result too favorable to President Obama.

Fourth, Rasmussen is weighting its likely voter model to 2012. In presidential years, most registered voters are likely voters. In midterm years, as 2014 will be, however, turnout is down and the electorate tends to skew towards voters who are, on average, older and whiter than voters in the registered electorate. That makes midterm electorates more favorable to the Republican party.

Democracy Corps recently tested the likely voter electorate for 2012 and 2014 (pdf), and discovered what you’d expect: the 2014 electorate is, at this point, projected to be 2pt whiter and have 2pt fewer 18-29 year-olds, proportionally, than turned out in 2012.

So, it’s no surprise that Rasmussen’s choice to weight its results by the 2012 exit polls creates a Democratic “House effect”.

Obama has a House effect, too. Among pollsters who surveyed registered or likely voters (besides Rasmussen), his controlled average net approval since his re-election is +6.8pt. (I only averaged registered and likely voters here because the general electorate has fewer minorities than the adult population at large, which makes it more conservative.)

Obama’s average net approval in Rasmussen’s polling since re-election is +10.6pt, which is nearly 4pt higher than the other pollsters’ results. ABC/Washington Post and Quinnipiac are the only pollsters who had it higher among registered or likely voters. That’s quite the turnaround from before the election, when Rasmussen rarely gave Obama a positive net approval, and was more Republican-leaning than actual results in 81% of the races it polled.

This Democratic House effect can likewise be seen on the national House ballot for 2014. Since the election, Rasmussen’s 11-week average has Democrats with a +7.4pt advantage. Even with biases against Democrats winning seats, a 7.4pt margin would almost certainly get them back the House – and with room to spare. For 96 weeks before the election, Rasmussen only found the Democrats ahead 5% of the time in the on the national House ballot, despite the fact that Democrats won it by a point.

There have only been two other pollsters since the election to survey likely voters on the House ballot: Democracy Corps and GWU Battleground. Democracy Corps pegged the Democratic lead at 4pt, while GWU Battleground put it at 3pt. Again, Rasmussen is nearly 4pt more Democratic than the average of non-Rasmussen results.

The Democracy Corps result demonstrates the problem with polling to the 2012 electorate. Democracy Corps has the Democratic lead on the House ballot at 7pt, which matches Rasmussen – under 2012 demographics. Under 2014′s likely demographics, the lead shrinks to 4pt.

All in all, I’m not really sure what Rasmussen is up to now. Its methodology has always been somewhat odd, but its response to a tremendous Republican bias in 2010 and 2012 is even stranger. It shouldn’t be weighting to the national exit poll, since this bad decision only shifts to a Democratic House effect. For its own sake, I can only hope that Rasmussen modifies its methodology again – and this time, improves it.

guardian.co.uk © 2013 Guardian News and Media Limited or its affiliated companies. All rights reserved. | Use of this content is subject to our Terms & Conditions | More Feeds

How the fate of gun control is tied to presidential popularity | Harry J Enten

Polling shows that Obama’s approval rating is closely correlated with opinion on gun control. That suggests trouble ahead

Want to know how people feel about President Obama’s gun control plan?

Simply ask whether they approve or disapprove of how the president and his administration are doing their job. The two questions are nearly perfectly linked, and that could have major consequences for the future of gun control legislation.

The latest ABC/Washington Post polls prove the strong relationship. Many individual gun proposals are highly popular. In fact, seven tested gun measures, including background checks and bans on assault weapons and semi-automatic handguns, have majority support ranging from 51% to 88%.

When you attach Obama’s name by calling it “Barack Obama’s proposals”, the Post discovered that 53% of Americans favor the proposals – nearly identical to Obama’s approval rating of 55% in a separate Post poll last week. Gallup found the same, with 53% in favor of Obama’s gun control plan, compared to his monthly approval rating of 52%.

Drill down to specific demographics and the link between approval of the gun plan and approval of Obama’s administration becomes even clearer. Neither the Post nor Gallup asked about the plan or Obama’s approval in the same poll. The Post did, however, enquire about Joe Biden’s favorability. Biden’s net favorable of +11 percentage points is very close to Obama’s net approval rating of +14pt from the prior poll, meaning that the two are closely correlated.

[Note: we test Biden because we are examining small sub-samples and the sampling error on margins (for example, for favorable minus unfavorable) of less than 350 people is about 10pt or greater, which makes comparing different poll sub-samples difficult. By restricting ourselves to the same sample of people, as we can with Biden's favorables and the gun package, we can compare the answers among the exact same group of respondents.]

Among the 24 subgroups tested, the correlation between a subgroup’s opinion towards Joe Biden and Obama’s gun plan is 0.98 – nearly perfect. Moreoever, 95% of the differences in subgroup net favorables on Obama’s gun plan are predicted by their respective opinions of Biden. You rarely see two variables this closely linked. That’s even higher than the strong explanatory power that evangelical voting had in forecasting the Republican primary.

The median difference between a group’s opinion towards Biden versus Obama’s gun package is 2pt, which matches the 1pt difference between favorability for Biden and Obama’s gun control package overall. That’s ridiculously small. It means that if a respondent liked Biden, who substitutes for the administration, then the person liked the gun plan. Among independent voters, Biden’s net favorable is +5pt, and the net favorable impression of Obama’s plan is +7pt. Republican responders register -51pt net favorables for both. Among region, the median difference is only 3pt. Biden, for example, had a net favorable of +12pt in the south, while Obama’s gun plan had a favorable of +10pt. In the battleground of the midwest, Biden stood at +4pt and Obama’s gun plan was at +5pt net favorable.

So much for the correlation, but what does it mean for the future of gun control legislation?

In short, it means the gun control debate is likely heading in the direction of healthcare. In that political fight, as with this one, individual proposals polled well, but attaching Obama’s name to a proposal polarized opinion.

The key difference this time is that the net approval for Obama and his administration is about 14pt higher than it was when the healthcare bill passed in March 2010. That’s at least part of the reason why Obama’s gun safety proposals are polling much higher than his healthcare reform bill did three years ago.

This is also the main explanation for why President Obama’s gun plan is doing fairly well among independents and southerners. Obama lost both groups in the 2012 election, yet his post-election bounce has temporarily endeared him to them.

The chances are that he can’t maintain this surge in popularity, as most second-term bounces don’t last as long as the first-term ones. Obama will likely maintain a positive net approval overall, but not among certain subgroups. If his subgroup approval eventually matches his election margins, then he’ll start to show negative numbers among independents and southerners.

The question, then, is whether the high correlation between support for Obama’s administration and its gun policy signals that support for gun control is also poised to drop. It makes sense that it would. I’d also anticipate that this drop will be among the same subgroups as for Obama’s overall approval. That’ll mean that the current support for the gun plan among independents and southerners goes up in smoke.

Many House representatives up for re-election in 2014 are likely aware of the relationship between Obama’s gun proposals and his approval. Right now, that’s not an issue, but a fall in Obama’s approval would make this high correlation a problem. I don’t believe that legislators from areas where Obama’s approval is negative would want to be associated with a bill whose popularity is tied directly to presidential ratings. I’m talking about senators from red states who are committed to opposition, or waffling, on gun control – like Max Baucus, Mark Begich, Tim Johnson, Mary Landrieu, Mark Pryor, and, to a lesser extent, Kay Hagan.

So, we shouldn’t be surprised by a decline in support for Obama’s gun control package, nor by more obfuscation from vulnerable Democrats, who want to tread very carefully on guns. This doesn’t mean any gun package is over before it’s begun. Universal background checks, which are supported by about 90% of the public, seem to be picking up some steam.

But gun control legislation, on the whole, will be difficult to pass – and not just on the face of the proposals, but because red state legislators facing re-election simply won’t want to be associated a bill so closely tied to the popularity of President Obama.

guardian.co.uk © 2013 Guardian News and Media Limited or its affiliated companies. All rights reserved. | Use of this content is subject to our Terms & Conditions | More Feeds

How Senate Democrats, not just House Republicans, will block gun control | Harry J Enten

A filibuster-proof majority in Senate is already a stretch, but red state Democrats up for re-election may make it unreachable

Vice-President Biden’s group will make recommendations to President Obama this week on gun control. Majorities of Americans support numerous new gun restrictions, yet I’m pessimistic that anything will get through Congress. Why?

You might expect me to cite the Republican-controlled House, but the chances that the Democratic-controlled Senate will pass anything are not much better.

The Democrats need 60 votes to achieve “cloture” or avoid a filibuster, and that seems near-impossible. Assuming all 55 Democrats vote for a piece of gun control legislation, another five Republicans must join the coalition. I can only think of four Republicans who are gettable.

Mark Kirk, from blue state Illinois, has a lifetime F-rating from the NRA and has voiced support for an assault weapons ban. Susan Collins, from Obama-voting Maine, and Dan Coats of Indiana have each received a C+ from the NRA, and worse grades from the Gun Owners of America. Finally, John McCain of Arizona only has a B+ from the NRA and a C- from the Gun Owners.

These four Republicans, plus all the Democrats, equal only 59, which, of course, isn’t 60. Every other Republican has at least an A from either the NRA or the Gun Owners. There would have to be a major change of heart from at least one Republican in order to avoid a filibuster or make cloture.

But even if you got that magical one Republican, the openness to discuss gun control from West Virginia Senator Joe Manchin, isn’t likely to be shared by six red state Democrats who are set to run for re-election in 2014.

The reason is that regardless of how Americans view gun control right now, research indicates that they are likely to be at least somewhat affected by cues from their party leaders. This is especially the case if the party is out of power, as the Republicans currently are. You saw this during healthcare reform debate of 2009 when most Americans were in favor of Obamacare at first, then turned against it once it became a partisan issue and Republican leaders resisted the reforms. Americans then opposed the new law even as they still supported most of the policies contained within it. A similar outcome is possible this time, as Republicans leaders have not indicated much of any movement on gun control.

Pew Research found that Americans who prioritize gun rights over gun control, as well as gun owners, are more likely to say that the Republican party does a better job of reflecting their views on gun control, by margins of 44 percentage points and 22pt, respectively. Americans against gun control are more likely to be politically active than their pro-control counterparts: they are 17pt more likely to to contribute money, contact a public official, sign a petition, or express an opinion on a social network. I can’t imagine a senator from a red state, especially one in which there are more guns per household than the national average, wanting to go up against a barrage from pro-gun forces.

That’s why Max Baucus of Montana, Mark Begich of Alaska, and Tim Johnson of South Dakota all have A ratings from the NRA. They all come from states ranked third or fourth in gun ownership – at least 57% of households have a gun in the home. Baucus voted against a renewal of the assault weapons ban in 2004; Begich said he’d vote against it even after Newtown; and Johnson has seen his NRA grade go from a C+ in 2003 to an A, with an NRA endorsement, during his 2008 re-election fight.

The electoral prospects for each man adds to the unlikelihood that any will cast a vote in favor of serious gun control legislation. According to Public Policy Polling (PPP), Baucus has a net approval rating of -3pt and leads a generic Republican candidate by only 3pt. Begich won election 2008 by only 1pt and is rated as “vulnerable” by the Cook Political Report, which also pegs Johnson as the incumbent most likely to lose in 2014.

Mary Landrieu of Louisiana, too, is likely a goner on serious gun control legislation, despite a C from the NRA. She voted against renewing the assault weapons ban in 2004, and pretty much every other gun control measure of the past eight years. She won re-election in 2008 by six points – against a relatively weak opponent and in a state that voted for Romney by 17pt. She is “at risk” per the Cook Political Report. In Louisiana, 44% of households have a gun, 14th most in the nation.

Only Kay Hagan of North Carolina and Mark Pryor of Arkansas could go in favor of gun control. Hagan has an F from NRA, though she voted in favor in loosening regulations across state lines and calls herself a strong supporter of the second amendment. Pryor has a C-. He also voted to renew the assault weapons ban in 2004, and has wavered only occasionally since.

Again, the issue is that the Cook Political Report puts both of them at risk, come election season. Hagan’s net approval rating of -2 per PPP means she can’t afford to lose many voters, even if her state ranks only 23rd in the nation for households with guns, at 41%.

Pryor might be in an even worse spot. In 2012, Obama lost Arkansas by 24pt, and Democrats lost their control of the state legislature for the first time since Reconstruction. Democrats had three of the state’s four representatives in Congress during Pryor’s last election, but don’t have a single one now. He simply doesn’t need enemies in a state where 55% of households have a gun – sixth most in the nation.

So, I don’t think you can count on any red state Senate Democrat who is running in 2014. Taking away these six leaves the pro-control caucus with 53 votes in the Senate, at most – even with the four Republicans. Counting Hagan and Pryor only leaves the pro-control caucus with 55 votes.

Let’s also be real here. Joe Manchin has only said that “everything should be on the table”. He hasn’t actually committed to anything concrete. Neither have Joe Donnelly of Indiana, Heidi Heitkamp of North Dakota, nor Jon Tester of Montana – all red state Democratic senators given A-ratings by the NRA – committed to anything specific.

That’s why the smart analysis says that the chances of Congress passing serious gun control legislation decrease by the day. The House is a foregone conclusion. When all these numbers start getting added together, I’m not even sure you can find a simple majority of senators to agree on tougher gun control. A filibuster-proof majority, meanwhile, is likely impossible.

guardian.co.uk © 2013 Guardian News and Media Limited or its affiliated companies. All rights reserved. | Use of this content is subject to our Terms & Conditions | More Feeds

Establishment v insurgent: the Capitol’s new political dynamic | Harry J Enten

The old left-right ideological model for how Congress votes has lost explanatory power. Something different is afoot in US politics

Over the past few weeks, I’ve written a lot about ideology and polarization. Most of that has focused on the left-right differences. But there’s another shaping theme in politics that is also present: establishment v insurgent.

Presidential campaigns, for example, usually come down to those who are backed by the establishment and those who are not. The candidates backed by the establishment usually win, and the outsiders almost always lose. It’s the main thesis of the great book The Party Decides.

That’s why many political scientists thought that Mitt Romney winning the bulk of congressional endorsements for the Republican nomination in 2012 meant that he would almost certainly capture the nomination. His establishment support came from both left and right of the party, and was actually slightly more conservative than either Gingrich’s or Santorum’s, despite Romney donor profiles indicating that his public was moderate.

Twelve years prior, George W Bush triumphed over the moderate John McCain on the back of establishment support. Likewise, Hillary Clinton saw her 2008 hopes fade because of her inability to lock up support from establishment congressional leaders Harry Reid and Nancy Pelosi.

No one can doubt that this establishment v insurgent dynamic has been present in Congress for a while, but “DW-nominate” scores indicate that this dichotomy is becoming increasingly important. These scores seek to rank congressional members’ ideology in two dimensions, based on their rollcall voting record. In my previous analyses of polarization, I’ve used the first dimension, which usually does a good job at placing members on the liberal-to-conservative spectrum. The second dimension has rarely been used in recent years because it hasn’t seemed to stand for anything.

In the past Congress, however, the second dimension has begun to have more explanatory power – among Republicans, especially. That is, something beyond just being conservative or liberal is beginning to predict voting patterns of congressmen and women. The vote-view folks and I think that this dimension is along an establishment v anti-establishment axis, though one might also argue that it is geographically based – since many insurgent Tea Party members are from the American south.

Consider the debt ceiling debate of 2011. People on both the liberal and conservative side of the aisle supported the bill. Oddly, House Democrats and Senate Republicans were the two groups most likely to oppose the bill.

If the voting had been strictly along partisan lines, we’d expect the best fit line to be straight up and down. That is, the vote could easily be determined as being to the left or to the right. Instead, we have more of a diagonal line that goes from upper left to lower right in the House and upper right to lower left in the Senate. What that means is that there are liberal and conservative elements behind the voting, but there is also something else happening. You might call it a rather strange vote in which the Tea Party caucus, progressive caucus, and black caucus voted all together – though, arguably, it could be explained as establishment House Democrats and establishment House Republicans voting together. In other words, how many times can we expect John Boehner, Eric Cantor, Steny Hoyer, Kevin McCarthy, Nancy Pelosi, and Debbie Wasserman-Schultz all to vote the same way on a relatively divided debate?

The coalition was slightly different with the recent fiscal cliff vote, yet it displayed a similar pattern. Again, there was a left-to-right element, with more Democrats voting in favor this time. But there was also a “second dimension” element.

The lines are, again, diagonal. The differences among Democrats largely disappeared, but they were as present as ever among Republicans. The “establishment” Republicans largely voted in lockstep with one another, while the anti-establishment folks also banded together, voting the other way. The fit is certainly not perfect, with establishment figures like Cantor and McCarthy voting no – thought that actually matches well with the press coverage afterward declaring that Cantor’s vote indicated a break with the establishment leadership, rather than the fact that he’d newly discovered a conservative soul.

This divide between establishment v anti-establishment was present throughout the 112th Congress. Whether it was the 2012 omnibus bill, or the highway and student loan funding bill of 2012, outsiders such as blue-staters Michele Bachmann and Marco Rubio voted alongside southerner Rand Paul and westerner Mike Lee. Others with similar left-to-right rankings, but who were closer to the establishment, such as Senator John Barasso, Representative Cantor, Representative Billy Long of Missouri, and, yes, even very conservative Mitt Romney-backed Representative Steve King of Iowa often voted the opposite way.

It’s necessary to note that while the insurgents seem to be rising, it was the establishment that won in all the situations. My guess is that they will continue to win, even if they need to adjust. That’s why I’m skeptical about whether we’re really likely to see a “Republican civil war” in the coming years, or even the 2016 presidential nomination season.

In practise, the establishment tends to line up behind the eventual winner of the nomination before a war breaks out. That’s why even the hard-fought 2008 Democratic and 2012 Republican nomination winners became quite clear by the end of February in the nomination seasons. Further, if Republicans do well in 2014, which they should given the midterm landscape, then this should placate those currently calling for heads.

The ability of the party establishment to hold onto power might explain why Paul Ryan voted for the fiscal cliff package. Ryan is pretty far to the right in the left-to-right ideological rankings. He’s even been seen as fairly anti-establishment over the long term. Yet, he voted for each and every one of the “establishment” positions on the four key financial bills discussed here. Ryan may be looking to capitalize on the establishment credentials he built up during his run as vice-presidential nominee. We’ll have to see if his future voting record also supports the idea that he’s shifted toward the establishment.

In the meantime, it looks likely that this establishment v insurgent divide will continue through to the next Congress. The votes for and against John Boehner for house speaker did not split according to the liberal-to-conservative spectrum for Republicans, but rather among the second dimension: between establishment v anti-establishment forces. In short, this should looks set to be yet another unproductive and unwieldy Congress.

guardian.co.uk © 2013 Guardian News and Media Limited or its affiliated companies. All rights reserved. | Use of this content is subject to our Terms & Conditions | More Feeds

Americans on the fiscal cliff deal: meh | Harry J Enten

The polls say most expected a compromise – though not so many like what they got. But the deal-makers fare slightly better

The fiscal cliff was averted last week as both the House and Senate agreed on an imperfect package. What do Americans think of this legislation and those who passed it?

Many in Washington did not think a deal would be reached, but Americans expected Congress would hammer out an agreement on time.

In every single Gallup poll taken during the month of December, the majority of respondents guessed that Congress would reach a solution that avoided the cliff. That majority shrank to 50%, but still held as late as 22 December, despite pessimistic reports. The ability of the public to predict better than most pundits fits with research that shows Americans also do a very good job of forecasting the results of presidential elections.

But while most Americans expected a deal, they didn’t much like it. By a slim 45% to 43% plurality, Gallup found more Americans disapproved than approved of the fiscal cliff deal. This broadly concurs with Pew Research’s finding that 38% approved and 41% disapproved of the deal. The percentage of Americans who registered agreement is far less than the 68% of Americans who’d said they wanted “compromise” from their Washington leaders back in December.

Independents and Republicans, in particular, looked for agreement – just not this one. Only 27% of Republicans and 39% of independents approved of the passed fiscal cliff legislation per Gallup. That compares with the approximately 60% of each group who had wished Congress to pass “compromise” legislation in December. Meanwhile, the 67% of Democrats who approved of the fiscal cliff deal largely matches the approximately 70% of Democrats who wanted a bipartisan piece of legislation.

The reason for this before-and-after discrepancy, as Pew Research discovered, is that 57% of Americans think President Obama got more of what he wanted, while only 20% feel the same way about Republican leaders. That affirms the fiscal cliff legislative rollcall, in which, by a near 2:1 margin, House Republicans opposed the law, while House Democrats approved it by more than a 9:1 margin.

That’s not to say Republican leaders made a mistake compromising, in the eye of the public. The percentage of Americans who approved of generic congressional Republicans’ handling of the fiscal cliff negotiations had been stuck in the 20s throughout the process. This percentage reached a low of 25% per Gallup and 19% per Pew, after the deal was passed.

Yet, Republican House speaker John Boehner who, despite much protestation from his own caucus, managed to get the deal through the House, actually received a boost in his approvals: 31% of Americans approved of Boehner’s handling of the fiscal cliff deal in Gallup polling – higher than any other congressional leader.

For Senate Republican leader Mitch McConnell, who played a leading role, with Vice-President Joe Biden, in crafting the final form of the fiscal cliff bill, approval of 28% was also higher than the generic Republican congressional approval. In both cases of these named Republican leaders, the net approval (approval minus disapproval) of about -20pt is about 20pt higher than the net approval of generic congressional Republicans, at -40pt per Gallup.

Some might say that named Republican leaders only have higher approval because it’s easier to hate a no-faced person, but I’m not sure I agree. Only 27% approved of Senate Democratic leader Harry Reid’s performance in Gallup’s polling, which is slightly lower than the percentage of Americans who approved of named Republican leaders. It’s also lower than the 34% who approved of the generic Democratic leaders in Congress. It does seem that Americans are willing to reward both the Democratic party they saw as more compromising and the Republican leaders who showed a willingness to compromise.

The ability to compromise is also probably part of the reason that President Obama has consistently received the highest ratings on the fiscal cliff negotiations from Americans: 46% of Americans approved of Obama’s handling of the fiscal cliff negotiations, while 48% disapproved. That approval is slightly down from earlier, though the -2pt net split is about 20pt better than those of our named congressional leaders. Given that Pew Research has the split at 48-40 for the president, I’m guessing that this is mostly statistical noise. Obama, of course, had been hounded from some on his own side for being too compromising on the fiscal cliff deal. Vice-President Biden, who took the lead with McConnell on final negotiations, also had a -2pt approval/disapproval split, with approval at 40% and disapproval at 42% per Gallup.

Indeed, Americans may be split on the overall bill and may not love those who compromised – but they like the compromisers more than those who stuck to their guns. In other words, voters expect their leaders to be adults and make deals. Further, American “optimism” in avoiding the fiscal cliff was rewarded. We will have to wait to see whether Americans will be as positive when it comes to predicting the outcome of the political battle over the debt ceiling in the next few months.

guardian.co.uk © 2013 Guardian News and Media Limited or its affiliated companies. All rights reserved. | Use of this content is subject to our Terms & Conditions | More Feeds

Why ‘gerrymandering’ doesn’t polarise Congress the way we’re told | Harry J Enten

Biased redistricting is commonly held up as the culprit for America’s increasingly partisan politics. If only it were that simple

You ever hear a point of view that is so infuriating that you want stick your head out the window and yell? I go bananas when I hear an opinion that goes against well-established political science literature.

That happened this past weekend when respected television journalist Tom Brokaw said the House of Representatives is becoming increasingly polarized because of gerrymandering.

Don’t get me wrong, I love Brokaw. It just so happens that he is wrong, and posts about the effect of gerrymandering on redistricting have been written over and over again in past months.

It could be that Brokaw doesn’t quite understand what gerrymandering is. For those who don’t, gerrymandering is the manipulation in the drawing of House districts to ensure a desired result. Brokaw’s assumption is that politics is becoming more polarized as the result of gerrymandering in districts in which Democrats and Republicans are increasingly safe from worrying about a competitive challenger from the other party.

While it is true that House districts are increasingly “safe”, this is the case even when controlling for redistricting. Last week, Nate Silver noted that there was an 8% increase in polarization independent of any effects of redistricting in 2012.

Much of this change is because states themselves are becoming more polarized. As I pointed out in my last article, the number of swing states in presidential elections is at an all-time low. The number of Democratic and Republican senators from states that lean Democratic and Republican on the presidential level respectively is way up. Partially as a result, there is an increasingly large gap in ideology between Democratic and Republican senators, which has tracked with the ideological gap in the House. So unless, we are, all of a sudden, redistricting state lines, then gerrymandering can’t possibly be the cause of what we’re seeing at the state level.

Part of what’s going on in the House is that there is a natural inclination for Democrats to crowd into cities and Republicans to spread themselves out in suburban and rural areas. In New York City alone, there were five Democratic members of Congress re-elected with at least 90% of the vote. Jowei Chen and Jonathan Rodden ran redistricting simulations on the precinct level and still ran up against a natural bias that prevents Democrats being able to win more seats because of their urban concentrations. There really isn’t any way to fix this issue – at least, not by redistricting.

Nolan McCarty, Keith Poole, and Howard Rosenthal computerized redistricting with the aim of creating “fair” non-gerrymandered districts. Even when they tried their best to have “many heterogeneous and competitive districts”, the predicted level of polarization didn’t differ much from what we have today. Keep in mind, of course, that this sort of redistricting also would reduce the number of minority members of Congress because it neglects the Voting Rights Act. I get the feeling that wouldn’t be particularly popular.

We see this in Nicholas Goedert examination of the 2012 House results, too. Goedert looked at seats in which redistricting was controlled by Democrats, by Republicans, by bipartisan committees or by courts. Controlling for past-vote-to-seat curves, Goedert determined that when redistricting was nonpartisan or bipartisan, Democrats underperformed by an average of 7pt (that is, they only take 50% of seats when projected to get 57%).

This is absolutely key. Fair redistricting just doesn’t have the impact you might think. Once we control for incumbency, as Eric McGhee does in the chart below, Democrats are at an inherent disadvantage that dates back to the 1950s – long before Republicans controlled a majority of the redistricting processes.

Speaking of the effect of incumbency (that is, incumbents run ahead of what the national house vote would indicate), you’ll notice that it had a major effect the last few cycles. According to Think Progress, Democrats would have needed to have taken the national house vote by 7pt in 2012 to have won a majority of seats. The author, Ian Millhiser, blames this on gerrymandering.

Here’s the problem. After the 2008 House election, which was fought under Republican-gerrymandered district lines of 2000 district lines seats, the Democrats, who were already the majority party in the House, actually controlled more seats than they should have – given a 10.5pt victory in the national House vote.

The vote-to-seat curve will likely bounce closer to equal when incumbent Republicans sense a wave and retire, as they did in 2008.

None of this is to say that partisan gerrymandering doesn’t have any effect. Sam Wang believes Republicans are far more likely to gain seats than Democrats when they control the redistricting process. Goedert’s analysis, which includes more states, agrees, but also shows that Democrats are likely to take more seats than they should if they gerrymander as well. Goedert demonstrates that Republicans gain more seats by about 18 percentage points in a state when they control redistricting. Democrats pick up only about 9pt more when they draw the lines. This difference is, in my opinion, likely a reflection of the natural disadvantage Democrats have in redistricting – as seen in the 7pt Democratic underperformance in the non- and bipartisan states.

Overall, even if all the states redid their lines under “fair” rules, and given the Democratic win by 1pt in the 2012 House popular vote, Republicans would likely still maintain about 225 seats in the new Congress.

To correct for this imbalance, you would have to gerrymander. That is, you would have to manipulate the drawing of House districts to ensure the desired result. You’d need to have weird lines that snake from neighborhood to neighborhood, ignoring geographical integrity. That’s not only against the law in many states, it would also make congressmen and women unrepresentative of at least part of their long, winding districts.

Yet, this doesn’t tell the entire story. All of this analysis is based on the idea that the partisanship of the district causes the partisanship of the legislator. However, the difference in ideology between Republicans and Democrats in competitive districts and Republicans and Democrats in uncompetitive districts respectively is actually relatively small.

The real difference is how Republicans and Democrats differ ideologically in similarly partisan districts. McCarty and Boris Shor found that even in 50/50 districts (that is, those split evenly between Democrats and Republicans), the ideological divergence between Democratic and Republican representatives has climbed much higher in the past 40 years. So, even if you were to gerrymander evenly competitive districts, you would not likely see everyone coming together in Congress.

Anyway you put it, Democrats and Republican legislators have greater ideological differences than ever. Gerrymandering probably doesn’t help, yet it’s not the major cause of the problem. Natural geographic divisions on the state and local level between Democrats and Republicans are a big cause in this growing polarization. Both of these, though, fail to account for the fact that even when controlling for district partisanship, there are increasing ideological differences between Democrats and Republicans in Congress.

guardian.co.uk © 2013 Guardian News and Media Limited or its affiliated companies. All rights reserved. | Use of this content is subject to our Terms & Conditions | More Feeds