Margin of Σrror

Margin of Σrror -

USA Today parts company with Gallup as its pollster: what took so long? | Harry J Enten

The company that predicted a win for Mitt Romney has suffered another blow. Let’s hope it learns from its mistakes

USA Today has decided not to renew a polling contract with Gallup that dates back to 1992. We cannot say at this point why USA Today and Gallup have got divorced, but one cannot help but wonder whether Gallup’s poor track record has something to do with it.

In case you don’t remember, Gallup continuously and incorrectly showed the Republican presidential nominee, Mitt Romney, leading during the final month of the 2012 election. Its final poll had Romney leading by a point – five points off the eventual margin of his defeat. The reasons for these errors can be traced back to too tight a likely voter model and an underestimation of minority share of the electorate.

If 2012 was Gallup’s only error, I might be more willing to forgive. The fact is that Gallup has only pegged the correct winner in 60% of elections since 1976 – which is not much better than a coin flip. That record drops to 25% since 2000. In four elections since 1976 in which the election margin was less than five points, Gallup has not called the right winner.

The fact is, however, Gallup’s presidential polling miscues are only the tip of the iceberg. After an extended investigation by Mark Blumenthal, into Gallup’s long-standing racial imbalance in its adult sample (which is supposed to reflect the adult population), Gallup claimed it was weighting its sample to the cell and telephone population. Of course, not everyone has a telephone. Gallup said it was making “judgement calls” and, regrettably, stuck by its methods.

It was only by October that Gallup revealed new, “slight” weight changes, when everyone else was showing something different than it was. The fact that Gallup altered its methods in the middle of the campaign, and long after its flaws had been pointed out, had to make even Gallup’s most ardent defenders flinch. Bizarrely, Gallup decided that it should change its adult sample, but not its samples of registered or likely voters.

And yet, the most troubling accusations lobbed Gallup’s way came when the justice department filed charges against it. The suit, brought in 2012, alleges that Gallup submitted $13m in false claims to the US government, by inflating the cost of polling work. These charges include milking the US mint, state department and other organizations.

When you add it all together, there is little reason to trust Gallup. It had a track record worthy of no praise and a history of poor weighting schemes, and it faces criminal charges. The fact that Gallup gets its name in the news every day by surveying every day, has been in the field since the 1930s and is so well known that “Gallup” is a synonym for “poll” in many countries made it especially dangerous regarding the spread of misinformation.

USA Today, with the highest print circulation in the United States, only added to Gallup’s ability to spread bad data. Gallup made USA Today look bad, as the only major news organization to be associated with a pollster that had Romney winning in the final week of the campaign.

So even if none of Gallup’s faults I have mentioned here had anything to with USA Today’s decision to break with Gallup, USA Today and everyone interested in good data is now better off.

I’m wishful that Gallup will straighten itself out. Either way, there are better polling companies out there at the moment.

Hopefully, USA Today will sign one of these better pollsters to do its surveys.

• Editor’s note: the headline on this article was amended from “USA Today drops Gallup” on 23 January 2013 © 2013 Guardian News and Media Limited or its affiliated companies. All rights reserved. | Use of this content is subject to our Terms & Conditions | More Feeds