FiveThirtyEight is Gone! Good Riddance!

Nate Silver's abomination has done more to damage political data interpretation than anything else previously or since.

This week, I’ve been celebrating the pulling of the plug on FiveThirtyEight, one of the worst political websites ever created. Now, I’ve already written about FiveThirtyEight (click here) and made a video about them as well (click here), so I don’t want to go too much into the problems with their methodology and techniques. But sufficed to say, this website being out of sight and out of mind is a net positive in the political world. But why?

Legitimizing Garbage Polls

Over the last few election cycles, specifically since Trump has been a candidate, there has been a proliferation of “garbage poll”. What makes these polls garbage is that their data isn’t verifiable, and the people who conduct these polls are sometimes anonymous or have no polling experience whatsoever. Still, FiveThirtyEight legitimized these so-called “polling firms” even with scant evidence of their legitimacy.

The best example of this is Patriot Polling, a so-called “polling firm” started by two high school students who never had any experience in polling, public opinion, or political science at all. However, they submitted a poll to FiveThirtyEight, and they gladly accepted these polls.

Another so-called polling firm is So Cal Strategies. This is one of those “firms” whose members are totally anonymous. I emailed So Cal Strategies to ask them if they could provide any information as to who they are. Instead, they spent more time berating anyone on Twitter who calls out thier polls instead of providing any shread of proof of their their legitimacy. Still, FiveThirtyEight accepted them as a poll to put in their aggregate.

Redefining what makes a “good poll”

Anyone that has uses polling in a political campaign knows that the underlying data, the crosstabs, provide the most valuable information (even if the MoEs are higher). It’s in these crosstabs where one can see if the poll is legitimate or absolute bullshit. And over the last few cycles, FiveThirtyEight considered Atlas Intel as the “most accurate poll”. However, all their crosstabs show how poor their polls actually are.

I’ve made a video about Atlas Intel (click here) to talk about all of their flaws but let me just mention a few of them again. In a multi-state poll that they conducted between September 20^th and 25^th, Atlas Intel said that Kamala Harris was going to win male voters in Arizona 55% to 43%, but lose female voters 55% to 44%, a total flip of reality. In their North Carolina poll it showed the same flip of male and female voters. But the most insane crosstab was that in Pennsylvania, Donald Trump was going to win 47% of black voters. Honestly, the craziness of their crosstabs showed that these polls were absolutely garbage when it came to the underlying data.

Changing the utility of a poll

FiveThirtyEight has redefined what an “accurate poll” is. Over the last two cycles, Atlas Intel was deemed the “most accurate” poll, even though we’ve just seen that their crosstabs are absolute garbage. Still, the reason why they have been considered “most accurate” is because FiveThirtyEight only considers one data point as the measurement of success, the topline horserace number. They don’t consider any of the underlying data whatsoever.

They have also changed the perception of polling, making polls look like tools to predict a winner instead of what they are actually used for, which is a snapshot in time of where the electorate stands. So, no, polls aren’t predictive tools.

But now since politics has moved into the world of degenerate gambling, the focus is more on who wins instead of whether the poll is accurate or not. I’m not going to say that this is entirely FiveThirtyEight’s fault (but maybe Nate Silver’s), but they have created a culture of using polls to “predict the winner” and the polls “being right/wrong”. This means that in the eyes of two-bit gamblers, a poll that can pick the winner of the election but is off by 10% is much better than a poll that picked the loser but was only off by 0.5%. The last poll is surely the “most accurate”, but because of FiveThirtyEight’s “winner” model, the former poll would be seen as the “most accurate” in eyes of those simply looking for “the winner”.

Of course, there are many reasons to celebrate the collapse of FiveThirtyEight. But the main reason might be that these garbage polls might no longer be given any oxygen. FiveThirtyEight had been committing political malpractice for years, legitimizing these fly-by-night polling firms. Yes, Real Clear Politics has their aggregate, but it’s never been seen as statistically driven as FiveThirtyEight was.

Adios FiveThirtyEight! Good riddance!