At FiveThirtyEight this week, statistics guru Nate Silver wrote a major apologia about his 2016 predictions called "How I acted like a pundit and screwed up on Donald Trump." Its ostensible purpose was to explain a string of apparent misdiagnoses from a statistician famed for getting things right.
Inadvertently, though, it explained a lot about what we campaign journalists in general have done wrong to pave the way for the seeming outlier of a Trump nomination.
Silver's voice is a big one in our business. After calling the presidential election for Barack Obama eight months early in 2008 and predicting 49 out of 50 state results, he re-wired the minds of a generation of campaign reporters. An ideological descendant of baseball statistician Bill James (whose Baseball Abstract was an annual purchase in my home growing up), Silver retrained political analysts to think in Moneyball terms, tuning out statistical noise and focusing on the actual path to electoral victory. The naming of his new site, FiveThirtyEight, was symbolic of this new emphasis on what mattered in presidential races, i.e. electoral votes.
A candidate who scored seemingly encouraging results in national polls while performing poorly on the state-by-state electoral map was the equivalent of the baseball player who hit for high average but didn't draw walks or hit for power.
With the analyses of Silver and his crew, reporters now had a much better grasp of who was actually winning races, especially primary/nomination races, which relied on arcane delegate rules that Silver made it his business to understand.
On the campaign trail, the success of FiveThirtyEight inspired heated in-plane debates. More than once I had to suppress a laugh listening to a reporter grumble that campaigns would now be less about "the issues" thanks to Silver and his lot, as though campaign coverage hadn't been 99.9% percent a horse race already.
Then 2016 happened. The normally cautious Silver howled from the rooftops that Trump was a temporary phenomenon. As late as November 23 of last year, he wrote a piece called, "Dear Media, Stop Freaking Out About Donald Trump's Polls" that among other things noted that Trump's numbers at the time — about 25 percent of the 25 percent of Americans who identified as Republican — roughly matched the number of Americans who believe the moon landing was faked.
Silver's site repeatedly put hard numbers on Trump's chances of victory. The results weren't pretty, as he notes:
"In order of appearance — I may be missing a couple of instances — we put them at 2 percent (in August), 5 percent (in September), 6 percent (in November), around 7 percent (in early December), and 12 percent to 13 percent (in early January)."
To diagnose what went wrong, Silver among other things went back to see what the numbers said about other "Trump-like candidates." He defines those as "candidates who led national polls at some point in the year before the Iowa caucuses, but who lacked broad support from 'party elites' (such as measured by their number of endorsements, for example)."
Using these criteria, he came up with the following list of close matches to Trump: George Wallace, Jesse Jackson, Gary Hart, Jerry Brown, Herman Cain and Newt Gingrich.
Howard Dean and Rudy Giuliani were added as "borderline" cases. Silver noted each had some degree of support among "elites," dimming a bit their status as "insurgents."
Because all of these candidates ultimately failed, Silver then called upon a statistical tool called a "uniform prior," which he explained thusly:
"According to the uniform prior, if an event has occurred x times in n observations, the chance of it occurring the next time around is this:
As Silver explains it, this equation tells him that Trump had a slightly better chance of winning than he'd thought, despite the data showing that all eight previous "Trump-like candidates" had failed.
"Under a uniform prior," he wrote, "Trump's chances of winning the nomination [were] either 1 in 8 (12.5 percent) or 1 in 10 (10 percent) — still rather low, but higher than the single-digit probabilities we assigned him last fall."
To sum up, going forward, Silver would have given Trump a slightly higher chance of success than he did last year. "Basically, my view is that putting Trump's chances at 2 percent or 5 percent was too low," he wrote. "But having him at (for instance) 10 percent or 15 percent… would have been entirely appropriate."
I like Nate Silver. I was a Baseball Prospectus addict and I really enjoy his political analyses.
But this explanation — essentially X+Y = we were off by eight-to-ten points — blows off the bigger-picture problem of how pundits and analysts use misleading shortcuts and clichés to think about politics. He unwittingly references this issue in his article.
One of the things I was trying to get at in this week's feature about Trump's amazing takeover of the Republican Party is that we've all gotten this wrong, for decades.
The tone of American political coverage for some time hasn't matched the reality of what voters have been going through. Even as America lost its manufacturing base and tens of millions of people were put out of good jobs, the campaign story for years remained the same weirdly celebratory soap opera. Every four years, we whipped up audiences into a lather over the same patriotic fairy tale of political athletes engaged in high-stakes rhetorical combat while chasing the ultimate power prize, the White House.
Reporters traveled tens of thousands of miles to cover these races, but not to tell stories about people they met on the road who'd lost their jobs, been bankrupted by health problems, become addicted to pills, etc.
Instead, we traveled all that way to focus on the same candidates who'd been with us on the plane from day one. They were the players in this rolling, immensely popular sports story, and to make the game accessible, we dumbed things down as much as possible.
Candidates for instance were always divided into the same few archetypes. It's really about six in total. Each party has presumptive frontrunners, acceptable challengers, and insurgents.
The "insurgent" enters the race as a plainly unwelcome interloper, preaching politics that fall outside the Beltway conception of the norm. He or she may be a socialist like Bernie Sanders, or a libertarian like Ron Paul. Or the candidate may simply not look enough like the press' idea of a president, as was the case with Dennis Kucinich, who additionally alienated reporters with "out there" ideas like the creation of a Department of Peace (I remember trail reporters scoffing at the idea of creating a government agency devoted to preventing problems like domestic violence).
In Silver's analysis, such "insurgent" candidates have always failed in the end, which is why it made sense to predict the same for Trump this time.
The problem with this shorthand is that while it may accurately describe something, it's not the politics of the United States. There are not six basic groups of Americans, all of them healthy, polite, dressed in thousand-dollar outfits, and speaking against picturesque backdrops in perfect, poll-tested sound bites.
America instead is a place where a huge plurality of the population is underemployed, pissed off, in debt and barely keeping their heads above water. A good 15 percent or so are not even doing that well, sitting below the poverty line, living in homes without adequate heat, sanitation or food. That portion of America doesn't appear anywhere in campaign coverage, not even as background.
It would have made more sense to have different labels. If there was a Poor Peoples' Party, A Disappearing Middle Class Party, and a Minimum Six-Figure Income Party, and all of them were described as legitimate and reasonable options in the press, people would have no problem pulling levers for the candidates who actually represented them.
Instead, people were influenced by shorthand terms we in the press cooked up that, whether we realized it or not, were both inaccurate and rhetorically weighted toward the status quo.
Even though "populists" and "insurgents" often pushed policies that favored 80 to 90 percent of the population, they were inevitably described as fringe radicals riding waves of emotion. Meanwhile, the "centrists" whose policies were actually hand-crafted by lobbyists to fit to a tiny slice of upscale voters — they were the Six-Figure Minimum Candidates — were always portrayed as middle-of-the-road pragmatists who believed in "nuance" and "getting things done."
It was a Bolshevik-Menshevik situation. Populist candidates were cast as peripheral nuisances, and candidates representing a tiny minority of wealthy donors were upheld as the safe majority choice.
Silver assumes that Trump is an outlier, but the real outlier was that this upside-down capsule description of American politics held for so long. The pre-Trump era of election cycles that endlessly shifted a few points back and forth in genteel, orderly contests between two heavily corporate-funded parties shouldn't have been so statistically predictable.
Our economy has been in decline since at least the Seventies and our political system over that time increasingly disenfranchised enormous numbers of voters. It should have been a surprise whenever the "insurgent" candidate didn't win.
And the fact that the surprise now comes in the form of someone as crazy as Trump, a lot of that is on us for too long ignoring voters, in favor of this ridiculous, dumbed-down game show we created.Watch how Donald Trump has flip-flopped support on various issues over the course of this election.