In 2016, as I was preparing to write my “Why Hillary Will Win” piece, I decided to have my able then-assistant, David Byler (now of Washington Post fame), do a bit of research. His job was to look up the share of the electorate that pollsters were anticipating for whites without college degrees and for African Americans.
What he found put an end to the piece. It seemed a big bet was being placed on 2012 levels of black turnout occurring in 2016 and, more importantly, that pollsters were badly underestimating turnout for whites without college degrees. In previous years, that hadn’t really mattered – whites with and without college degrees voted Republican at roughly the same levels. Underestimating the share of whites without college degrees and overestimating whites with college degrees wouldn’t have mattered in 2012 or 2008, because their votes were fungible.
On a hunch, I went back and looked at the poll errors for 2013-15, and it became apparent that the errors for 2016 followed much the same pattern: They were concentrated in areas with large numbers of whites without college degrees. Indeed, the size of the poll error correlated heavily with whites-without-college-degree share (p<.001); you could explain about one-third of the difference in the size of poll miss just from knowing the share of the electorate that was whites without a college degree.
We all know what happened next. Trump surprised observers by winning states that Republican presidential candidates hadn’t carried since Debbie Gibson and Tiffany fought it out for top placement in the Top 40 charts. The misses were particularly pronounced in the Midwest.
Most pollsters attributed the misses to the failure to weight by education, and when one brings up the errors from 2016 with respect to the 2020 election, the answer typically is “pollsters now weight by education, so they’ve fixed it.
But have they? We actually have a pretty nice sample from 2018 to draw upon. If pollsters have really figured out where they went wrong in key states in 2016, we should see a marked improvement over 2016 and 2014.
So, I went back and looked at the Democratic bias in the polls for swing states in 2014, 2016, and 2018. I could not use North Carolina, since there was no statewide race there in 2018. One problem I encountered is that in 2018 many states were under-polled, so RCP didn’t create an average. I’ve gone back and averaged the October polls for those states, if available (note that we don’t have three polls in October for Minnesota in 2016, hence the asterisk there). As a check on this approach, I’ve also included the error from the 538 “polls-only” model for 2018.
The results are something of a mixed bag, but overall it isn’t clear that the pollsters have really fixed the problem at all. While the bias toward Democrats was smaller in 2018 than in 2016, the bias overall was similar to what we saw in 2014, especially in the Midwest. If people remember, the polls in 2018 suggested that we should today have Democratic governors in Ohio, Iowa and Florida, and new Democratic senators in Indiana, Missouri and Florida. Obviously this did not come to pass.
Moreover, almost all of the errors pointed the same way: Republicans overperformed the polls in every Midwestern state except for Minnesota Senate/governor and Wisconsin Senate (none of which were particularly competitive). This is true, incidentally, across the time period: We see marginal Democratic overperformances in the Michigan and Minnesota Senate races in 2014, but otherwise pollsters have consistently underestimated [ … ]