It might be hip to say that the “polls are broken” or the “polls are wrong,” but so far this cycle, it has not been true.
There have been nine congressional (House and Senate) and governor elections with polling in 2017 and 2018, and the polling from them has been pretty good, historically speaking. Let’s take a look first at the six special House elections with polling in the final three weeks of the campaigns: Arizona 8, Georgia 6 (the special election and runoff), Montana at large, Pennsylvania 18 and South Carolina 5.
Among the 27 polls across these six contests, the average absolute error per poll was just 4.0 points. That’s actually 1.1 points better than the average poll in House specials from 2004 to 2016, which was 5.1 points. The average public poll in the recent Arizona 8 special election, for example, was off by only 4.1 points from the final result. All the polls suggested a far tighter race than the +25 Republican partisan lean of the district.
The same story can be told when looking at the three major statewide contests this cycle: Alabama US Senate, New Jersey governor and Virginia governor. Of the 31 polls conducted in the two governor contests, the average error was 5.2 percentage points or right about at the 5.1 percentage point average for polling dating from 1998 to 2015. The average error in New Jersey was just 1.7 points, compared with 6.6 points in the highly polled Virginia race.
The average error for the 16 polls in the final three weeks of the Alabama Senate race was 5.9 percentage points. That may seem high but it’s really not for a special election held in December of an off year. The average error for Senate elections not held on Election Day of a midterm of a presidential year from 1998 to 2016 was 5.8 percentage points. Among the three gold standard pollsters (i.e. those who use live interviews, call cellphones and are transparent about their methodology), the average error dropped to only 3.8 percentage points.
All the polls in Alabama signaled that the race could be tight. As I’ve said before, polls are tools. In Alabama, these tools told us that the race was going to be far tighter than we’d expect for a Senate race in a state Trump won by nearly 30 points.
Of course, many of the complaints about polling after 2016 had little to do with the accuracy of individual polls. They had to do with the fact that polling overall seemed to be biased against Republicans. I always thought these critiques were lousy because arguments were made in the reverse direction after 2012. The bias against a particular party in the polls, however, changes from year to year and is more random than anything else.
That’s why it’s not surprising that the polls have not been biased against the Republicans in the off-year cycle. Across the eight races this cycle with polls in which there was only one Democrat and only one Republican, the Democrats have done on average 1.5 percentage points better than the average poll in the final three weeks of the campaigns.
The errors have been mostly random. Republicans did better in the Georgia 6 runoff and Pennsylvania 18 special. Democrats did better in the Alabama Senate and Virginia governor’s races. The randomness of the errors is exactly what you’d expect given history.
Turning to the fall campaign this year, the randomness of the poll errors so far is a sign that neither side should be expecting to do better than the polls. Democrats cannot count on outperforming their polls even if the generic congressional ballot is not showing the same shift toward the Democrats as the special elections have. Republicans cannot count on the polls being off, even though Republicans and Trump did better than the polls in 2016.
None of this is to say the polls will be perfect. Rather, we just don’t know who is going to do better than the polls suggest.