Dan Hopkins' post last week on (the lack of) partisan bias in pre-election polling gives us some sense of the overall accuracy of trial-heat polls, albeit in Senate and Gubernatorial races. What I want to look at today is how well district-level trial-heat polls perform in U.S. House elections, with an eye toward eventually using them to predict outcomes in individual races.
Using data provided by the folks at pollster.com, I've taken average poll readings over the last forty-five days of the campaign from 91 contests in 2006 and 89 contests in 2008. The figure below shows the relationship between the averages and the eventual outcome. In both years, there is a strong relationship, suggesting that poll leaders generally go on to win House elections.
Certainly it is the case that in 2006 the polls became better predictors of final outcomes as election day approached. In 2008, however, there wasn't a very steep increase in the predictive accuracy of polls as the election drew nearer. More to the point, the overall correlation across the forty-five days prior to the election (first figure) is virtually identical to the correlation found for the last two-weeks of polling. Also (though recognizing the cases are different due to restricting analysis to districts with polls in the last two weeks), only 77% of poll leaders (combining both years) in the last two weeks of the campaign went on to win their elections, compared to 85% using the forty-five day average.
Although these data illustrate a fairly obvious point--that candidates who lead in the polls tend to win-- it is comforting to see the strong connection between polls and outcomes, especially given that the data come from many different types of organizations, polling in a variety of different local contexts. No doubt, taking some of those factors into account might shed light on conditions that make polls better or worse predictors of outcomes. But that will have to wait for another day.