The Washington Post

Why the Senate forecasting models differ

“There’s not a data journalism bubble (IMO) but there’s quite possibly a senate forecasting model bubble,” tweeted Nate Silver yesterday after Election Lab was launched.  Thus far, I think there are three formal forecasting models out there: ours, The Upshot’s (affectionately known as Leo), and that of political scientist Alan Abramowitz.  In addition, there are Silver’s initial expectations (not yet formalized in a model, I believe) and the informed views of handicappers like the Cook Political Report, the Rothenberg Political Report  and Larry Sabato’s Crystal Ball.  (The Upshot has a useful chart summarizing the views of all.)

There is a rough consensus that the GOP is likely to gain Senate seats, but no necessary consensus on whether they will gain enough to control a Senate majority.  Our current forecast gives the Republicans a higher likelihood of taking the Senate (82 percent) than does The Upshot’s (54 percent).  The Upshot’s team provided some thoughts yesterday on this discrepancy.  Here are some of ours.  (See also my conversation with Chris Cillizza.)

First, it’s important to note how similar the models are.  Both models capture whether it’s a midterm or presidential year, Obama’s vote share in each state in 2012, and some sense of the national mood. We capture national mood via measures of economic growth and presidential approval; The Upshot relies on the generic ballot.  (In our experience, substituting one measure for the other doesn’t affect the forecast very much.)  Both models also have a measure of each candidate’s previous highest elective office (if any).

In terms of the differences between the models, four main things stand out to me:

1) Leo is based on elections from 1992-2012, while Election Lab is based on 1980-2012.  As we have noted, the time period a model uses can affect its conclusion.  A model based on 1952-2012 generates more favorable forecast for the Democrats than one based on 1980-2012.  To be sure, there is no one correct answer as to which time period is more appropriate. As a general rule, forecasters tend to prefer more elections to fewer — that is, more data to less data. But there is always the risk that elections from decades ago are not very similar to elections today.  Moreover, certain factors — like fundraising — are not available for elections in the 1950s and 1960s.

2) Leo includes approval ratings for incumbent Senators.  For example, this could affect the forecast for North Carolina.  Kay Hagan’s approval rating is only 41 percent, according to one recent poll.  This could help explain why The Upshot sees North Carolina as a toss-up, while our current model gives Hagen an advantage.

3) Leo includes fundraising.  We’ll be adding this to Election Lab soon — for both the House and Senate. Preliminary analysis suggests that it will render the two sets of forecasts more similar: For example, the Democratic advantage in Hawaii, New Jersey and Minnesota will increase.  Our forecast for Iowa will shift to a toss-up.

4) Leo includes polls.  We’ll be adding this as well.  Early polls are certainly related to the eventual outcome in Senate races, but the predictive power of pre-election polls increases sharply as the campaign goes on.  For this reason, Leo, like similar previous models, puts more weight on other factors than polls, but the polls could also explain a few discrepancies.  Again, North Carolina comes to mind, since polls show Hagan in a tight race against her likely opponent, Thom Tillis.  At the same time, the polling average suggests a 64 percent chance that she is leading Tillis —  which is not that far out of line with our forecast.

So, in short, most of the discrepancies between these two models won’t persist.  Indeed, to the extent that these and any other models rely increasingly on a polling average as the election approaches, they’ll be generating similar Senate forecasts.  This is one reason why the 2012 presidential election forecasts that relied mostly or entirely on polls — 538, Votamatic, Princeton Election Consortium, Pollster — were all saying the same thing by the end of the campaign.  Personally, I think it’s valuable to have as many models as possible.  Indeed, the average of the forecasting models is likely to be the best prediction of all.

One last question might have occurred to some readers: why roll out a forecast based on a model that’s not “complete” — that is, without factors like fundraising or polls?  There actually is some value in doing so, as I’ve written about here.  A model that rolls out in a sequential fashion — as ours has here and here — helps identify the role of specific factors.

For example, the fact that an initial forecast based on 1980-2012 was favorable to Republicans told us something about the basic political landscape.  Then when the forecast became more favorable to Republicans after we took into account the previous political experience of candidates, we learned something about the quality of the candidates that Republicans were recruiting.

If incorporating fundraising changes the forecast, then we’ll know something else about the potential strength of the candidates.  If adding polls shifts the forecast, then we can investigate what the polls might be capturing that the model isn’t — especially factors unique to a race.  Moreover, we’ll also be able to identify whether the polls actually move toward our earlier forecasts, as often happens in elections.

In short, there are lots of ways to extract useful knowledge from a forecasting model (including even when the model proves to be wrong!).  We’ll have more to say about what we’re learning in the coming months.

John Sides is an Associate Professor of Political Science at George Washington University. He specializes in public opinion, voting, and American elections.

The Freddie Gray case

Please provide a valid email address.

You’re all set!

Campaign 2016 Email Updates

Please provide a valid email address.

You’re all set!

Get Zika news by email

Please provide a valid email address.

You’re all set!
Show Comments
The New Hampshire primary is Tuesday. Get caught up on the race.
New Hampshire primary: What to expect
New Hampshire will hold a traditional primary just eight days after the Iowa caucuses. Polling in the Granite state has historically been volatile in the final weeks before the primary. After the Iowa caucuses, many New Hampshire voters cement their opinions.
The Post's Ed O'Keefe says ...
Something has clicked for Bush in New Hampshire in the past few days. What has transpired by no means guarantees him a top-tier finish in Tuesday’s Republican primary here, but the crowds turning out to see him are bigger, his delivery on the stump is crisper and some of his key rivals have stumbled. At the least, the developments have mostly silenced talk of a hasty exit and skittish donors.
The feminist appeal may not be working for Clinton
In New Hampshire, Sen. Bernie Sanders is beating Clinton among women by eight percentage points, according to a new CNN-WMUR survey. This represents a big shift from the results last week in the Iowa caucuses, where Clinton won women by 11 points.
New Hampshire polling averages
Donald Trump holds a commanding lead in the next state to vote, but Marco Rubio has recently seen a jump in his support, according to polls.
New Hampshire polling averages
A victory in New Hampshire revitalized Hillary Clinton's demoralized campaign in 2008. But this time, she's trailing Bernie Sanders, from neighboring Vermont. She left the state Sunday to go to Flint, Mich., where a cost-saving decision led to poisonous levels of lead in the water of the poor, heavily black, rust-belt city. 
55% 40%
Upcoming debates
Feb. 11: Democratic debate

on PBS, in Wisconsin

Feb 13: GOP debate

on CBS News, in South Carolina

Feb. 25: GOP debate

on CNN, in Houston, Texas

Campaign 2016
State of the race

To keep reading, please enter your email address.

You’ll also receive from The Washington Post:
  • A free 6-week digital subscription
  • Our daily newsletter in your inbox

Please enter a valid email address

I have read and agree to the Terms of Service and Privacy Policy.

Please indicate agreement.

Thank you.

Check your inbox. We’ve sent an email explaining how to set up an account and activate your free digital subscription.