Comparing the 2014 US Senate Election Models

So how did all the modelers do? Pretty good overall. While all the models did pretty well, there were misses by all of them: specifically, none of the models predicted the outcome of North Carolina correctly.

Following Dr. Sam Wang from the Princeton Electoral Consortium, I have ranked the different models using a Brier score. It seems like my scores came out slightly different from his, probably because of the data I was able to find, but the results seem consistent.

As input the numbers provided by the New York Times Senate Model Comparison. I have converted each of the totals provided by them to the probability of a Republican win. Therefore, if they predicted a Democratic win by 75%, the number was converted to a Republican win by 1-D% resulting in a 25% chance of a Democratic win.

After calculating the Brier score for each of the models, here are the outcomes in order of best to worst (lower Brier score is better):

Place	Name	Brier Score
#1	Daily Kos	0.091
#2	Washington Post	0.104
#3	PredictWise	0.125
#4	538	0.126
#5	HuffPo	0.139
#6	NYT	0.139
#7	SanityIID	0.161
#8	PEC	0.172

As you can see, all the various models did really well.

Links:
Princeton Electoral Consortium – http://election.princeton.edu/
Wikipedia: Brier Score – http://en.wikipedia.org/wiki/Brier_score
New York Times: 2014 Senate Model Comparison – http://www.nytimes.com/newsgraphics/2014/senate-model/#comparisons

US Senate Final Update

These are the final predictions that will be released from "El" on this election cycle. It looks like it is going to be a Republican blowout unless there is something systematically wrong with the polling.

El is predicting 48 seats for the Democrats and Independents, and 52 seats for the Republicans. If Orman wins and caucuses with the majority party, that will make yield a total of 47 seats for the Democrats and Independents, and 53 seats for the Republicans and Independents.

The individual probabilities for each state as predicted by El are as follows:

And in non-graphical form:

Alaska

Sullivan 56% chance of winning

Begich 44% chance of winning

Arkansas

Cotton 86% chance of winning

Pryor 14% chance of winning

Colorado

Gardner 77% chance of winning

Udall 23% chance of winning

Georgia

Perdue 87% chance of winning

Nunn 13% chance of winning

Iowa

Ernst 68% chance of winning

Braley 32% chance of winning

Kansas

Roberts 37% chance of winning

Orman 63% chance of winning

Kentucky

McConnell 95% chance of winning

Grimes 5% chance of winning

Louisiana

Cassidy 95% chance of winning

Landrieu 5% chance of winning

Michigan

Land 5% chance of winning 

Peters 95% chance of winning

New Hampshire

Brown 30% chance of winning

Shaheen 70% chance of winning

North Carolina

Tillis 24% chance of winning

Hagan 76% chance of winning

Adding all these seats together gives us the following results for the elections. While there are some ties listed, in the prediction, any ties go to the one with any lead whatsoever. The easiest way to do this is to only look at the short term prediction. In the following chart, a "1" signifies that the state is predicted to be won by the Republicans. A "0" signifies that the state is predicted to be won by Democrats.

Finally, we want to look at the sums of all these numbers together with the states that either aren't up for election this year or are considered safe seats by both the Democrats and Republicans.

US Senate Update for October 30, 2014

The races are stabilizing according to the El model. Republicans are still quite likely to take control of the Senate after the midterm elections.

Michigan, New Hampshire, and North Carolina are likely to go Democratic.

Kentucky, Louisiana, Iowa, Arkansas, and Georgia are likely to go Republican.

The remaining three states that I have been following are a little bit more erratic and unpredictable. I want to look at Colorado, Kansas, and Alaska a little bit closer.

Colorado has been leaning toward the Republicans for a while now. Recent polls that have been released, however, have had it tilting more toward a tie or even giving Udall (D) a slight lead over Gardner (R). But most of these polls are from Democratic partisans. While I'm not insinuating that the Democrats would purposefully skew the polls in their favor, I would like it better if there were some polls released from neutral or even Republican partisans to balance it out. This is a state that will need to be watched.

Kansas and Alaska, on the other hand, are closer to being actual ties. Their polling seems to have them switching hands quite frequently. Currently Kansas is leaning toward the Independent candidate, Orman; and Alaska is leaning toward the Republican candidate Sullivan (R). Will these slight leads continue to hold through the election or will they change hands yet again?

Update for October 27, 2014: Statistically, Republican Continue to Gain

The latest run of El shows that the Republicans continue to make slight gains almost across the board. This has the effect of solidifying their lead and increasing the chance that they will hold a majority of US Senate seats even before any potential runoff elections.

One of the few bright points for Democrats might be Alaska. While polls are always difficult to do accurately in Alaska, the recent polling has been showing improvements for Begich (D) in addition to the staggering number of people that his campaign has contacted in person. Putting these two factors together gives the Democrats at least a fighting chance in Alaska. Only time will tell whether Begich's get-out-the-vote plan will provide enough of a boost to counter Sullivan's (R) apparent lead in the polls.

Arkansas seems to be slipping further from the Democrats. Will the recent push by former President Bill Clinton make a difference? Has President Clinton already failed to produce results? While it is always possible for people – especially the undecided – to change their minds at the last minute, it looks more like anyone not already firmly supporting Pryor (D) is leaning toward Cotton (R).

Recent polling shows Colorado and, surprisingly, Georgia to be very volatile. There isn't enough stability in the polls for El to predict which candidate is actually leading. The short term predictions show that Udall (D) has at least tied Gardner (R) in Colorado, but we will have to see whether this is a polling anomaly or whether this is the start of a last minute movement. The long term predictions are still showing in Gardner's (R) favor. As for Georgia, it is trending in the opposite direction. In Georgia, the short term is trending in Perdue's (R) favor.

Iowa remains very close, but despite this closeness, Ernst (R) continues to maintain a lead even if it is by the closest of margins. While there is one recent poll that shows Braley (D) in a tie race, all the other recent polls are still pointing in Ernst's direction.

Kansas is finally showing more what I would expect. El's long term prediction shows that the race is now a tossup between Roberts (R) and Orman (I). Since the Republicans have increased their funding and the expertise of their personnel in Kansas after the strong showing of Orman, we will have to see whether the polls continue their trend past the even mark and toward a Roberts lead.

Kentucky? Almost assuredly Republican. Recent polls are showing McConnell at the greatest possibility to win that El will provide.

Louisiana? Most likely Republican as well. What will matter as far as Louisiana goes is whether Cassidy (R) can gain enough votes to avoid a runoff election with Landrieu (D).

Michigan and New Hampshire both look like wins for the Democrats. In Michigan, El gives Peters (D) a 95% chance of winning; that's the largest chance that the model will predict. And while not quite as good in New Hampshire, El gives a solid prediction of victory to Shaheen (D).

As for North Carolina, El still gives the race to Hagan (D) over Tillis (R) despite the recent close polling numbers by Tillis. The margin is just enough in Hagan's favor for El not to consider the race a tie from a numerical perspective.

October 23, 2014 US Senate Update: Bad News for Democrats

The US Senate races seem to have mostly stabilized. This is not good news for the Democrats. Most of the states we are watching have stabilized in favor of the Republicans. While there is still a statistical chance that the Democrats can retain control of the US Senate, the probabilities are looking rather remote.

The statistics are favoring the Republicans heavily. In addition, if Orman wins and caucuses with the Republicans, decrease the Democrat count by one.

Here is how the statistics look from state to state:

I would argue that the mathematical model doesn't show the complete picture in Kansas. Because of Taylor (D) dropping from the race and Roberts (R) and the Republicans pouring money and talent into the state, the recent polls are showing the state in a virtual tie. Combined with the lack of a polling history with only Roberts and Orman (I) on the ticket, and I believe that our model is misleading.

If you are interested in seeing all the latest charts, please go to the "El -- US Senate Race at a Glance" page.

Georgia Goes North, Kansas Goes South, and Colorado Returns to the Middle

"El", the SanityIID model for the Senate race has been updated for October 19, 2014. The full set of charts can be found on the "El – US Senate Election at a Glance" page. Overall, the Democrats chances of maintaining control of the Senate have improved slightly to 33.4%, but the Republicans still hold a statistically significant probability of 63.4% of taking control of the Senate.

There haven't been many state level changes since my last writeup. Michelle Nunn (D) has the chance to gain ground in the Georgia race for the US Senate after the Democrats poured $1 million into the race. While Nunn (D) hadn't been far behind Perdue (R), she had been behind consistently. The latest three polls, however, have either shown Nunn (D) tied or slightly leading Perdue (R). While these results are interesting, it is too early to determine whether this is a short term gain or the start of a real movement for the Democrats in Georgia.

Kansas is having the opposite movement from Georgia. Republicans have come to the rescue of Senator Roberts (R). Roberts has access to new people and more money in an attempt to keep Orman (I) from taking the traditionally Republican seat. While Orman might caucus with either the Democrats or the Republicans, if Robers wins then the seat stays firmly in Republican control. And the money and expertise has been making a difference in Kansas. The latest polls show that Roberts is either tied or slightly ahead of Orman. The odds of Kansas staying in Republican hands is increasing with each passing day.

And a tiny bit of good news for Democrats in Colorado. Recent polling there has shown that, in the short term at least, the Democrats are pulling the state back to a tie status. Both of the recent polls that were released in Colorado were partisan polls done for the Democrats, so the gains have to be taken with just a hint of skepticism. Still, El takes all polls whether partisan or not and lets them average out over time. Even with the new polls, its only enough for Udall (D) to pull the state back to a short term tie with Gardner (R).

Senate Race updated for October 16, 2014

Polling has been updated at "El -- US Senate Race at a Glance". Republicans continue to solidify their lead as we approach election day.

The most probable outcome is for Republicans to win 51 seats. If Orman then caucuses with the controlling party, that will give the Republicans a grand total of 52 seats.

One thing that has changed is that the second most probable outcome is that the Republicans outright win 52 seats, and, again, if Orman caucuses with them, getting a grand total of 53 seats.

More graphs, as well as a state by state breakdown, can be found at the "El -- US Senate Race at a Glance" page.

October 11, 2014 US Senate Race Updated

The "El -- US Senate Race at a Glance" page has been updated. Quick overview? No significant changes since last run.

If you look at the graphs on the overview page, you can see that the model makes an error in a calculation. The error is caused by the latest polling being so stable. Since the short term polling is so stable, that collapses the "normal curve" for the statistics, an error that El doesn't currently account for. The end result is that El shortchanges the Democrats one seat in the state by state tallies. The overall statistics for the other graphs, however, are not in error.

US Senate Update for October 9, 2014

The quick overview? The Republicans take a statistically significant lead.

It has been a while since I have posted an update, so let's give every state that El (the SanityIID model) is following.

In Alaska, El is still screaming a warning about a lack of data. There just isn't enough polling happening in Alaska for a race that appears to be this close. Still, the recent polls are all going in Sullivan's (R) favor.  The long term probability for a Republican win in Alaska is now at 65%. Expect this percentage to continue to rise unless Begich (D) starts getting better polling numbers.

The new polling in Arkansas shows that the state is slipping more firmly into Cotton's (R) hand. Current long term estimates show Cotton with an 81% chance of winning the state.

Colorado appears to be leaning Republican with Gardner (R) having a 65% long term chance of taking the state. Udall (D) has slipped behind and the race seems to be stabilizing in the Republican's favor.

Georgia seems to be leaning toward the Republican's as well. The lastest polls show Perdue (R) with a 77% probability of taking the state over the long term. Nunn (D) has been consistently falling behind in the polling. If upcoming polls don't any increase for Nunn, expect the probability of Perdue to win to increase over time.

Democrats are losing ground in Iowa as well. Ernst (R) currently has a 71% long term chance of taking the state. While the mid term and short term chances are a little less stable, they are all still showing strong support for Ernst (R) over Braley (D).

The Independent candidate, Orman (I), is still showing well in Kansas. There have been a few recent polls that are startling in their sudden change, but the long term chances are still showing solidly for Orman (I) over Roberts (R). We will have to wait and see whether the most recent polls in Kansas are outliers or whether they are the start of a comeback for Roberts.

Long term predictions for Kentucky are showing McConnell (R) with a commanding lead over Grimes (D). The long term predictions show McConnell (R) with an 86% chance of winning the election. The short and mid term predictions aren't quite as rosy for McConnell, but they still show him with a solid lead.

Louisiana? Republican. Virtually all polls have been favoring Cassidy (R) over Landrieu (D). While El does give a data warning for not enough polling, there is still a 78% long term chance of a Republican victory in Louisiana. The mid and short term probabilities only increase for the Republicans.

Michigan is the first good news story for the Democrats. Long term, Peters (D) has an incredible lead over Land (R). The long term probability of Peters winning is pegged at the limit of El's accuracy. It is also reported in the news that the Republican's are going to pull their television commercials in favor of spending that money on more competitive races. While there will always be a statistical possibility that Land (R) could win, the chances are very, very small indeed.

New Hampshire also looks good for the Democrats. Shaheen (D) is still holding a stable lead over Brown (R). Long term, the Republicans only have a 23% chance of having New Hampshire in their column.

And finally, the last state that the El model is following: North Carolina. North Carolina is another spot that looks good for Democrats. While the polling is close, it is also stable. Not a single poll is showing in the Republicans' favor. Hagan (D) is leading Tillis (R) by a margin that pegs the model in the Democrats' favor. The only clouds on the horizon is that it wouldn't take much of a change to begin dramatically swinging North Carolina toward the Republicans.

For the full details and charts of the latest model run, check the "El – US Senate at a Glance" page.

Senate race update: October 8, 2014

I am on vacation. Because of this, there won't be any write-up today on the Senate race. The totals and graphs of the latest polls have been updated, however. For a quick overview of how the Senate race is shaping up, please check the "El -- Senate Race at a Glance" page accessible from the top of this site.

Raw Graphs without explanation: