Thursday, November 6, 2014

Comparing the 2014 US Senate Election Models

So how did all the modelers do? Pretty good overall. While all the models did pretty well, there were misses by all of them: specifically, none of the models predicted the outcome of North Carolina correctly.

Following Dr. Sam Wang from the Princeton Electoral Consortium, I have ranked the different models using a Brier score. It seems like my scores came out slightly different from his, probably because of the data I was able to find, but the results seem consistent.

As input the numbers provided by the New York Times Senate Model Comparison. I have converted each of the totals provided by them to the probability of a Republican win. Therefore, if they predicted a Democratic win by 75%, the number was converted to a Republican win by 1-D% resulting in a 25% chance of a Democratic win.

After calculating the Brier score for each of the models, here are the outcomes in order of best to worst (lower Brier score is better):

Place
Name
Brier Score
#1
Daily Kos
0.091
#2
Washington Post
0.104
#3
PredictWise
0.125
#4
538
0.126
#5
HuffPo
0.139
#6
NYT
0.139
#7
SanityIID
0.161
#8
PEC
0.172

As you can see, all the various models did really well.

Links:
Princeton Electoral Consortium – http://election.princeton.edu/
Wikipedia: Brier Score – http://en.wikipedia.org/wiki/Brier_score
New York Times: 2014 Senate Model Comparison – http://www.nytimes.com/newsgraphics/2014/senate-model/#comparisons

No comments:

Post a Comment