Blog

Geek Challenge Results: March Madness Part II

Geek Challenge Results: March Madness Part II

A big question that lingered after looking at the results of the geek challenge was how much luck played into the outcome against how strong any specific entry was over any other.  Most entries relied heavily on seeding and used other factors to tweak the effect of the overall seeding. Only Dan Freve chose a different approach, weighting strength of schedule heavily and actually using a negative coefficient on the tournament seed. His formula is reasonable, somewhat mimicking the RPI rating used by the NCAA’s selection committee.

First, we looked at how much randomness gave optimum results. To normalize the analysis, we looked at it as a ratio of the randomness coefficient K divided by the maximum coefficient for rating.

There was a wide variation in K values, ranging from the low of Ken (0.35 times his max coefficient) to a high of Kevin (3 times his max coefficient).  We found a near optimum K given the actual tournament results and random number sets for each entry by simple brute force, subject to using a resolution of 1/20th the maximum coefficient value for that entry.  The optimal K seemed to lie roughly around the maximum coefficient for most entries. Bruce was noticeable exception, as his random numbers did not favor a Kentucky team that his coefficients favored and that won the tournament. Choosing a low randomness in his case allowed his entry to minimize the impact of his unlucky random numbers. More on just how unlucky Bruce’s brackets were is discussed below.
 

  Actual
Score
Maximum
Score
Actual/
Maximum
Best K Actual K Best K/
Max (W,X,Y,Z)
Dean 2348 2417 97% 1.10 1.50 1.10
Dan 2259 2262 100% 7.25 7.00 1.45
Ken 2081 2149 97% 0.60 0.35 0.60
David 1938 2017 96% 3.40 8.00 0.85
Kevin 1900 2232 85% 4.00 12.00 1.00
Alex 1895 2305 82% 3.00 6.00 1.00
Bruce 1685 2070 81% 0.25 8.00 0.05

 

Alex, Bruce, and Kevin did themselves damage by choosing high K’s, while the rest of the field obtained better than 95% of their points with their chosen randomness factor. In David’s case, choosing a relatively high K had little impact on his score.

Each entry was given a unique set of 630 random numbers to pick the 63 games in each of 10 brackets. So it is possible that our winner, Dean, either had great skill in choosing coefficients, or was lucky in the random number draw, or some combination of the two. In order to figure out how much of Dean’s victory was luck and how much was choosing a good entry, we ran the random numbers to find the results for 10 different outcomes using this year’s brackets to estimate the mean score and standard deviation for each entry. Using those statistics, we determined which entries were relatively lucky, and what the win probability for each entry was. We then re-ran the analysis using last year’s bracket to see how robust the results are year to year. 

Name 2012
Actual Score
2012
Mean
2012
Std. Dev.
2012
Deviation
2012
Probability
Win %
2011
Probability
Win %
Dean Schmitz 2348 2256 205 0.4 50% 25%
Bruce Polson 1685 2171 128 -3.8 20% 30%
Alex Krejcie 1895 2167 112 -2.4 8% 14%
Dan Freve 2259 2115 155  0.9 14% 0%
David Gosse 1938 2072 130 -1.0 7% 24%
Ken Brey 2081 2061  76  0.3 1% 2%
Kevin Ferrigno 1900 1935  83 -0.4 0% 5%

The mean is an indication of how good an entry is for the specific teams and outcome of this year’s tournament. The deviation is how much luck someone had, with positive numbers being good luck and negative numbers bad luck. So Dan had the most luck, but it wasn’t enough to overcome Dean’s high mean and modest luck.

Dean’s entry dominated this year’s tournament and he had an estimated 50% chance of winning. He would have been in the mix last year as well. Bruce’s entry also stood a fair chance to win in both years. David and Alex stood a better chance last year, while Dan’s entry was much better for this year’s tournament. Ken and Kevin not only stood at the bottom of the heap in terms of average, but with low standard deviations, our brackets had little chance of making a significant impact.

Bruce and Alex led the unlucky crowd. I was surprised by just how unlucky Bruce was, as in a normal distribution only 1 in 14000 entries would be worse than Bruce’s. I had to double check his entry and that our score distribution was in fact close to normal over the range of data we produced. We ran the other 6 entries on Bruce’s random numbers and it produced similarly bad outcomes (-2.7 to -4.8 deviations) for every entry except Kevin’s—where it performed really well. (+1.0 deviation) (Note: Had I just used Bruce’s set of random numbers for every entry, I would’ve won by a landslide.)  The conclusion reached is that while our distribution is close to normal within about 2 standard deviations of the mean, larger outliers can occur more often than would be expected. This is because in some situations, the results obtained can be very dependent on one or two random numbers if they affect the bracket that results in your maximum score. In Bruce’s case, his random numbers were very unfavorable to Kentucky. Even though he rated them highly, they did not wind up winning a single one of his brackets--costing him 320 points on his best bracket and total score.

Finally, we asked the question of what were the best values possible. To answer this question, we used the same methodology above and ran the 10 entries over a broad range of parameters to narrow in on what appeared to be an optimal region. Our initial run varied all parameters from -16 to +16 using increments of powers of 2. We found that the strategy of weighting heavily on seed and tweaking the other parameters was in fact a good strategy. We set this value to 8 and varied the other parameters around 0 to try and find a near to optimal solution.

Based on the results we refined the optimum parameters down to:

W = 0 to 1
 X = -3 to -2
 Y = 2 to 3
 Z = 8
 K = 6 to 8

The combinations that produced the top mean scores are shown below. As the results show, we were able to make improvements on Dean’s entry. Since these results are very dependent on the specific outcome of this year’s tournament, there is no guarantee that they will be the best choices for next year’s tournament.

W X Y Z K Mean Std.
Dev.
0 -2 2 8 6 2385 114
0 -3 3 8 6 2383 145
0 -3 2 8 6 2382 157
0 -2 3 8 6 2381 142

 

Thanks to everyone who entered. We’re already thinking ahead to next year, where you may be asked to pick the winning formula of geography, mascot, and school colors to predict the tournament’s winner.
 

Comments

There are currently no comments, be the first to post one.

Post a comment

Name (required)

Email (required)

CAPTCHA image
Enter the code shown above: