How did draft position affect fantasy football league performance in 2024? (12-man leagues, snake draft)

[u/rsrgrimm]

To assess how draft position affected league performance, I looked into over 400 12-man leagues (all snake drafts) and plotted win ratio, normalized points earned (normalized within a given league to account for various scoring and roster settings), and final league ranking for each draft position.

Surprisingly, 1st pick performed worst on average across all metrics.

League data collected from Sleeper API.

[u/SkilledB]

This reads like ”I can’t read graphs”. Of course they all kinda look the same because the tails of the box and whisker are all at 1 and 12, meaning plenty of finishes at the top and bottom from all draft spots. What’s inside the box are the significant takeaways from this.

[u/ScientistFromSouth → u/SkilledB]

To be fair, box plots then don't make sense in this case because they are designed for continuous data and not rank order. You literally can't read them for discrete data. Second his statistical findings were that statistically first draft finished average place 6 and 10th player finished average place 7 which he reported to like 5.7 vs 6.9 (which once again doesn't make sense in a discrete rank order). 1 place difference in a game with twelve players feels like an insignificant net change and basically random.

[u/SkilledB → u/ScientistFromSouth]

First, agreed this visualization type definitely isn’t the best for this data.

However, if the sample is big enough to not produce random results, then the 1.2 difference is huge. There isn’t supposed to be a systematic benefit to any single draft spot using the snake draft format. This data (again, assuming a large enough sample) shows that last year, there was a non-neglible difference caused by draft position.

[u/ScientistFromSouth → u/SkilledB]

So a few issues:

1. The guy ran ANOVA to justify statistical significance even though that only says that there is at least one statistically significant non zero slope for a linear model fit to ideally independent continuous data which isn't true for rank order data that isn't independent or continuous.

2. He seems really hung up on this 1.2 outcome and the "largest p value". He's likely p hacking. I would like to see a pairwise interaction matrix of all of the relative interactions and relative finish positions. I would also like to see some multiple test correction like Turkey HSD. However he would need to run something for rank order data specifically.

3. I know this sub like aesthetic plots, but come on. For plots this wide, some grid lines that show the 50% win neutral probability or color coding for median win rate above below 50% would make this infinitely better