Published August 2026. Figures below were last reconciled against the live board Aug 10, 2026, after a corpus-integrity pass that removed betting-prop, DFS, and retrospective-commentary claims that had been miscounted as live start/sit calls, moving both publisher ranks and the sit-edge percentages below (the pattern itself held). Every claim links to its evidence and updates as the board does; check the linked record for the current number.
A few weeks ago I published a ranking of fantasy football podcasts by verified accuracy: every start/sit call they made on air, checked against what actually happened on the field. About 350,000 people read it, and the most common question in the comments was some version of the same thing:
"Fine, but do the good ones stay good? Will 2024's rankings predict 2025?"
Fair question. It's also a dangerous one for me, because the easiest possible marketing for this site is "here are the trustworthy shows. Done." If accuracy persists, I get to sell a list. So I pre-registered the test (methodology locked in writing before seeing any results) and ran it against three seasons of verified calls.
Here's what the data said. It's not what my marketing department (me) was hoping for.
Fourteen shows made enough verified calls to be judged fairly: at least 100 scored start/sit claims in both 2024 and 2025. (The original study cohort was 12; two more shows crossed the volume floor as the database grew.) Rank them by accuracy in 2024. Rank them again in 2025. Ask: do the rankings agree?
The rank correlation between seasons was 0.23: statistically indistinguishable from zero (permutation p = 0.43, and the confidence interval spans from meaningfully negative to meaningfully positive). Of the top half of shows in 2024, 4 of 7 stayed in the top half in 2025, a coin flip. Same for the bottom half.
The receipts make it concrete:
I also checked specifically for the exception: a show that topped the field both seasons. There isn't one. No publisher was #1 in both 2024 and 2025. I looked for that on purpose, because I'd rather report the inconvenient exception than have you find it.
Honesty about the limits, because that's the entire point of this site: fourteen shows is a small sample. At this size, the test only had the statistical power to detect a strong persistence effect. A moderate one could exist and slip through undetected. So the precise claim is: if podcast accuracy carries over from season to season, the effect is too weak to see even in the largest verified database of these calls that exists. Nobody should be picking a 2026 show based on its 2024 record: that much the data supports plainly.
And one robustness note I'm genuinely proud of: since the original run, the cohort grew from 12 to 14 shows and one publisher's feed was reinstated, materially changing their numbers. I re-ran the entire pre-registered battery on the current data. Every conclusion held. This result is not fragile to the database evolving, which is good, because the database evolves every week.
One finding refused to die, and it's the most useful thing in here for your actual roster decisions:
Shows are meaningfully better at telling you who to sit than who to start.
| Season | Sit accuracy | Start accuracy | Edge |
|---|---|---|---|
| 2023 | 65.5% (n=710) | 51.8% (n=3,413) | +13.7 |
| 2024 | 62.8% (n=3,081) | 49.5% (n=14,470) | +13.3 |
| 2025 | 62.1% (n=3,098) | 48.4% (n=15,831) | +13.6 |
About a 13-point edge, every season, across every show. "Bench him" advice carries real signal. "Start him" hype is where accuracy goes to die. I ran a stricter version of this at the show-season level too: of 33 show-seasons with enough calls in both directions, 16 matched the full pattern (start accuracy indistinguishable from a coin flip and sit accuracy significantly better than one). About half. The pattern is real; it just isn't universal.
If last year's winners don't predict this year's, then a static "here are the trustworthy shows" list (the thing I could most easily sell you) is the wrong product. It would be stale by the time you drafted.
That's not a reason to stop keeping score. It's the reason keeping score has to be continuous. Analyst performance changes over time; that's exactly why yesterday's rankings aren't enough, and why every number on this site updates as games resolve. Yesterday's winners aren't guaranteed to be tomorrow's. Scout keeps score in real time: that's the product, and this study is why.
The database behind this: 40,451 verified start/sit claims across 23 publishers and three seasons. The pipeline is AI-assisted and outcome-gated: software reads show transcripts and extracts the calls, every extracted claim must quote the exact words said on air, and, this is the part that matters, no AI decides whether a call was right. Sleeper's official box scores do. The AI reads; the scoreboard grades. Every claim on this site links to the words, the date, and the outcome, so you can check any number in this post yourself. When a figure couldn't be reproduced from a committed script during this analysis, it was thrown out rather than published.
This post is also me keeping a promise: the year-over-year test was the follow-up I owed everyone who asked when the original rankings went up. The result cuts against my own easiest sales pitch, and it's published anyway, because a credibility site that hides its inconvenient findings isn't one.
This season I'm running a small experiment: every Sunday morning, a short hand-written email with the verified calls on your players: what the shows said, what their record is on exactly that kind of call. I'm writing these myself, so it's capped at 100 people while I learn what's useful. Enter your Sleeper username and it covers your actual roster automatically.
Every number in this article traces back to a receipt. See the live board, or how the bar is actually computed.