2016 – 2025 · nflverse + BDB · 328,000 plays
The look does tell you something.
The question is how much.
A defender who knows only the down, distance, score, and game clock identifies run vs. pass correctly 70.9% of the time. Add the full pre-snap picture — formation, personnel, motion, hash, box count — and that rises to 76.4%. That 5.5-point gap is real signal across 328,000 plays spanning 10 seasons. The challenge is measuring which teams give more away consistently.
Off / Run vs Pass
0.841
Look model AUC · +0.051 over situation
Def / Man vs Zone
0.679
Look model AUC · +0.014 over situation
Def / Blitz
0.656
Look model AUC · +0.027 over situation
ATL off/pass tell (2025)
0.153
bits/play · highest 3 yrs running
What "tell_bits" means: log-loss gain (in bits/play) from knowing the pre-snap look on top of game situation. Zero = the look adds nothing. Larger = the defense can read it more easily. League average is 0.076 bits/play for run-pass in 2025.
Offense · Run vs Pass · 2025
Who's most readable pre-snap?
Empirical-Bayes shrunken estimates. Each team's raw tell_bits is pulled toward the league mean based on its bootstrap uncertainty. Error bars are 95% posterior intervals. Teal = CI above league mean. Blue = overlapping mean. Dim = CI below mean. Orange = ATL.
Run-Pass Tell — EB shrunken tell_bits with 95% CIs
Vertical line = league mean (0.076). ATL, CAR, and TB have CIs that clear the mean.
Reliability Analysis
Why team rankings are noisy — and what that means
Half-season stability (Spearman ρ = −0.01, p = 0.96) and cross-season stability (ρ ≈ 0 in both adjacent pairs) show rankings shuffle. That's a sample size problem, not a "no signal" problem — the EB analysis finds τ² = 0.00058 (SNR ~0.66), meaning roughly 60% of the observed spread between teams is real.
Within-season: weeks 1–9 vs 10–18
ρ = −0.01. Perfect stability = along diagonal. ATL stays high; most teams scatter.
Cross-season: 2023 → 2024 → 2025
ρ ≈ 0 in both pairs (not significant). ATL is the only team consistently above the pack.
The signal IS there — at the play level. "Which teams are most readable" is noisy at ~1,000 plays/team/season. But "which specific looks are predictable for ATL" is a sharper, more stable question. tells team ATL surfaces the exact formation × personnel × motion combos that gave away the call — that's what a defensive coordinator actually scouts.
BDB 2026 Supplementary Data · 2024 Test
Can the offense tell what coverage is coming?
Multiclass prediction of Cover 0 / 1 / 2 / 2-Man / 3 / 4 / 6 from the pre-snap look. Trained on 2023 BDB data, tested on 2024 weeks 14–18. The key finding: the situation model (35.3%) beats the look model (34.3%). Coverage calling is more situational than look-driven — team identity makes it worse (32.8%), showing scheme changes dominate year to year.
shell_gain by defense (top 16)
acc_look − acc_situation. Positive = pre-snap look reveals the shell beyond game context.
Overall accuracy — 7-class prediction
Why situation > look here: Coverage shells (Cover 3 on 3rd-and-long, Cover 1 in two-minute drill) are called primarily based on down and distance, not the offensive formation. The look gives you ~1% more accuracy than knowing the game situation alone.
BDB Tracking Data · Supervised Learning
Man vs zone from tracking: a trained classifier
Rather than hand-crafting heuristics, we trained a HistGradientBoostingClassifier on 10 features derived from the full post-snap tracking sequence (frames 1–22, ~2.3 sec). Ground truth: charted is_man labels (2023). Test: held-out weeks 14–18.
Coverage Classifier AUC
0.752
vs 0.500 random baseline
Accuracy
69.5%
vs 56.8% majority-class baseline
Training plays
10,034
2023 wks 1–13 · test: wks 14–18
Feature importances (permutation)
mean_dist_to_primary = how close each defender stays to the receiver they were nearest to at snap across the full route. Man defenders stay ~1 yard closer on average.
The key insight: distance-based proximity is what matters, not identity churn. In man coverage, defenders stay closer (mean 7.0 yds) to their snap-assigned receiver throughout the route vs. zone (8.1 yds), with lower variance (std 1.18 vs 1.26). The "nearest-receiver switches" signal we hypothesized had negligible importance in the fitted model.
Progress
What we built
Three workstreams completed. Each one moves the core question forward: are team-level readability rankings meaningful, and what drives them?
✓ DONE
01
Historical expansion
Expanded from 4 to 10 seasons (2016–2025). 328k plays (was 133k). off_pass AUC improved to 0.841 and three teams now clear the league mean after EB shrinkage: ATL, CAR, and TB. FTN features gracefully absent for pre-2022 seasons — HGBC handles NaN natively.
More training data
◑ PARTIAL
02
BDB spatial features at snap
Extracted defender depth, lateral spread, and deep-safety count from frame 1 of the BDB 2023 tracking data (14k plays). Features stored in plays.parquet. Not yet integrated into models — covariate shift prevents use in 2024/25 evals. Unlocks fully once BDB releases 2024/25 tracking.
Richer per-play signal
✓ DONE
03
Coverage classifier
Supervised man/zone classifier from tracking features: AUC 0.752, accuracy 69.5% (baseline 56.8%). Key discovery: distance-stability-to-primary-target is the dominant signal — not the identity-churn hypothesis. Model persisted as coverage_clf.pkl for future inference.
Tracking-derived labels
Bottom line: The model is working — AUC 0.841 on run-pass prediction is strong. With 10 seasons of data, three teams (ATL, CAR, TB) now have CIs that clearly clear the league mean. Next steps: integrate 2024 BDB tracking when available, apply coverage classifier to improve is_man labels for older seasons, and convert tell_bits to EPA.