Potential shaping
39.8% wins and 60.1% lane coverage, versus 35.7% and 56.3% for the outcome-only control. Rewards teach progress toward a useful position.
New: guard refinement · three variants, five seeds & best checkpoints →
New experiment: curriculum vs shaping · 10M fresh reps each →
BEHAVIOR LAB / 02 — CONTROLLED EXPERIMENT
TEACH THE GUARD.
Four methods. Same body. Same movement options.
Which training actually improves protection?
1.8MFRESH TRAINING REPS
100K × 3REPS × SEEDS PER METHOD
4 + 2METHODS + CONTROLS
FIXEDRATINGS & PHYSICS
Better training helps.
The movement controller still gets in the way.
Potential shaping and curriculum lead on average. But every method’s guard wins more often with its learned opening tactic and authored steering than with its learned movement adjustments. Treat these as promising directions, not a finished blocker.
Three training seeds show considerable variation. Shaping’s 0.2-point lead over curriculum is too small to call a reliable winner.
01 / HELD-OUT RESULTS
Protection, measured.
Each frozen policy faces the same 5,000 fresh reps: five equally sampled DT opening tactics with authored steering. Values are means across three training runs; the small ranges show the lowest and highest run, not confidence intervals.
| Training method | G win rate ↑ | Lane coverage ↑ | Lane error ↓ | Wrong-way motion ↓ |
|---|
Coverage = time with G between DT and QB and within 0.8 yd of their connecting line. Lane error = distance to that finite line segment. Wrong-way motion = free-movement time traveling laterally away from the intercept line while misaligned. These are geometric proxies, not a validation of NFL technique.
Imitation looks calmer, loses more
Wrong-way movement falls to 16.0%, but wins fall to 27.2%. A smoother path does not guarantee a protected pocket.
02 / MATCHED REPLAYS
Watch the same rep twice.
Left is the outcome-only control. Right is your selected method. Both use the same setup and random seed; their decisions can create different contact events. Opening tactics are learned independently.
Outcome-only control
0.00sLoading policy…
Potential shaping
0.00sLoading policy…
GUARDDTQBDashed line: direct rush laneTrails: player movement
Loading matched policy pair…
Study policies use a method-specific format. Keep these in this study viewer; the main lab’s importer does not support the imitation classifier. Your existing million-rep players remain in the main lab.
03 / STRESS TESTS & ABLATION
Does it hold up elsewhere?
Guard win rates across separate tests. “Authored steering” disables learned movement while retaining that same policy’s learned opening tactic. The trained DT is the previous million-rep model, frozen throughout evaluation. Recovery starts G 1.7 yd left or right of its usual position.
| Training method | Full policy | Authored steering | Trained DT | Recovery starts |
|---|
5,000 reps per policy for full vs authored; 2,000 against the trained DT; 1,000 recovery reps. All evaluations are held out from training. Shared test scenarios are paired comparisons, not independent evidence from 15,000 different setups.
04 / LEARNING OVER TIME
Progress is uneven.
Checkpoint tests use a separate, fixed 1,000-rep development set. Lines average three training seeds; shaded bands show their range. Final rankings above use a different held-out set.
05 / WHAT WE CHANGED
One training method at a time.
Coached imitation
A scripted coach labels the best available movement on the learner’s visited states. The policy aggregates those labels. Coach-controlled actions fade from 80% to zero by 60,000 reps; tests use the learner alone.
DAgger-style imitation. The teacher sees current velocities; the learner keeps the existing coarse observations. That information gap can limit imitation.
Ross et al. / DAgger ↗Weighted curriculum
First 20%: head-on bull rush. Next 30%: bull and arc rushes. Final 50%: half ordinary reps, 30% lateral / feint rushes, 20% recovery starts. All final tests use the same scenarios as the other methods.
Bengio et al. / Curriculum learning ↗Potential shaping
Add 0.25 × [0.94 Φ(next) − Φ(current)] to each movement transition’s reward. Φ combines lane alignment (50%), pocket cushion (30%), and movement control (20%). Terminal Φ is zero; winning and losing still give +1 / −1.
Weights are experimental design choices. Coarse observations mean the paper’s ideal guarantees are not automatic in this prototype.
Ng et al. / Reward shaping ↗Prioritized replay
Retain 8,192 movement transitions. After each rep, reuse 16 memories, favoring large prediction errors. Stochastic sampling and importance weights reduce sampling bias. A uniform replay control gets the same extra update budget.
Equal fresh reps do not mean equal compute. Replay performs 1.6M additional updates per run.
Schaul et al. / Prioritized replay ↗Shared controls, limitations & reproducibility
All 18 runs start fresh. Only G trains; DT receives zero updates. Guard ratings: strength 89, speed 72, technique 70, weight 315 lb. DT: strength 74, speed 87, technique 70, weight 280 lb. Physics, action set, state bins, and decision cadence are identical. All methods retain the same outcome-only opening-tactic learner.
The learner still lacks velocity observations and chooses relatively large positional offsets every 0.3s. These experiments test the training method within that constraint; they do not establish that the controller can learn realistic footwork. During engagement, contact motion remains authored.
No methods are combined, and no hyperparameters were tuned after the held-out results. Three seeds are a screening study, not definitive statistical evidence. Initial recovery evaluation had a one-sided offset bug; the evaluation was corrected and rerun for all frozen policies. Training and the other tests were unaffected.
Recovery duration is conditional on regaining coverage and can look deceptively fast if failed recoveries are excluded. The downloadable summary includes unrecovered rates alongside that metric.
Download experiment manifest ↓ · Download training methods source ↓
NEXT / PLAYER DEVELOPMENT
Pretrain in practice.
Keep learning after games.
Yes. This is continued reinforcement learning with a persistent policy for each player. The full-game training layer is a next step; it is not implemented by this study.
- Play with a stable policy. Record observations, decisions and per-play role outcomes during the game.
- Review after the game. Mix recent game experience with retained practice and older game memories. Use small, bounded updates so one match does not overwrite the player’s training.
- Test before promotion. Compare the candidate with its previous version on a fixed opponent suite, then keep or roll back the update.
- Save the player. Persist its policy version and memory under a player ID. Keep physical ratings separate from tactical learning.
Use play-level protection outcomes for G and pressure outcomes for DT; team wins alone give weak credit to individual decisions. Pure imitation needs ongoing coach labels or a compatible RL fine-tuning step. It cannot learn from game rewards by itself.