Model scorecard

How grading works

• Frozen pre-fight. Predictions are written by an automated job while an event is upcoming and stop updating once it starts. Only records generated before the stored event time are graded — nothing is back-filled.
• Eligibility is explicit. Every valid frozen prediction with a recorded winner counts: favorites, underdogs and prelims. Missing results, invalid mappings and late records stay out of the score and remain visible in the exclusion audit.
• Accuracy is the share of fights where the fighter the model gave ≥50% actually won.
• Brier score and log-loss grade the probability, not just the pick. Lower is better; confident misses are penalized more. A 50/50 forecast has Brier 0.250 and log-loss 0.693 on any set of binary results.
• Live calibration groups the model’s chosen fighter by confidence, then compares average confidence with how often those fighters actually won.
• Simple baselines. The same graded fights are scored using Elo expected outcomes. The fixed 50/50 line is theoretical and does not claim an accuracy because it does not choose a side.
• Model vs members. Where enough members picked a fight before it started, their majority pick is graded on the same fights as the model and the Elo baseline. Fights with too few picks or an even split are left out of that comparison only.
• Current model: v0.8-2026-10-04, trained on historical fights with point-in-time features only — no data from after each fight leaks in.