Bundesliga — Validation

Every played match: our pre-match forecast vs. what actually happened. The receipts. 🧾

18 graded matches

50.0%

Outcome accuracy

9 / 18 results called

50.0% vs Market 55.6%

5.6%

Exact scoreline

1 / 18 spot‑on

0.231

Avg. RPS (lower better)

coin‑flip baseline 0.25

0.231 vs Market 0.223

7.6%

Skill vs. coin‑flip

RPS improvement over uniform

Market = de-vigged bookmaker consensus over 18 of 18 graded matches. Both figures above are scored on that same subset. SFMMO trails the closing market by 3.4% on RPS.

Outcome accuracy — share of matches where our most-likely result (1 / X / 2) was right. RPS (ranked probability score) grades the whole probability forecast — it rewards being confident and right, and is the standard metric for football results (lower is better, 0 = perfect). Skill is how far the model beats a no-information coin-flip forecast. All numbers are the SFMMO's pre-match probabilities, scored against the final result.

Wait — how can 50.0% accuracy still beat a coin-flip on RPS?

Because accuracy and RPS measure different things, on different scales — they're not in competition.

Accuracy only asks: was our single top pick right? It's blind to the probabilities, and a model almost never makes draw its top pick — so a draw-heavy round drags it down even when the forecast was good.
RPS grades the whole probability spread (e.g. 55 / 25 / 20) against a uniform “coin-flip” (33 / 33 / 33). The coin-flip never commits, so it's never punished hard — which makes it a surprisingly tough baseline, and means a few % of skill is a genuine edge.

Our committed forecast cuts both ways (lower RPS = better):

Match	Our forecast	Result	Our RPS	Coin-flip RPS
Favourite delivers	55 / 25 / 20	home win	0.12 ✓	0.28
We back home, it's a draw	55 / 25 / 20	draw	0.17	0.11 ✓

When a favourite wins we crush the baseline; on a draw, the hedging coin-flip actually beats us. Matchday 1 had 9 draws in 24 games, so the wins and losses partly cancel — the net edge is the “skill” figure above. A few % over uniform is roughly what bookmakers manage, so it's a real, healthy edge.

Matchday 1

Match	Forecast (1 / X / 2)	Our call	Predicted	Actual
Bayern Munich v VfB Stuttgart	69.1 / 18.3 / 12.6	Bayern Munich	2–0	5–1	✓
Elversberg v Bayer Leverkusen	20.6 / 22.5 / 56.9	Bayer Leverkusen	1–1	3–2	✗
RB Leipzig v Borussia Mönchengladbach	61.3 / 21.9 / 16.9	RB Leipzig	1–1	3–0	✓
Mainz 05 v Paderborn	63.6 / 21.8 / 14.6	Mainz 05	2–0	0–0	✗
Union Berlin v Eintracht Frankfurt	41.8 / 27.4 / 30.9	Union Berlin	1–1	3–3	✗
1. FC Köln v Hoffenheim	32.3 / 25.4 / 42.3	Hoffenheim	1–1	3–2	✗
Borussia Dortmund v Hamburger SV	66.1 / 20.2 / 13.7	Borussia Dortmund	2–0	2–0	✓ ⚛
Freiburg v Werder Bremen	44.2 / 24.9 / 30.9	Freiburg	1–1	4–1	✓
Augsburg v Schalke 04	49.1 / 27.0 / 24.0	Augsburg	1–1	3–0	✓

Matchday 2

Match	Forecast (1 / X / 2)	Our call	Predicted	Actual
VfB Stuttgart v 1. FC Köln	55.0 / 22.9 / 22.0	VfB Stuttgart	1–1	4–1	✓
Hoffenheim v Borussia Dortmund	32.4 / 23.9 / 43.7	Borussia Dortmund	1–1	2–3	✓
Bayer Leverkusen v Union Berlin	59.1 / 22.1 / 18.8	Bayer Leverkusen	1–1	4–0	✓
Borussia Mönchengladbach v Elversberg	55.8 / 23.0 / 21.2	Borussia Mönchengladbach	1–1	3–4	✗
Werder Bremen v RB Leipzig	22.8 / 22.4 / 54.8	RB Leipzig	1–1	3–1	✗
Paderborn v Freiburg	25.6 / 26.0 / 48.4	Freiburg	1–1	0–1	✓
Schalke 04 v Bayern Munich	9.2 / 16.2 / 74.5	Bayern Munich	0–2	0–0	✗
Hamburger SV v Mainz 05	35.8 / 29.0 / 35.2	Hamburger SV	1–1	0–5	✗
Eintracht Frankfurt v Augsburg	46.7 / 25.3 / 28.0	Eintracht Frankfurt	1–1	1–4	✗

✓ = our most-likely result was correct · ⚛ = exact scoreline nailed · multiclass Brier score 0.612. Click any match for its full forecast breakdown.