Methodology — how every number on Quant Data was produced
Methodology
This page exists so you can argue with us. Every figure quoted anywhere on this site is produced<br>the way it is described here, and where a test failed, the failure is written down in the same<br>detail as the successes.
Three rules everything follows<br>Held-out years. A model is never scored on data it was fitted to. We split by calendar<br>year, not by random rows — shuffling intraday bars leaks tomorrow into today and inflates<br>everything.
Pre-registration. For the claim and event tests, the rule, the entry, the barriers and<br>the success definition are all written down before the outcome is computed. This is what stops<br>you from quietly trying eleven variants and reporting the one that worked.
Publish the failures. A win rate is only informative if you know how many things were<br>tested. The failures are on this page in the same tables as the confirmations.
The day-type model
A day type is a coarse description of how a session behaved: did it trend one way all day,<br>chop sideways, grind upward inside a channel, or reverse. Experienced chart readers name these on<br>sight. The model does the same thing, in probabilities.
Data. S&P 500 E-mini futures (ES), minute bars from 2010 to 2026, resampled to<br>5-minute bars, sliced into US regular-hours sessions. 3,946 sessions in total.
Labels. Five classes — TrendUp, TrendDown, Range, TrendingRange, Reversal — assigned<br>from the shape of the completed session. The majority class covers 37% of days, which is the<br>baseline any honest accuracy number has to beat.
Features. 18 per-bar values describing where price sat within the session so far,<br>how far it had travelled, and how the current bar related to the previous session's close.<br>No indicators, no news, no order-flow data.
Model. A small sequence classifier — 3 layers, width 96, dropout 0.15 — that reads the<br>session's bars in order and pools them. It is deliberately small: of the 3,946 sessions, only<br>761 carry a day-type label (491 train, 111 validation, 159 test; a pool of roughly<br>3,000 sessions is used unlabelled for pretraining), and with 761 labelled examples a bigger<br>model would memorise rather than generalise.
Calibration. Raw classifier scores are overconfident. We fit a single temperature<br>parameter on validation data so the stated probability matches observed frequency. Measured<br>calibration error is under 10 percentage points, meaning when the API says 70% the outcome<br>happens roughly 70% of the time.
Results, on years the model never saw<br>Full session, top-1<br>66%<br>vs a 37% majority baseline
Full session, top-2<br>80%<br>truth is in the model's top two classes
At 90 minutes in<br>53.5%<br>18 five-minute bars; top-2 68.6%
Calibration error<br>stated probability vs observed frequency
The 90-minute number is the one that matters for live use, and it is the smaller one. Two hours<br>into a session the market has not decided yet, and a model that claimed otherwise would be lying.
Single stocks are out of distribution. The model was fitted to index futures. When you<br>call /v1/brooks/NVDA it will answer, because comparing your own read against it is a<br>useful exercise, but that answer has not been validated on single-stock behaviour and carries<br>no accuracy claim. Same for Hong Kong listings, A-shares and non-index futures: descriptive<br>only.
The price-action claims fact-check
Trading courses are full of confident probabilities. We took the ones that are specific enough to<br>be testable, wrote each as an explicit rule before measuring anything, and ran it over<br>3,946 ES trading days of 5-minute data, 2010–2026.
ClaimTraditionally quotedMeasured One of the day's two extremes is already set, 5 minutes in (1 bar) 20%26.3% One of the day's two extremes is already set, 35 minutes in (7 bars) 50%58.3% One of the day's two extremes is already set, 90 minutes in (18 bars) 90%81.2% On a bull day, the low of the day forms early in the session 90%83.3% On a bear day, the high of the day forms early in the session 90%80.4% After an opening-range (18-bar) breakout, the opposite extreme holds 90%83.5% Trading-range breakouts fail (the famous "80% rule") 80%52.0% / 62.4% / 69.7%
The last row is measured within 5, 10 and 20 bars respectively — the claim's truth depends<br>entirely on how long you are willing to wait, which the claim never says. Every row covers the<br>same 3,946 days except two: the opening-range breakout can only be scored on the 3,578 days<br>that produced a breakout at all, and the “80% rule” row counts range-breakout<br>events rather than days — roughly 5,000 to 9,700 events depending on the bar window.
Meta-verdict: the direction is right every time. Where the teaching quotes a big round<br>number — 90%, 80% — the measured rate comes in below it by roughly 7 to 10 percentage points,<br>and by considerably more on the "80% rule". The two earliest checkpoints run the other way: an<br>extreme is pinned slightly sooner than the courses say. A workable calibration for the rest...