Track record

Every published brief’s key judgments are converted into falsifiable, dated propositions. After each deadline an automated resolver searches open sources and scores the call — including the misses. The probability a judgment implies is taken from its own ICD-203 estimative wording (“likely” ≈ 65%), so this page answers one question: when CrisisBrief says likely, how often does it happen?

It keeps two records apart. Forecasts sealed since 2026-10-05 carry their whole contract: what counts as yes, which sources decide it, and whether their silence means no, all fixed before the outcome was known. Earlier forecasts were sealed with the question alone; they are the legacy record below, and they are not mixed into these scores.

Sealed contractsince 2026-10-05

2 decided forecasts in 2 event families. A question asked again in later briefs is one family, not several results, so the cold start counts families and the ranges below resample whole families. Mean Brier 95% range 0.360–0.490; directional accuracy 0–0%.

Audit

A sample of the resolver’s decisions is checked every week by AI auditors from model families the resolver does not use. They see the sealed contract and nothing the resolver did, and run their own searches. An auditor is a model too, so its agreement is not the truth: it is graded on forecasts a prediction market settled, shown beside it. Where two auditors from different families reach the same answer on reports they cite against the resolver’s, the label is corrected, but only once the first auditor has scored 90% on at least 10 settlements, 3 or more of each outcome. The corrected label and the one it replaced are both kept in the ledger.

What this does not measure: agreement is one figure across the kinds below, each weighted by how sparsely it was sampled, so a kind with few audits is shown only as a count. An item the auditor could not check after 3 attempts is abandoned: out of the agreement figure, and a miss in the auditor’s own grade. That grade comes from questions a market settled; on questions no market covers, the auditor’s accuracy is assumed from it, not measured.

Legacy record2026-06-11 – 2026-10-05

These forecasts were sealed with the question only. The resolver of the time could find that something happened but had no way to decide that it had not, so its labels lean to yes; the only nos of that time came from prediction markets settling. They are left out of the calibration scores above. The ones still pending will be resolved with criteria written now, after the fact, which is weaker evidence than criteria fixed in advance; how many have them is shown below.

Methodology: propositions are machine-extracted at publication; judgments that cannot be externally verified are excluded rather than forced, and a question that fails the validity checks (no deadline, two questions in one, a placeholder) is not sealed. After a forecast’s window closes, the resolver searches for reports dated inside it on days 1, 7, 30 and 60. A yes or no needs a report it retrieved and cites, and a second model must reach the same answer on its own. Silence does not yet count as no. A failed search is retried, never read as an answer. Anything still undecided on day 60 is voided, with its reason shown above. Every attempt is kept. Raw data: /api/track-record. Briefings: /briefings.

The CrisisBrief Indexrun 2026-10-12 · method v1

The CrisisBrief Index (CBX) is the mean of four sub-scores the analysis model gives each day, computed by code under a published method. This asks whether it carries information: does a 7-day move in the index predict what happens over the next 30 days? The question, the outcomes and the rule for the verdict were fixed before the data could answer them, and the verdict is published whatever it says.

The primary outcome, the only one with a verdict, is the worldwide count of armed-conflict events in GDELT over the next 30 days against the 30 before. The index is written from the same news GDELT counts, so an index that only echoed the week’s news would look predictive. The test that matters is the last column: whether the index predicts the outcome beyond the outcome’s own last week.

What this does not measure: moves, not the level of the index, and over 30 days only. A day without two published readings 7 days apart gives no point, and a point is used only once every day it reads is in. GDELT days are never revised, so a GDELT point is the same in every run; a market point can move if a price is revised, and market outcomes are shown, never judged. ρ is Spearman’s rank correlation; momentum is the outcome’s own last 7 days. Points a day apart share most of their days, so they count as fewer independent points: the intervals are on an effective n, discounted by both series’ autocorrelation. They are 99% intervals, not 95%, because the verdict is checked again every week: on simulated series with no relation, checking a 95% interval weekly for a year gave a verdict at some point about one time in six. They appear once there are 6 non-overlapping 30-day windows, and the verdict with them. Every point: /api/cbx/loop.

For information and analysis only. CrisisBrief forecasts geopolitical events, not financial instruments. Nothing here is investment, legal, or trading advice, or a recommendation or solicitation to buy or sell any security. Calibration reflects past performance on event forecasts and does not predict future results.