Brier score

The Brier Score grades a forecaster's ability to predict outcomes. Like a weather man's ability to predict rain the next day.

The score is computed as $S = \frac{1}{N}\sum_{i}^{N}(f_i - x_i)^2$, where $f$ is the predicted odds of something happening, (70% chance to rain would be 0.7,) and $x$ is the outcome (1 if correct, 0 otherwise).

The Brier score requires that predictions meet three criteria.

• The prediction must refer to an event that either will or will not happen. It will rain more than one inch.
• The prediction must come with a probability from 0% to 100%.
• The prediction must have a time frame. The company will go bankrupt by December. If the predicted outcome happens outside of the time frame the prediction failed.

https://commoncog.com/blog/how-do-you-evaluate-your-own-predictions/ https://en.wikipedia.org/wiki/Brier_score