scoringrules.rps_score

Contents

scoringrules.rps_score#

scoringrules.rps_score(obs: ArrayLike, fct: ArrayLike, k_axis: int = -1, *, onehot: bool = False, backend: Backend = None) → Array#

(Discrete) Ranked Probability Score (RPS)

Suppose the outcome corresponds to one of \(K\) ordered categories. The RPS is defined as

\[\text{RPS}(F, y) = \sum_{k=1}^{K}(\tilde{F}_{k} - \tilde{y}_{k})^2,\]

where \(F \in [0, 1]^{K}\) is a vector of length \(K\), containing forecast probabilities that each of the \(K\) categories will occur, with \(\sum_{k=1}^{K} F_{k} = 1\) and \(\tilde{F}_{k} = \sum_{i=1}^{k} F_{i}\) for all \(k = 1, \dots, K\), and where \(y \in \{1, \dots, K\}\) is the category that occurs, with \(\tilde{y}_{k} = 1\{y \le i\}\) for all \(k = 1, \dots, K\) [1].

The outcome can alternatively be interpreted as a vector \(y \in \{0, 1\}^K\) of length \(K\), with the \(k\)-th element equal to one if the \(k\)-th category occurs, and zero otherwise. Using this one-hot encoding, the RPS is defined analogously to as above, but with \(\tilde{y}_{k} = \sum_{i=1}^{k} y_{i}\).

Parameters:
obsarray_like

Category that occurs. Or array of 0’s and 1’s corresponding to unobserved and observed categories if onehot=True.

fctarray

Array of forecast probabilities for each category.

k_axis: int

The axis of obs and fct corresponding to the categories. Default is the last axis.

onehot: bool

Boolean indicating whether the observation is the category that occurs or a onehot encoded vector of 0’s and 1’s. Default is False.

backendstr

The name of the backend used for computations. Default is ‘numpy’.

Returns:
score: array_like

The computed Ranked Probability Score.

References

[1]

Epstein, E. S. (1969). A scoring system for probability forecasts of ranked categories. Journal of Applied Meteorology, 8, 985-987. https://www.jstor.org/stable/26174707.

Examples

>>> import scoringrules as sr
>>> import numpy as np
>>> fct = np.array([0.1, 0.2, 0.3, 0.4])
>>> obs = 3
>>> sr.rps_score(obs, fct)
0.25999999999999995
>>> obs = np.array([0, 0, 1, 0])
>>> sr.rps_score(obs, fct, onehot=True)
0.25999999999999995
Contents