The mean difference between the model and observations, measures the tendency of the model process to over- or under-estimate the value of a parameter. Smaller absolute bias values indicate better agreement between measured and calculated values. Positive bias means overprediction, negative means underprediction.
diff = model_data - observations bias = diff.mean()
Also called the root-mean-squared deviation, it's a measure of the differences between the observed and predicted values. Smaller RMSE values indicate better agreement between measured and calculated values.
Defined as the standard deviation of the difference between model and observations, normalised by the mean of the observations. Smaller values of SI indicate better agreement between the model and observations.
scatter_index=100.0*(((diff**2).mean())**0.5 - bias**2)/observations.mean()
Rank histograms (sometimes called verification rank histograms or Talagrand diagrams) are a way to show how reliable an ensemble forecast is compared to a set of newly observed data. In other words, they show the bias for the model. If an ensemble forecast is accurate, the rank histogram — a graph of observed data — will be flat. Deviations from a uniform distribution (i.e. histogram blocks that are above or below the red line) mean that the model isn’t completely accurate. These types of diagrams are not commonly used outside of ensemble forecasting. (2016 Statistics How To)
Calculated as the difference between the observations continuous distribution function (CDF) and the forecasts CDF. Defaults to the Mean Absolute Error in the case of a deterministic forecast.
Brier Score for an ensemble for exceeding the specified threshold. It calculates the mean squared error between predicted probabilities and the expected values. The score summarizes the magnitude of the error in the probability forecasts. The score falls between 0.0 and 1.0, with perfect skill having a score of 0.0.