cyeva: A Universal Python Toolkit for Weather Forecast Accuracy Evaluation

cyeva is an open‑source Python library for fast, automated evaluation of deterministic weather forecast accuracy.

Installation

pip

$ pip install cyeva

Note: Specify a version for production, e.g. $ pip install cyeva==0.1.0b0.

conda

$ conda install -c conda-forge cyeva -y

Usage

Temperature

Calculates RMSE, MAE, RSS, chi‑square and accuracy ratios for continuous temperature data.

import numpy as np
from cyeva import TemperatureComparison

np.random.seed(0)
obs = np.sin(np.arange(100)) * 20 + np.random.random(100) * 5 * np.random.choice([1, -1])
fcst = obs + np.random.random(100) * 5 * np.random.choice([1, -1])

temp = TemperatureComparison(obs, fcst, unit='degC')
print('accuracy ratio within 1 degC:', temp.calc_diff_accuracy_ratio(limit=1))
print('accuracy ratio within 2 degC:', temp.calc_diff_accuracy_ratio(limit=2))
print('rss:', temp.calc_rss())
print('rmse:', temp.calc_rmse())
print('mae:', temp.calc_mae())
print('chi square:', temp.calc_chi_square())

Precipitation

Handles non‑continuous precipitation with metrics such as TS, ETS, bias, accuracy ratios, false‑alarm and miss ratios.

import numpy as np
from cyeva import PrecipitationComparison

np.random.seed(0)
obs = np.random.random(100) * 50
fcst = np.random.random(100) * 50

precip = PrecipitationComparison(obs, fcst, unit='mm')
print('rss:', precip.calc_rss())
print('rmse:', precip.calc_rmse())
print('mae:', precip.calc_mae())
print('chi square:', precip.calc_chi_square())
print('accuracy ratio:', precip.calc_accuracy_ratio())
print('binary accuracy ratio:', precip.calc_binary_accuracy_ratio())
print('false alarm ratio:', precip.calc_false_alarm_ratio())
print('miss ratio:', precip.calc_miss_ratio())
print('ts:', precip.calc_ts())
print('ets:', precip.calc_ets())
print('bias:', precip.calc_bias_score())

Wind

Evaluates both speed and direction with metrics such as wind‑scale accuracy, direction score, and bias.

import numpy as np
from cyeva import WindComparison

np.random.seed(0)
obs_spd = np.random.random(100) * 10
obs_dir = np.random.random(100) * 360
fct_spd = np.random.random(100) * 10
fct_dir = np.random.random(100) * 360

wind = WindComparison(obs_spd, fct_spd, obs_dir, fct_dir)
print('difference accuracy ratio within 1 m/s:', wind.calc_diff_accuracy_ratio(limit=1))
print('wind speed rmse:', wind.calc_rmse())
print('wind direction rmse:', wind.calc_rmse(kind='direction'))
print('wind direction score:', wind.calc_dir_score())
print('wind scale stronger ratio:', wind.calc_wind_scale_stronger_ratio())

Other Continuous Elements

Generic Comparison object can be used for variables such as pressure or relative humidity.

import numpy as np
from cyeva import Comparison

np.random.seed(0)
obs = np.random.random(100) * 100
fcst = obs + np.random.random(100) * 30 * np.random.choice([1, -1])
fcst[fcst > 100] = 100
fcst[fcst < 0] = 0

comp = Comparison(obs, fcst)
print('accuracy ratio within 10 %:', comp.calc_diff_accuracy_ratio(limit=10))
print('rmse:', comp.calc_rmse())

Utility Functions

identify_wind_scale(5.5)

returns 4 identify_direction(45) returns 1 (NE) for 8‑sector mode get_least_angle_deflection(20, 85) returns 65

Performance Testing

Continuous Benchmark framework measured execution speed of key functions. For TS score calculation:

Benchmark results (operations per second) for varying data sizes:

Data size 1E3 – ~3400 ops/sec – ~0.0003 s per call

Data size 1E6 – ~5 ops/sec – ~0.2 s per call

Data size 1E7 – ~0.5 ops/sec – ~2 s per call

Community Contributions

Project is Apache‑licensed. Source repository: https://github.com/caiyunapp/cyeva. Documentation: https://cyeva.readthedocs.io/zh-cn/latest/index.html.