WeatherGenerator logo

The WeatherGenerator-FastEvaluation package

A modular evaluation and visualization package for verifying forecast data and model outputs in the WeatherGenerator project. This package reads the output of the inference stage from the WeatherGenerator and provides statistics or exports to other tools.

The WeatherGenerator-FastEvaluation tool is designed for rapid diagnostics of WeatherGenerator model outputs, in contrast to the evaluation frameworks used in national meteorological services.

The key motivations behing this package are:

  • Rapid turnaround: Enables fast scoring and plotting directly from WeatherGenerator output, supporting short development cycles without waiting for operational verification pipelines.
  • No format conversion: Works directly on native model outputs (e.g., Zarr). Avoids costly GRIB/netCDF conversion steps required by traditional tools, significantly reducing preprocessing time.
  • Stream-level analysis only: Provides evaluation against existing WeatherGenerator streams.

Because the scope is restricted to developer-oriented diagnostics, the evaluation workflow remains lightweight, and integrated into model development.

Features

The WeatherGenerator-FastEvaluation module has the following features:

  • compute performance metrics and diagnostics for forecast or model outputs
  • produce maps, time‑series, and other visualizations for qualitative & quantitative evaluation
  • handle gridded and non gridded data (observations / reanalysis)
  • export the WeatherGenerator output into grib/netCDF files suitable to be used by the project partners.

Input

The WeatherGenerator-FastEvaluation package supports multiple inputs: - The WeatherGenerator .zarr files generated by the inference step of the WeatherGenerator chain. It reads both Target and Prediction from local files (WeatherGenReader). - CSV files with pre-computed scores for an easy comparison with the other models. the scores should be stored as:

,parameter,level,number,score,step,date,domain_name,value
0,t,925,0,mef,0 days 12:00:00,2022-10-01 00:00:00,n.hem,0.031371469251538386
1,t,925,0,mef,0 days 12:00:00,2022-10-01 12:00:00,n.hem,-0.010387031341104752
2,t,925,0,mef,0 days 12:00:00,2022-10-02 00:00:00,n.hem,0.030255780718550083
3,t,925,0,mef,0 days 12:00:00,2022-10-02 12:00:00,n.hem,-0.028894746338016246

The structure is flexible and new readers can be added on demand.

Stability and documentation

The package is functionally stable, and its core evaluation workflows are used in active development. However, documentation is currently internal: design notes, detailed examples, and workflow guides remain private and will be opened gradually as they are consolidated.

Quick Start — Running the Evaluation Workflow

After the inference step you can run evaluation (on CPUs) as:

uv run evaluate --config <path to config file>

The default config file is at: WeatherGenerator/configs/evaluate/eval_config.yml

More instructions can be found here: https://gitlab.jsc.fz-juelich.de/esde/WeatherGenerator-private/-/wikis/home/Common-workflows/fast-evaluation

Licence

This package is licensed under the Apache‑2.0 License.