Welcome to WeatherGenerator-FastEvaluation’s documentation!

WeatherGenerator-FastEvaluation is a modular evaluation and visualization package for verifying forecast data and model outputs in the WeatherGenerator project. This package reads the output of the inference stage from the WeatherGenerator and provides statistics or exports to other tools.

Overview

The WeatherGenerator-FastEvaluation tool is designed for rapid diagnostics of WeatherGenerator model outputs, in contrast to the evaluation frameworks used in national meteorological services.

The key motivations behing this package are:

  • Rapid turnaround: Enables fast scoring and plotting directly from WeatherGenerator output, supporting short development cycles without waiting for operational verification pipelines.

  • No format conversion: Works directly on native model outputs (e.g., Zarr). Avoids costly GRIB/netCDF conversion steps required by traditional tools, significantly reducing preprocessing time.

  • Stream-level analysis only: Provides evaluation against existing WeatherGenerator streams.

Because the scope is restricted to developer-oriented diagnostics, the evaluation workflow remains lightweight, and integrated into model development.

Features

The WeatherGenerator-FastEvaluation module has the following features:

  • compute performance metrics and diagnostics for forecast or model outputs

  • produce maps, time-series, and other visualizations for qualitative & quantitative evaluation

  • handle gridded and non gridded data (observations / reanalysis)

  • handles comparisons across different runs using the WeatherGenerator zarr files in both .zarr and .zip format.

  • handles comparisons across runs with different masking strategies (e.g. MTM vs forecasting even if they have different forecasting steps).

  • export the WeatherGenerator output into grib/netCDF files suitable to be used by the project partners.

Input

The WeatherGenerator-FastEvaluation package supports multiple inputs:

  • The WeatherGenerator .zarr files generated by the inference step of the WeatherGenerator chain. It reads both Target and Prediction from local files (WeatherGenReader).

  • CSV files with pre-computed scores for an easy comparison with the other models. The scores should be stored as:

,parameter,level,number,score,step,date,domain_name,value
0,t,925,0,mef,0 days 12:00:00,2022-10-01 00:00:00,n.hem,0.031371469251538386
1,t,925,0,mef,0 days 12:00:00,2022-10-01 12:00:00,n.hem,-0.010387031341104752
2,t,925,0,mef,0 days 12:00:00,2022-10-02 00:00:00,n.hem,0.030255780718550083
3,t,925,0,mef,0 days 12:00:00,2022-10-02 12:00:00,n.hem,-0.028894746338016246

Stability and documentation

The package is functionally stable, and its core evaluation workflows are used in active development. Design notes, detailed examples, and workflow guides remain private and will be added gradually to this documentation as it is consolidated.

Licence

This package is licensed under the Apache‑2.0 License.

Check out the Documentation section for further information, including how to use the config files to generate plots and scores.

Note

This project is under active development.

Examples

Documentation