Data sets associated with "Non-Gaussian Detection using Machine Learning with Data Assimilation Applications"

Manuscript abstract:
In most data assimilation and numerical weather prediction systems, the Gaussian assumption is prevalent for the behaviour of the random variables/errors that are involved. At the Cooperative Institute for Research in the Atmosphere (CIRA) theory has been developed for different forms of variational data assimilation schemes that enables the Gaussian assumption to be relaxed. For certain variable types, a log normally distributed random variable can be combined in a mixed Gaussian-lognormal distribution to better capture the interactions of the errors of different distributions. However, assuming that a distribution can change in time, then developing techniques to know when to switch between different versions of the data assimilation schemes becomes very important. Given this ability to change the formulation of the data assimilation system enable us to select the more optimal scheme for the different distributed situations.

In this paper, we present results with a machine learning technique (the support vector machine) to switch between data assimilation methods based on the detection of a change in the Lorenz 1963 model's z component's probability distribution. Given the machine learning technique's detection/prediction of a change in the distribution, then either a Gaussian or a mixed Gaussian-lognormal 3DVar based cost function is used to minimise the errors in this period of time. It is shown that switching from a Gaussian 3DVar to a lognormal 3DVar at lognormally-distributed parts of the attractor improves the data assimilation analysis error compared to using one distribution type for the entire attractor.