Abstract

There is growing interest and activity in producing data-driven weather and climate forecasts – systems which have learnt on data from either observation of process based models, and are then able to produce a forecast using machine learning and statistical methods. These methods show promise, but to date there is limited investigation into precisely what these models are learning, and how they are making predictions. To improve confidence in these new methods, we need to ‘open the black box’ of data-driven methods and begin to understand which processes are being captured in these models, and with this any limitations they have.

In this work, we focus on a very simple regression model for ocean temperature. We assess the sensitivity of the developed regressor to its inputs using two methods. Firstly, we directly analyse the coefficients to give insight into how this particular model relies on the different input variables. Secondly, we perform a series of withholding experiments – retraining a set of new regressors, each with a single input variable withheld at the training stage. These experiments give us more general insight into what is needed for a regression model to be able to make skilful predictions. We then further analyse some of these experiments to see more precisely how model skill is impacted by certain input variables.

Our results show that for this simple regression model, the behaviour is much in line with the physical understanding we have of the system, indicating that the model is learning, to some extent, the physics of the system. This gives increased confidence in our ability to one day use data-driven models alongside traditional process based models.