User:Simon Pedley/sandbox

Forecast change theory seeks to understand how forecasts change. It has applications in helping forecast users make better decisions (cite), and in the pricing of financial derivatives based on the forecasts (cite).

Notes on this article

 * Perhaps I should define well calibrated up-front: definition needs to cover mean zero, and also the change being independent of the error
 * My proof by contradiction is a bit weird. Isn't it just that well-calibrated directly means that change in the mean is zero, or not?

Overview
Decision theory shows that in many situations, users of forecasts need to understand the size of potential forecast changes in order to be able to make good decisions (cite ecl page, and several of the papers from that page, and others (some of those commented out ones)).

This arises, in particular, in the following situations:
 * When the forecast user has the option of waiting until the next forecast before making a decision (cite same 3 papers as on ecl page)
 * When individual forecasts are expensive, and the forecast user has to decide which forecasts to purchase (cite me, that w&f paper)
 * When individual forecasts are complex to process and convert to directly useful information, and the forecast user has to decide which forecasts to process (ditto)

Motivated by this, forecast change theory seeks to understand and predict forecast changes (cite)(cite)

The Impossibility of Predicting Changes in the Mean of Well-Calibrated Forecasts
Consider a series of well-calibrated forecasts for a fixed point of time in the future. As we approach the event and the forecasts change, the mean of the successive forecasts should not be predictable in advance, i.e., the changes in the mean should have expectation of zero. This can be demonstrated using Proof by contradiction as follows.(cite) If the changes in the mean were predictable, then the method being used to make the prediction could be used to improve the calibration of the forecast, which implies the forecast was not well-calibrated in the first place.

In mathematical terms, define the mean of the current forecast as $$f_0$$, and the mean of the next forecast as $$f_1$$. We assume that $$f_1$$ is well-calibrated, and predictable in the sense that $$E(f_1-f_0)) = g \ne 0$$. Based on $$f_1$$ and $$g$$ we can create a new calibrated forecast with mean $$f_1'=f_1-g$$. This new forecast is unpredictable, since $$E(f_1'-f_0)=E(f_1-g-f_0)=g-g=0$$. This shows that $$f_1$$ was not well-calibrated in the first place, since we have been able to calibrate it. This is a contradiction which implies that if $$f_1$$ is well-calibrated then $$g=0$$, and $$f_1$$ is unpredictable.

This argument is logically equivalent to the Efficient-market hypothesis in financial economics.

As a result, changes in the mean of well calibrated forecasts $$\delta f$$ can be modelled mathematically using a random walk in which the expectation of the size of the changes is zero, known as a Martingale.(cite)(cite)

where $$\sigma$$ is the standard deviation of the random walk, and $$W$$ is a random number.

However, there is no reason why the variance of the changes in forecast $$\sigma_{\delta f}$$ should not be predictable.

The Relationship Between Forecast Changes and Uncertainty
Given equation ($$), and the assumption that the forecast is well-calibrated, it can be shown (cite, recent and old) that the size of changes in forecasts leading up to an event are related to the forecast uncertainty by the following expression:

where $$\delta f$$ is the forecast change moving from an earlier time $$t_2$$ to a later time $$t_1$$, and where $$\sigma_{t_i}$$ is the forecast uncertainty at time where $$i$$.

From this equation, we see that the size of forecast uncertainty can be calculated from the size of forecast changes, but the forecast changes cannot be calculate from the forecast uncertainty without additional information.

Forecasts with Normal Errors
For forecasts with normally distributed errors, the forecast change is also normally distributed.

Estimating the variance of forecast change is then sufficient to determine the distribution of possible forecast changes.

There are then three ways to estimate the variance


 * If the distribution of forecast errors is not state-dependent, the variance of changes can be estimated directly from past errors


 * Alternatively If the distribution of forecast errors is not state-dependent, the variance of the forecast changes can be derived using the correlation between ensemble spread and forecast change in ensemble forecasts (cite).

Forecast with Non-Normal Errors
For many forecasts, the errors may not be normally distributed. The distribution of errors, and the distribution of changes, must then be modelled using historical forecasts (cite various).

References -Jewson and Ziehmann -2 papers with Gabriele -Papers I’ve cited in the papers with Gabriele, including that hurricane paper

Decision Theory

 * cite several of the met papers, including hurricanes

Pricing Financial Options

 * cite BS wiki page
 * cite JBZ textbook