Measuring water market prices

A range of statistical methods are tested empirically using Australian water trade data collated by the Bureau of Meteorology (BOM). This testing is undertaken for a variety of different water market types / locations, including those with a high frequency of trade—such as the southern Murray-Darling Basin (MDB)—and those with limited trade. Each method is applied to 27 specific water allocation markets and 21 entitlement markets across Australia for the period 2014-15 to 2016-17.

Four main methods are considered including:

• Discrete price methods which produce price estimates for specific time periods (e.g., months) as currently used by most analysts. In particular:
• 2SD – Monthly / quarterly mean price after applying two-standard deviation (2SD) based exclusion of outliers
• Median - Monthly / quarterly median price
• ‘Smoothing’ methods, which provide continuous (e.g. daily) price estimates via a statistical model. In particular:
• LOESS – Robust local quadratic regression using a LOESS framework (Locally Weighted Scatter-plot Smoother)
• GAM – Robust penalised regression spline using a GAM framework (Generalised Additive Model)

The empirical performance of these methods is assessed using both qualitative (visual inspection of charts) and quantitative (i.e., cross-validated error metrics) evidence. Here good performance requires both accuracy – estimates adequately capture short-run variation in water market price levels – and robustness – estimates are not influenced by obvious outliers in the data. A complete set of results is available via an online dashboard.

The GAM smoothing method achieved the best overall performance

Overall, the GAM method achieves the best performance, with lower error metrics than alternatives in most markets, both for allocation and entitlement trade. The GAM method demonstrates superior accuracy – better capturing short run changes in market prices – and superior robustness – being less influenced by outliers.

The benefits of smoothing methods (e.g., GAM) depend on the type of market

Generally, the benefits of smoothing methods (i.e., increases in accuracy and robustness) were greater in entitlement markets than for allocations. For allocation markets, GAM achieves significant improvements (over 2SD and Median) within the sMDB, given the high frequency of trade and clear price signals.  In these markets, smoothing methods provide a more accurate estimate of current market prices, accounting for changes in the most recent weeks and days of trading which can be missed by discrete methods. For allocation markets outside of the sMDB all methods achieve similar levels of performance.

Smoothing methods can struggle during market closures

Smoothing methods (including GAM) can produce unreliable results during brief periods of market inactivity in otherwise high frequency markets, such as when trading is temporally suspended by regulators. For this reason, smoothing methods need to be combined with a filter rule, to ensure that no prices are reported on days with no or limited trade activity.

Traditional methods (e.g., 2SD) performed the worst overall

2SD was outperformed by other methods in most allocation and entitlement markets. In some markets especially those with infrequent trade, 2SD can be influenced by low outliers. As a result, 2SD produces lower estimates of market price on average relative to other methods. In testing, median proved to be more robust, consistently outperforming 2SD. Monthly median prices worked best in allocation markets and quarterly prices were preferred in entitlement markets.

At a minimum Median should be adopted in place of 2SD

In testing, Median consistently outperformed 2SD. Median is also easy for analysts to implement and easily understandable by stakeholders. Monthly median prices for allocation markets and quarterly median prices for entitlement markets provided a reasonable starting point when analysing water register data.

Effort could be put into further developing smoothing methods like GAM

While the GAM method achieved superior performance, it remains harder to implement and harder to explain to stakeholders than simple statistics like median. For these reasons, many analysts may be reluctant to adopt GAM. Further effort may be required to address these concerns. In particular, common software could be developed to fully implement the GAM method, including an appropriate filter rule. This software could then be used by all analysts with minimal effort and minimal risk of implementation error.

Effort could be put into improving water trade data at the source

Beyond the use of statistical methods, consideration should be given to improving the quality of government water register data at the source. There may also be opportunities to make greater use of alternative data sources, including price data recorded by private water exchanges and brokers.