Although we were warned that Liddell’s closure might put up prices in the NEM, I don’t think we expected that this would happen via a series of price spikes in the small hours of Monday May 1st, less than three days after the last Liddell unit tripped offline for the final time. Paul has published a quite detailed analysis of the spikes, tracing them to behaviour of a particular NEM constraint equation. I’ll refer to his post but not repeat its content here.
It turns out that these spikes ultimately flowed from a series of – to be diplomatic – quirks in AEMO’s market systems.
A handover problem
At the simplest level, what happened seems to be a “handover” problem between different parts of the systems and software used by AEMO to operate the spot market:
- Some of the system components that hold and prepare data for input to the market solution process were “aware” of Liddell’s retirement, and ceased including inputs about the three Liddell units after midnight on April 30th.
- Other components, specifically constraint specifications fed to the core market solver software, instructed the solver to refer to and expect to find data from the Liddell units amongst the inputs flowing from these upstream systems.
- In the absence of this data, the market solution fell back, by design, to using “default values” for the missing Liddell inputs and produced a much more security-conservative but also higher-priced dispatch outcome than necessary.
Some more detail
Even a simplified description of the fuller chain of events here may seem pretty opaque to many readers, it but runs like this:
| 1 | Liddell’s last three units physically ceased production on April 24th (Unit 4), April 26th (Unit 2), and April 28th (Unit 1). |
| 2 | AEMO’s Market Management System (MMS) – the very large operational database which stores data on virtually all aspects of electricity market configuration and dispatch inputs and outputs – includes a unit registration table storing “End Dates” for these three units, set to May 1st 2023 00:00. This represents the last time at which these units would be treated as operational, dispatchable generators in the NEM. |
| 3 | Because of this configuration, from 12:01 am on May 1st, some key inputs to the market solution process ceased including entries for any of the Liddell units. These inputs describe the status of each NEM “dispatchable unit” including its current bids and dispatch (output) levels. |
| 4 | These inputs are drawn (in part) from data held in the MMS and then fed every five minutes into the NEM Dispatch Engine (NEMDE), the optimisation solver software that determines dispatch and pricing results for the NEM. |
| 5 | Although these Liddell inputs disappeared from the “dispatchable unit” section of NEMDE’s input stream, there were other important parts of that stream, in particular the coded representations of some constraint equations drawn from a separate part of the MMS, which still referenced Liddell units and instructed NEMDE to include “initial values” for those units in constraint calculations. Amongst these was the N>>NIL_33_34 constraint described in detail in Paul’s article. |
| 6 | The required initial values include the units’ physical generation levels, or “Initial MW”, at the start of the dispatch interval being solved by NEMDE. Knowing the initial physical state of the power system – generator outputs, transmission line flows and ratings, and the status and capabilities of transmission and generation plant is critically important for maintaining secure system dispatch, which is the key objective of NEMDE’s constraint equations. |
| 7 | Removal of Liddell from the “dispatchable units” data fed to NEMDE meant that from the first dispatch interval of 1 May, and each subsequent interval, NEMDE could no longer find an input specifying the initial generation level at each Liddell unit in processing the relevant constraints. |
| 8 | By design, for any required data it does not find in the relevant section of its input file, NEMDE falls back to using “default values” – also stored in the MMS as part of every constraint specification. |
| 9 | For the N>>NIL_33_34 constraint equation, these default values were set to 500 MW for each Liddell unit. This was not the only constraint where NEMDE fell back to using default values for Liddell generation (but note that values different from 500 MW may have applied in other constraints). However this constraint happened to be the one where this had the greatest impact on the market – again as outlined in Paul’s article. |
| 10 | Use of Liddell’s 500 MW default values in this constraint was inherently conservative – it greatly reduced the “headroom” apparently available under that constraint and required NEMDE to solve and dispatch the market as though the relevant transmission capacity was much tighter than it really was – hence the higher spot prices that resulted. |
| 11 | This is what you’d hope a fallback or default option would do: a fallback position that implicitly assumed more headroom than physically available could compromise system security and lead to far worse outcomes than a few high prices. |
Even gorier detail
For the few readers who might want to track in more detail through this, I’ll finish with some screenshots of MMS and NEMDE inputs with limited commentary illustrating some of the key points above. This will presume knowledge of MMS tables and constraint specifications, as well as NEMDE XML input files.
Liddell units registration end date (2)
Here are the MMS registration table entries for Liddell units – note the END_DATE of 2023-05-01 00:00 for the remaining three units:
Liddell units disappear from NEMDE input file’s “Trader” (dispatchable units) section (3-4)
The left-hand screenshot is from the NEMDE input file for the last Dispatch Interval on April 30th. The right-hand shot is from the input file for the first DI on May 1st: Liddell (LD) units have dropped out:
N>>NIL_33_34 constraint specification (as well as others) in the MMS continues to refer to Liddell data (5, 9)
The version active early on 1 May specifies 500 MW default values to be used on the constraint RHS if Liddell inputs are missing. Note that AEMO’s initial fix from about midday on 1 May was to reset these default values to zero, matching actual Liddell output. A later update completely removed references to Liddell from this constraint.
Excerpt from AEMO’s NEMDE documentation on Constraint Right Hand Side processing specifying how missing “Trader terms” required for constraint calculation get handled (8)
Internally, NEMDE refers to dispatchable units as “Traders” and corresponding values fed to the RHS of constraints as “Trader terms”. Because Liddell units had disappeared from the Traders section of its input file, NEMDE fell back to using the Default Values stored in the MMS constraint specification tables.
The net result
Finally we can see how this played out for the N>>NIL_34_35 constraint highlighted in Paul’s article from this screenshot of ez2view’s Constraint Equation Details viewer, showing the sharp drop in the constraint’s Right Hand Side (RHS) and forcing the constraint to bind, with its Left Hand Side value having to remain lower than the suddenly reduced RHS value:
=================================================================================================
About our Guest Author
 |
Allan O’Neil has worked in Australia’s wholesale energy markets since their creation in the mid-1990’s, in trading, risk management, forecasting and analytical roles with major NEM electricity and gas retail and generation companies.
He is now an independent energy markets consultant, working with clients on projects across a spectrum of wholesale, retail, electricity and gas issues. You can view Allan’s LinkedIn profile here. Allan will be occasionally reviewing market events here on WattClarity Allan has also begun providing an on-site educational service covering how spot prices are set in the NEM, and other important aspects of the physical electricity market – further details here. |
Great find! I Hope this gets fixed soon and in the future the constraints will get updated automatically to remove unused terms.