There’s been a number of times over preceding months and years when I have thought about writing an article such as this one, but have not done so.
However through the evolution of the 2022 Energy Crisis (which has been centred on Q2, culminating in June … at least thus far), I’ve seen a number of instances in which analysis of what’s happened has (in my view) not considered that other dimension of time – giving added impetus to the task of publishing this article today.
(A) The linear progression of time
It makes sense that we most often think about the evolution of time in a linear scale as illustrated in the image below (with reference to the growth and ageing of a particular person):
As the person ages, they age from the left to the right (except in the case of Benjamin Button, of course).
From cartoon world and back into the NEM, analysis about changes to variables of interest (such as electricity prices) will typically follow the same pattern – such as:
1) The price this year compared to the price last year;
2) The volume of electricity produced from a particular asset …
(a) this month compared to last month
(b) or perhaps, compared to the same month last year
(c) or, with an added level of complexity, this month compared to the same time such-and-such conditions held in the market
… and so on…
It’s very common to see analysis that follows that standard dimension of time.
(B) Reviewing ‘Actual’ against prior ‘Expectation’
But it’s important to remember that other dimension of time can also be important for analysis on occasions – with that other dimension related to how:
1) we compare expectations ‘now’ (with respect to some point in the future) in relation to what actually unfolds.
2) we could also compare what our expectations were back at some point in the past (with respect to some point in time that was in the future at the time) in relation to what actually unfolded.
Stepping back into cartoon-world briefly to illustrate:
1) As a young kid, their expectation might be ‘when I grow up, I want to be a fireman (or woman)’
2) As a teenager, their dreams will have changed (and maybe they are dreaming then of being the next Ronaldo).
3) Early in the working career, they might have expectations that they will have ‘secured the corner office’ (or whatever it is that’s their career expectation at the time).
4) By the time they reach middle age their expectations will probably have been modified by their life experiences to that point (hopefully not to the point that too much of the Dilbert cartoons resonate, mind you).
5) In later life they might be given more to reminiscing about those points in their past that made them happy (or perhaps sad).
Here’s an illustration:
By rotating the standard dimension for the progression of time to a diagonal, we unlock a method of illustrating these other dimensions:
1) future expectations ahead of a given point of time are the horizontal dotted lines; and that
2) ‘looking up a vertical’ becomes a way of reviewing how varying expectations for the future converged (or not) on what actually transpired.
—
Given the underlying complexity of the National Electricity Market there are many of these ‘future expectations’ that are published by the AEMO that take the form of formal forecasts … in P5 predispatch, P30 predispatch, ST PASA and MT PASA. This has been continuing for more than 20 years, with some variations.
We’ve built a (very large) database of these in a modified MMS Schema and store it in the cloud:
1) making it accessible to our ez2view clients to review through the ‘Forecast Convergence’ widget, which uses a similar triangular grid representation as illustrated above.
2) we also have some clients who subscribe to access this ‘MMS Historical’ directly via SQL.
It’s important to note, up front, that other expectations are not published in such a regularly repeatable fashion, and accessible in the MMS … but they can still be quite important in understanding what actually transpired given the inter-related complexities of the NEM.
(C) Examples using ‘Forecast Convergence’
It’s specifically for this purpose (i.e. ‘looking up a vertical’) that we introduced the ‘Forecast Convergence’ widget into ez2view some years ago, and have continued to enhance since that time …
… such as in June 2022, driven by the 2022 Energy Crisis, we introduced the ability to trend through successive LOR level 1, 2 and 3 forecasts using the Forecast Convergence Grid.
Here’s some examples of where it’s been used….
| Subject Matter |
Examples |
|---|---|
Price Outcomes |
Answering the question ‘did price outcomes match expectations?’ is one of the central questions asked by many stakeholders in the National Electricity Market … hence something we have also asked in prior WattClarity articles, including the following: (a) At the bottom of this review of price outcomes for Q2 2021 we included a different style representation of ‘Forecast Convergence’ to highlight how price outcomes in the physical market were significantly above the prior expectations for them in the financial market. (b) The complexities of even answering the question ‘was there any warning of this price spike’ were explored in this article about overnight volatility in South Australia on Thursday morning 8th September 2022. It’s also worth noting the use of a sketch of the ‘Forecast Convergence’ grid to illustrate the potential challenges with Potential Tripwire #1 in relation to ‘The invisible 5-minute Trading Periods’. |
‘Forecast Convergence’ for the Supply-Demand balance expectations |
Price outcomes are primarily the result of supply-demand balance (with some other factors thrown in as well). Understandably, there have been a number of articles written about the evolving forecasts for supply-demand balance on WattClarity, such as: (a) On 27th January 2022 we wrote about ‘Forecast tight supply/demand balance in Queensland on Monday 31st January, Tuesday 1st and Wednesday 2nd February 2022’. (b) In this article looking at 9th March 2022 we used the ez2view widget to explore ‘The difficulty of forecasting the supply-demand balance!’. (c) In this article on 14th June 2022 we also used the ez2view widget to explore ‘11am forecast for LOR3 across the NEM…’. |
Lack of Reserve expectations? |
It was logical to extend ‘Forecast Convergence’ in ez2view to also enable another view of Supply-Demand balance … by the tracking of LOR expectations in AEMO forecasts: (a) We noted the implementation on 20th June 2022 with ‘increased visibility of LOR3 forecasts’. (b) This new functionality was used on 10th August looking forwards to 15th August 2022 in writing ‘load shedding briefly forecast for South Australia’. |
Demand Outcomes |
One part of the supply-demand equation relates to electricity demand. The forecastability of electricity demand is therefore quite central to electricity market operations: 1) It was explored in GenInsights21 more systematically: (a) Such as Linton mentioned when writing about ‘is the predictability of day-ahead demand improving?’ 2) On WattClarity in recent years there have been a number of articles written about changing expectations for electricity consumption – such as: (a) On 12th February 2021 writing about how ‘Victoria’s snap lockdown drops demand forecast by 1,000MW’. (b) On 2nd February 2022 we wrote about ‘demand forecast warming for QLD …’ for that afternoon.
|
Availability of any units? |
Readers should note that this ‘Availability’ measure: (a) might be viewed for a single unit; (b) or aggregated to focus specifically at any sub-set of units; (c) or used to look at all Scheduled and Semi-Scheduled units together … noting the invisibility of other Non-Scheduled units in this respect. … with the following being some examples… Availability of coal-fired units We’ve looked more systematically at this data in a number of ways: 1) It was explored in GenInsights21 more systematically: (a) In Appendix 16 within GenInsights21 we demonstrated how the forecastability of coal unit availability 24 hours in advance is not changing as significantly as you might believe if you listened too closely to other commentary elsewhere. (b) A different representation of ‘Forecast Convergence’ was introduced in the Q2 2022 issue of GenInsights Quarterly Updates, and we will continue updating these for subsequent quarters. 2) Also on WattClarity we’ll feature articles looking at specific points in time: (a) Will link to some later. Availability of Wind and/or Solar units Notwithstanding the unfortunate historical invisibility (or at least opacity) of underlying physical plant availability for Wind and Solar units, the same approach can be used to review energy-constrained Availability. This question might be asked with respect to either (or both): (a) the energy-constrained availability of Wind and Solar (together, or individually); and/or (b) the actual output of Wind and Solar (together, or individually). This particular question is explored in a number of ways: 1) Within Appendix 16 of GenInsights21 we provided more systemic analysis of the degree to which the ‘error’ in forecasting Wind and Solar (energy constrained) Availability is growing larger as installed capacity continues to grow. (a) Some of this analysis was featured in Marcelle’s subsequent article ‘What does the future hold for Wind and Solar in the NEM – seen via GenInsights21?’. Analysis now being updated periodically via GenInsights Quarterly Updates. 2) Through articles on WattClarity we have looked more specifically at different examples: (a) Back in December 2019 Allan used ‘Forecast Convergence’ for Wind and Solar to explore ‘Out of the blue, an LOR2’ (b) In his article on 4th February 2022, Allan used ‘Forecast Convergence’ to look at Wind and Solar Availability as part of his review ‘Expect the unexpected – it’s the NEM after all’. (c) For instance on Thursday 12th May 2022 we posted this article about how the ‘drop in forecast wind conditions … briefly triggers forecasts of load shedding…’. (d) More to come… Also worth noting 1) From GenInsights21: (a) In Appendix 16 within GenInsights21 we highlighted the different nature of forecastability of Availability for different fuel types … as seen in this ‘Mystery DUID’ quiz (results were in the report). (b) Following the 2022 Energy Crisis, we specifically explored the same thing with a different representation of ‘Forecast Convergence’ was introduced in the Q2 2022 issue of GenInsights Quarterly Updates. 2) In specific articles on WattClarity® we also explored: (a) such as in this article on 17th September 2021, we wound the clock backwards to look forwards to 31st August 2021 and the level of ‘Aggregate Scheduled Availability’ across all fully Scheduled units.
|
Output of any units? |
As with Availability, it is possible to complete this analysis with respect to all units, or a subset of units. In particular, it’s worth highlighting the following: 1) In our GenInsights21 report we featured a number of different pieces of analysis relating to ‘Aggregate Scheduled Target’ (‘AggShedTarget’ = the sum of the Dispatch Targets for all fully Scheduled units* operating in the NEM at that time): * these units, in aggregate (i.e. coal, gas, liquid, hydro, battery discharge and negawatts) represent the fully dispatchable units – the ones being called on to make up whatever difference there is between Underlying Consumption on the one side and the supply of VRE on the other hand. Hence it’s critically important to understand how the requirement for these services is changing over time as the NEM moves from a ‘baseload and peaking’ dispatch model to a ‘VRE and flexibility’ dispatch model. (a) In Appendix 15 we provided analysis of how the requirement is changing over time, in terms of both: i. the aggregate level of AggSchedTarget across the NEM; and ii. the increasing demand for fast ramping (up, and down) of AggSchedTarget. (b) In Appendix 16 which we called ‘Forecast Convergence’, we provided analysis of how it’s becoming increasingly challenging to forecast (24 hours in advance) what the requirement will be for AggSchedTarget … which has implications for market design deliberations (such as in relation to the much-maligned Capacity Market question). This analysis in both appendices: (a) Was discussed as part of this presentation to Smart Energy Council audience on 5th April 2022 (see the table underneath to know where to skip to, to listen in to discussions just about those two appendices). (b) Was also one of the insights that resonated with PV Magazine in this article upon the release of GenInsights21. (c) Was also discussed in the June 2022 issue of ‘Smart Energy Magazine’ (see p49/72). 2) We’re also now updating some of this analysis each quarter via GenInsights Quarterly Updates for a select (small) group of clients. 3) More will also be shared via WattClarity articles…
|
Others… |
The functionality will continue to expand in future upgrades to the ez2view software … |
.
So stay tuned to the ‘Forecast Convergence’ tag on WattClarity for more…
(D) Where ‘expectation’ data is not so readily available
Of course it’s important to remember that (despite the richness of the data sets published by AEMO … and other organisations like ASX Energy (in relation to price expectations in the futures contracts), and also now FEX Global) there are other expectations that are not published in public data like the AEMO forecast data sets, but are still quite relevant to analysis of particular outcomes seen in the NEM.
For instance, changing expectations for how much ‘fuel’ might be required for a particular generating unit (be that coal, gas, or energy stored in hydro or batteries) is quite relevant … but not clearly visible in the published data.
