I’ve written today about 3 Potential Tripwires looming in market changes that take effect during October 2021 … two with respect to Five Minute Settlement (5MS) and one with the Wholesale Demand Response Mechanism (WDRM).
In this article I will try to explain what we mean about Potential Tripwire #1 (those ‘invisible’ 5-minute trading periods) … at least from the perspective of forecasts in advance.
—————
Reminder – this series of articles is written as a service to our broader readership, to help ensure as many as possible are aware of these potential tripwires:
(1) Some of our readers will already be aware of these tripwires (and you may well be one of them) – but conversations we’ve had with others demonstrate that others readers won’t be aware (in which case, this article is for them).
(2) For those who are (or who become) aware, this issue may be a non-issue for you (a bit of a yawn, even) … but for others it could potentially be an issue:
(a) I’d personally hate to be in the position months after October 2021 if I had to admit “Well, I knew this was coming, and was a potential challenge for you, but did nothing about it”.
(b) If you have direct questions about it, please give me a buzz (+61 7 3368 4064 or via LinkedIn etc)
(3) For this reason, we’d ask that you please circulate this article (and the two that follow) to others that you know of who could be affected
(4) We’re building out our software to do what we can to help those who use it keep these things in mind as we roll into October 2021 and beyond…
—————
With that reminder out of the way, let’s get on with explaining this ‘invisibility’.
(A) Compare and contrast the forecasting time horizons (pre-5MS and post-5MS)
In the following table I have tried to briefly sum up what’s changing.
| pre-5MS | post-5MS | |
| Dispatch Prices |
Published ~20 seconds into each 5-minute dispatch interval |
No change from the way it currently works |
| Trading Prices |
Applies to each 30-minute period. Not known with certainty until 25.20 minutes into a trading period (hence known as the 5/30 issue). |
Applies to each 5-minute period. Known with certainty approximately ~20 seconds into each trading period. |
| P5 Predispatch Prices |
Published ~40 seconds into each 5-minute dispatch interval and containing 12 forecasts … but note one is for the current dispatch interval and so is redundant, as the actual dispatch price has already been published at that point. Hence only 11 useful forecasts. |
Essentially no change in the process. However note that bids will be able to be different for each of the 5-minute periods in advance, so this may well mean more variability between connected 5-minute period than is currently the case. |
| P30 Predispatch Prices
Note that AEMO don’t actually label this ‘P30 Predispatch’, they just label it ‘Predispatch’ – but this is something we have done for a long time to be unambiguous. |
Published ~40 seconds into each 30-minute trading period. Extend out until 04:00 tomorrow, or the day after – providing a forecast for every trading period over that range. Because the trading price is the average of the 5-minute dispatch prices, this provides some clue to what might happen in each discrete dispatch interval. |
The mechanics of this process is not going to change a whole lot compared to how it currently works (part of the MVP constraint noted below). —- The way it works pre-5MS is able to work that way because generator ‘Initial Bids’ are for a full half-hour period. In the pre-5MS environment, it’s only at real time that bids can change on a 5 minute basis. —- However that’s not going to be the case post-5MS. Instead, generator Initial Bids will be able to vary for each 5 minute period. Because of this, a decision was made to select the 6th period in each half hour (i.e. ending HH:00 or HH:30) as the bid files to sample – which leads to this potential tripwire…. |
(B) Using the ‘Forecast Convergence’ widget to illustrate
Many years ago now, we introduced a ‘Forecast Convergence’ widget in our ez2view software to help clients understand how AEMO’s successive forecasts are changing (this built on something that had been in deSide® even earlier).
It’s important to understand that the market requirement to follow successive market forecasts goes well beyond looking at AEMO forecast prices … and covers both:
1) Inputs to NEMDE … such as DUID Availability (i.e. Generators and Loads), and Demand forecasts, and so on…
2) Outputs from NEMDE … be they DUID Dispatch Targets, or Interconnector Targets, or Prices (for Energy and 8 x FCAS commodities).
… and that this applies to P5 predispatch, P30 predispatch, ST PASA and MT PASA. That’s why we have built (and will continue to expand) the widget in ez2view in this way!
(B1) An explanation of how the ‘Forecast Convergence’ grid works
Given the continued volatility last night (i.e. Wed 9th June 2021) it was logical to focus on the QLD region, and specifically the 17:35 dispatch interval, which was when one of the spikes occurred – here’s a view of the ‘Forecast Convergence’ widget Time-Travelled back to 17:35 yesterday:
As per the annotation, if we look up the vertical to 17:35, we can see how AEMO’s successive P5 forecast gave some clues that the final price might be volatile. We can see that (with this data) the first direct clue was the P5 predispatch run published at 17:05:20 (with the forecast up at $13,988/MWh).
However (currently) the P30 predispatch forecast also provided some clues – if we flip ‘Forecast Convergence’ to ‘DAY’ view the context of the widget changes:
As per the annotation, the box at the top (empty) is where the 18:00 trading price would appear, after it is published at ~17:55:20. But at the time of the 17:35 spike, that’s not known with certainty.
Looking up the column, however, we see that the AEMO P30 predispatch process also gave clues of the likelihood of spikes at 17:35 … even 24 hours in advance. This would be plenty of time for anyone to prepare for a response (whether that be a generator, a battery, or a demand response unit, etc… – even the slower responding ones).
—-
It’s important to understand that this P30 Predispatch process is changing such that 5 out of 6 (new 5-minute) Trading Periods become invisible come 5MS.
(B2) More context to the way ‘Forecast Convergence’ works
Each widget within ez2view comes with its own focused help page for each widget … and there are lots of them! The Widget Guide for ‘Forecast Convergence’ in v8.0 is currently here.
To understand more of how Forecast Convergence works, here are two other sources of reference:
1) Where we have remembered to do it, we’ve tagged some articles when we have used the widget … but you will see we’ve not been too reliable in using that tag.
2) Relying on the magic of search, here are the articles Google finds referencing the use of ‘Forecast Convergence’ widget
(B3) Mocking up an extended ‘Forecast Convergence’ using both P5 and P30 Predispatch
In order to illustrate the structure of the AEMO’s forecasts (i.e. P5 Predispatch, and P30 Predispatch) and how they work together, I have mocked up an extended ‘Forecast Convergence’ grid looking at 5-minute prices (currently Dispatch Prices) under the current structure of the data set.
In other words, I have combined the two ‘HOUR’ and ‘DAY’ grids above to show how it would currently work:
… a reminder to click on the image to open a larger view.
There’s probably a (manual) transcription error or two in translating all of the prices (and preceding forecasts) but it should be enough to illustrate the general context, which is…
1) There were 11 x sequential P5 Predispatch forecasts for the 17:35 dispatch price published over a (slightly less than) 60 minute period beforehand; and
2) Given the P30 Predispatch forecasts for the 18:00 Trading period encompasses the 17:35 dispatch interval, those earlier P30 forecasts gave some indication of what the price might be for 17:35 as early as ~12:31 on the day before (Tuesday 8th June).
—-
After 5MS commences, however, this forward visibility is substantially reduced, unfortunately, to the point whereby 17:35 becomes invisible until <60 minutes out! Here’s how it looks:
As we can see, there are a fair number of cells in the grid highlighted with ‘invisible’ trading period forecasts – which, put simply, would mean that the first time market participants would see a direct forecast pertaining to 17:35
(B4) But can’t I just look at the forecast for 17:30 or 18:00, I hear you ask?
Seems like a logical question to ask, I know!
However let’s think back to the reason why the P30 predispatch process is changing in the first place … and that is because the bids for 17:35 will be able to be different (in advance) from the bids for 17:30 and the bids for 18:00.
Indeed, we have been wondering how long it will take for Market Participants (those big, bad greedy coal generators, but also the newer entrants as well) will recognise this ‘invisibility’ as a potential advantage, and explore how offering different bids might help them achieve the market outcomes that they want.
Say, for instance, you offered 1000MW at $30/MWh for 17:30 and 18:00 – but, for 17:35, 17:40, 17:45, 17:50 and 17:55 you offered that 1000MW at $15,000/MWh.
1) This would need not be a rebid … it could, for instance, be your Initial Bid that you specify to AEMO prior to 12:30 the day before … but stays invisible to anyone else (even in terms of price effects) until quite close to real time.
2) The first time anyone else would be aware of this bid profile would be 55 minutes from dispatch!
We’ve not had time to think this through as much as we’d like at this point – but the point is that I’d be very surprised if participants don’t at least try to see if they can catch others napping…
—
So we’ve now done our job (through this article) of ensuring that you at least had a chance to be aware of this … and (as noted here earlier today) would appreciate you passing on the message, please!
(B5) Upgrading ez2view version 9x (including ‘Forecast Convergence’ to deal with these complications…
Once we have ez2view v9.0 released as a starting point for operations during 5MS, there are other things we can do in ez2view (i.e. beyond how it’s already been modified for the changed data structure) to make the challenge clearer.
Contact us directly, if you would like to know more about what we’re doing.
(C) Possible step to remedy this?
During the long consultation period that accompanied the implementation of Five Minute Settlement, in the appropriate Working Group a number of different people (on a number of different occasions) requested that the P5 Predispatch process be extended more than 55 minutes out … e.g. perhaps 2 hours, or 4 hours, or 6 hours. We were keen to see this happen, as well!
However my understanding is that this was not possible given two factors:
1) Generally the huge workload required of AEMO and everyone else to just get to a ‘Minimum Viable Product’ (MVP), if you like, about how 5MS would work; but also
2) The solve time being what it is with NEMDE and P5 Predispatch (it’s a complex process), extending it further forward (and expecting the solution to be updated every 5 minutes) perhaps could not even be physically completed. There’d obviously be work involved to try to optimise this, which AEMO and others just don’t have time for now.
—
My hope is that, once 5MS is ‘bedded down’ at the MVP level and we get through 1st October 2021, we can revisit a number of shortcomings – such as this one.
Leave a comment