On Saturday 29th August, I wrote about how ‘Victorian Scheduled Demand plunged..’ and how it was probably a ‘…new daytime record low’ point (note also the notes appended to that article specifically about Portland Aluminium smelter, and its significance to these records).

I followed this up on Sunday 30th August with (the first part of) a deeper exploration of what went on during that period – though time constraints precluded me digging as much as I would have liked.

Also worth noting that Giles wrote about ‘Wind and solar farms face more cutbacks due to new network constraints’ on Sunday 30th August on RenewEconomy.  That article focused on South Australia, but there was also pain in VIC and QLD as well.

On Monday 31st August I noted that the AEMO gave notice on LinkedIn about three new winter minimum Operational* Demand records over the weekend – in VIC, SA (Sat 29th) and also in WA (Sun 30th) … and also here on Twitter.

*  Note that I had spoken about Scheduled (i.e. Market) Demand – which is slightly different than Operational (i.e. Grid) Demand, hence my lower numbers.  For those who forget the many different ways in which electricity demand is measured, here’s the detailed explanation.

AEMO followed this note up with this newsroom article about ‘A record breaking winter weekend’ on Tuesday 1st September – with associated push here on Twitter and here on LinkedIn.

———

Well, in a sign of this rapidly evolving energy transition, we’ve already moved beyond the events of 8 days ago with:

Subsequent Event #1)  the Scheduled Demand dropped further on Sunday 6th September to set what’s clearly a new record minimum point at any time of the day.  Whole of NEM demand on Sunday 6th Sept was a little higher than it had been 8 days previously.

Subsequent Event #2)  today (Mon 7th Sept) the ESB has released this ‘Post 2025 Market Design Consultation Paper’ which, amongst other things, speaks to the looming challenges related to these minimum demand events …. including the closure of coal-fired power in the coming decades.

———.

… but let’s return to Saturday 29th August for a deeper look, because  of keen interest to me for a number of different reasons:

Reason 1)  Firstly, it pertains to minimum demand, which is fast growing to equal peak demand as the most stressful pinch point of all the periods that we encounter in the NEM.

Reason 2)  This very low level of Grid (and Market) Demand has been juxtaposed underneath the increasing supply of energy from new renewables capacity layered onto remaining supplies from synchronous units which (at least in the medium term) will be the mainstay of ‘keeping the lights on services’ like inertia.   There’s obviously some dissonance at work here, which speaks both to:

2a)  The Macro picture, about how we continue to chart this energy transition into the future; and

2b)  The Micro picture, about how individual market participants deal with the challenges.

Reason 3)  Because the NEM was swamped with capacity over the weekend, negative prices were present through QLD, VIC and SA for much of the day.

3a)  In response to these negative prices, a number of Wind Farms and Solar Farms curtailed production to avoid paying money to generate;

3b)  There are several ways through which they might have curtailed this production:
i.  the recent AER Issues Paper has raised concerns with those which adopt Paradigm 2;
ii.  so I was interested to see whether I could spot this kind of behaviour in what I can see of what happened on Saturday.

3c)  As I have noted before  (such as here) this type of ‘switch off at negative price’ behaviour is not just isolated to larger Semi-Scheduled plant – in fact, the looming challenge for AEMO might even become greater for Non-Scheduled and even smaller invisible (particularly solar) generators tucked away in the distribution grid.

Reason 4)  More broadly than the specific behaviour highlighted in the AER Issues Paper, it’s times like these (with many different assets all changing output) that seem to encapsulate higher systemic risk in the ‘balancing supply and demand’ dynamic.  Hence I remain keen to investigate as many events as possible, in order to understand what’s going on with respect to:

4a)  Individual asset ‘unders and overs’ – which we can assess in terms of ‘Raw Off-Target’; and

4b)  Collective ‘unders and overs’, such as for all Semi-Scheduled assets – which we have been assessing as Aggregate Raw Off-Target

… so in between times over the past week,  I’ve continued to explore deeper.

(A)  Trended results, at an Aggregate level – for all Semi-Scheduled assets

The following day, I presented this trend of Aggregate Raw Off-Target over 12 daytime hours through Saturday 29th August 2020:

As I had recently done in this Case Study of 7th November 2018, I found it useful to present the results of unbundling the aggregate trend (still shown in the bar chart on the RHS axis) to show what amounts to the same type of trend for 93 individual DUIDs which were operational at the time (with that data shown on the LHS):

Keep in mind the following bids of context:

Context #1)   It’s busier than last time, because:

#1a)   There are more DUIDs operational than there were in November 2018; and

#1b)   We’re looking over a longer time range

… however the chart still achieves the desired effect of highlighting a significant number of large discrepancies just in this 12-hour period to focus on below.

Context #2)   Results for the individual DUIDs are scaled on the axis on the LHS – which is twice the magnification of the axis on the RHS which shows the aggregate (in the bars graph).

Context #3)   Remember that positive numbers are under-performance (with respect to Target) whilst negative numbers are over-performance – as explained here.

Context #4)   Solar Farms are coloured yellow and wind farms are coloured green.

Context #5)   For some of those DUIDs sporting ‘large’ deviations I’ve highlighted them unique colours, to make them visible on the busy trend.

Context #6)   The choice of what’s ‘large’ is a little subjective – however I have based it on two things:

#6a)   Practical considerations, like my time constraints in drilling into each – and also my sense of how long even a patient reader here might stay with me to get to the bottom of this (quite long) article;

#6b)   More holistically, though, I have also made the choice based on the fact that 30MW has been the historic threshold where a dispatchable asset must be registered with the AEMO and operate as a Scheduled Generator.  

… Seems a good criteria to also use in looking at the growing number of invisible, and quite random ‘phantom generators’ we have entering the NEM which are operating as the ‘evil twin’ of a Semi-Scheduled generator which would be fully predictable in a dispatch time horizon through the AWEFS and ASEFS processes.  Cue Austin Powers audio?

(B)  Unbundling the results

So with all of the above in mind, we’ll step into unbundling the results in two different ways – firstly focusing on 8 discrete dispatch intervals, and then singling out a couple of other DUIDs that show curious behaviour …

(B1)  8 x Specific Dispatch Intervals

… for a first pass, I’ve flagged 8 particular dispatch intervals of interest out of 144 in total:
1)  6 x dispatch intervals, where the dispatch price in QLD or SA dropped well below –$100/MWh (i.e. clearly below ‘negative LGC’) and sometimes as low as –$1,000/MWh; and
2)  Another 2 x dispatch intervals where the Aggregate Raw Off-Target was in greater than +200MW (i.e. collective under-performance).

With each of these 8 x dispatch intervals, I’ve scanned quickly and have highlighted some behaviours and results that particularly jumped out at me.  Given the quickness of the scan:
1)  It’s almost certain I have overlooked other things which would be of interest to readers here; and
2)  It’s possible I have made mistakes in the quick interpretations.

Most of the images below have been taken from various configurations of our ez2view software, leveraging heavily on the ability for the user (i.e. me, in this instance) to design their own dashboard, and also the powerful Time-Travel functionality which shows the market as it appears at the chosen Dispatch Interval, including the AEMO forecasts that occurred at that time.

(B1a)  At 06:35 the QLD dispatch price drops to –$222.94/MWh

Let’s start at 06:35 where we see the price dropped in the Queensland region at the same time as there was a small blip in NSW – but well before solar generation had ramped up to play any real role:

With respect to the above snapshot, I have annotated 3 specific observations:

Observation #1)   First and foremost, we see the key role (in this instance) that an FCAS (Lower 6 second contingency) bid from Gladstone Unit 6 had played in setting the price for ENERGY in the region.

This outcome is one of the surprisingly common instances we categorised as ‘Complex’ in this review back in February 2019 (i.e. occurs roughly 20% of the time because of the complexity in multi-region, multi-commodity co-optimisation).

Observation #2)   We see both that this price drop had been forecast in advance and that a number of the solar farms have already deployed auto-bidders (there are several we know of on the market) which rebid the plant even before it was actually able to produce any output:

#2a)   As noted on the image, there seems certainly a scope for a subsequent WattClarity® article about ‘The Rise of the Auto-Bidder’ which could explore the state of that particular market. 

… or perhaps we might save that for the next update to the Generator Report Card 2018 to assess in more detail.

#2b)   Certainly I wonder about the actual effectiveness of another piece of fancy tech that is being introduced to the NEM, and in particular about their real compliance with the AER’s Rebidding Guidelines.

Observation #3)   Finally, I have also tagged the ‘Q_NIL_STRGTH_MEWF’ System Strength constraint which was seeking to limit the output of Mt Emerald Wind Farm in Far North Queensland.

(B1b)  At 08:20 the SA dispatch price drops to –$1,000/MWh

At 08:20 we see that the dispatch price for South Australia dropped to –$1,000/MWh for the first time that day.

Let’s start with this snapshot from 08:15 which illustrates that the AEMO (in P30 predispatch) had been fairly consistent in forecasting that the price would drop – however the forecasts were that the drop would not occur until the 10:00 Trading Period.  This turned out to be about an hour later than what actually happened:

With this in mind, we wind forward 5 minutes to the 08:20 (i.e. price drop) dispatch interval, and look at a focused view of LKBONNY2 wind farm in South Australia, as it reveals something interesting:

As noted on the image, the two ‘actual’ dispatch intervals shown in the ‘Unit Dashboard’ widget show two different types of behaviour:

Observation #4)   At 08:15, the unit’s CPD Price (at –$22.31/MWh) is below the bid price for the unit (-$3.10/MWh).

#4a)   As such, NEMDE wants to dispatch less of this (relatively) expensive energy;

#4b)   However NEMDE can’t wind the unit down entirely, as its bid ROC Down (i.e. rate of change down) rate is only 3MW/min, or 15MW in the dispatch interval;

#4c)   Hence the dispatch Target is set as 59MW – 15MW = 44MW for the 08:15 interval; and

#4d)   In order to tell the generator that it has a target it needs to follow (keeping in mind the leeway given to Semi-Scheduled plant) the Semi-Dispatch Cap (SDC) flag comes on.

#4e)   In this particular instance, the unit reduces output exactly as the Target suggests, to land at 44MW (‘Actual’) by the end of the dispatch interval.  All good so far…

Observation #5) The 08:20 dispatch interval is a different case entirely

#5a)   The mechanics are basically identical – from NEMDE’s point of view, it can only ask for a further 15MW reduction because the ROC Down rate is still 3MW/min;

#5b)   However in this case the unit trips offline entirely, dropping from 44MW to 0MW at the end of the period.

#5c)   The most obvious difference here is that the Dispatch Price is a whole lot lower, at –$1,000/MWh, hence we presume a whole lot less attractive to an operator which had offered their volume at almost $1000/MWh above that mark.

#5d)   This seems to be a clear case of an operator adopting the Paradigm 2 perspective discussed in response to the AER Issues Paper .

#5e)   This hypothesis seems to be reinforced further when we see that the participant actively bids the volume out of the market in the 08:25 dispatch interval with the rebid reason referencing the price floor …. but only after they have physically removed the volume in the first place:

Observation #6) Readers need to understand that the AER has seemed to confirm that this type of behaviour is not currently against the market rules, though there are quite a number of people for whom it does not seem ‘fair’ and equitable (i.e. Paradigm 3) – and there are some, like me, who wonder what the long-term sustainability and scalability is of such a practice.

#6a) I do note, in this instance, that the aggregate Raw Off-Target for all Semi-Scheduled units right across the NEM was +27MW, so the +30MW Raw Off-Target for Lake Bonney 2 unit did not amount to a big thing in aggregate, in this instance.

#6b) Given the company is just in the process of changing shareholders, now, I am curious to see if the approach adopted by the generator might change in future – regardless of the outcome of the AER consultation process?

 (B1c)  At 08:50 the SA dispatch price drops to –$1,000/MWh

At 08:50 the dispatch price again dropped, in the middle of the 09:00 trading period.

In this case, the Aggregate Raw Off-Target for the dispatch interval was just +1MW (i.e. across all Wind and Solar plant the unders and overs perfectly cancelled out).   That’s a perfect illustration of the ‘diversity will fix it’ school of thought – however more broadly we’re wondering about the extent to which diversity is not a magic wand.   Hence all these Case Studies of extreme DIs, for instance.

With this in mind, I’ve singled out the largest Raw Off-Target in the dispatch interval, which was a 45MW over-performance by Daydream Solar Farm in QLD:

Observation #7) Without investigating in more detail, a trend like this might be because of the passage of cloud cover over the solar farm, resulting in:

#7a) A large (74MW) under-performance in the 08:25 dispatch interval – perhaps when the cloud passed unexpectedly; and

#7b) A large (-45MW) over-performance shown above for the 08:50 dispatch interval.

Observation #8) Over the 12-hour period, there were 13 dispatch intervals in total where DAYDSF1 had a Raw Off-Target deviation larger than 30MW:

#8a) This was 7 large under-performance DIs and 6 large over-performance DIs – passage of cloud cover, perhaps (have not investigated)?

#8b) In total, this represents 9% of the dispatch intervals in the 12-hour time range scanned here – which is 3rd worst for the day across 93 units.

#8c) Looked at this another way, the DAYDSF1 ‘phantom generator’ was quite active.

 (B1d)  At 09:10 the SA dispatch price drops to –$1,000/MWh

At 09:10 the dispatch price again dropped, but this time the Aggregate Raw Off-Target was a collective (+174MW) under-performance.  Two large ones jump out at me (both in SA) in this dispatch interval – with the largest being a major under-performance at Lake Bonney 2 Wind Farm:

Observation #9) There is a major under-performance at LKBONNY2 – again coincident with a drop to the Market Price Floor:

#9a) This one (+91MW) is larger than what happened earlier;

#9b) We see that the unit ramped up faster than expected the dispatch interval beforehand (-34MW) followed by the unexpected trip in the next dispatch interval

#9c) Also a bit perplexing is  to see that the Available Capacity also drop to 0MW in this dispatch interval.

#9d) As per last time (not shown in the image above) the next dispatch interval sees capacity bid up to high prices with the rebid reason again referencing the price drop.

The other unit was SNOWTWN1 where it appears that something different was happening:

Observation #10) In the 09:00 dispatch interval, the generator had volume offered quite high, because of the negative price excursion in the prior trading period.  Volume was reverted down to the –$1,000/MWh price band for 09:05 but the unit failed to start on both the 09:05 and 09:10 dispatch intervals.

(B1d)  At 13:55 the SA dispatch price drops to –$900/MWh

In early afternoon (after a break where prices were consistently negative, but not hugely so) we see the price again drop late in a trading period – this time at 13:55.

Aggregate Raw Off-Target was a moderate (+45MW) under-performance.  Of all of the 93 units assessed for this dispatch interval, only one had Raw Off-Target larger than 30MW in either direction, and this was Susan River Solar Farm in QLD:

Observation #11) The chart in this snapshot for SRSF1 spans the past 3 hours, and shows the output bouncing up and down like a yo-yo over that period – presumably due to cloud cover.  The +34MW deviation in this dispatch interval is clearly not the largest for the unit through the day, with the one immediately before being +53MW.

(B1e)  At 16:25 the Aggregate ROT was +296MW

At 16:25 the Aggregate Raw Off-Target was the the largest it was for the day, and (at +296MW) very close to the threshold we’ve been using to define ‘extreme’ in these Case Studies of extreme DIs.

Observation #12)  Of the 93 DUIDs registered on the day, there were three which under-performed by more than 30MW each – accounting for 143MW of the total of 296MW … so less than half.

#12a)  There were another 7 DUIDs whose under-performance was at least 6MW that all collectively contributed much of the 153MW net difference between the top 3 and the aggregates.

#12b)  Time and space precludes looking at all of them, here – but don’t infer that they were not important in their own way.

Here are the ‘worst’ three:

1)  Nevertire Solar Farm in NSW (+35MW at 16:25)

Here’s NEVERSF1:

Observation #13)  Whatever is affecting NEVERSF1 in this 16:25 dispatch interval has been affecting the plant (and/or NEMDE) since 15:25:

#13a)    At 15:25 the output unexpectedly dropped to 26MW (i.e. a Raw off-Target value of +60MW then), and has remained stuck there since that time.

#13b)    For reasons unknown, this persisted since that time, with the imbalance amount reducing with ASEFS’s assumed solar decline through the afternoon.  Hence the +35MW error at 16:25 is the lowest in the past hour.

.

2)  Bannerton Solar Farm in VIC (+58MW at 16:25)

We see a 58MW under-performance at BANN1 for this dispatch interval:

Observation #14) We see here that the under-performance of +58MW arose because the unit was asked to hold at 60MW, but tripped (for unknown reasons) down to 2MW:

#14a) Through the afternoon, I can see that two constraints (the focused ‘V_BANSF_BBD_60’ and the more general ‘V>>V_NIL_9’) have been seeking to ‘constrain down’ the output of BANN1, which might be one of the reasons why the unit shifted its volume down to –$1,000/MWh bid band from 15:05 (however note that this bid had been lodged the day before at 10:57).

#14b) The estimated trading price for the 16:30 trading period was –$7.87, so it does not seem likely that price was a factor in this sharp drop in output.

.

3)  Snowtown 1 Wind Farm in SA (+50MW at 16:25)

Remember that SNOWTWN1 also featured in a dispatch interval earlier in the day:

Observation #15) First thing that jumps out in this image is how SNOWTWN1 has been quite active in rebidding its capacity over the past 3 hours, with the form of the rebid reinforcing the use of an automated bidder in this case (another cue for a subsequent WattClarity® article about ‘The Rise of the Auto-Bidder’ )

Observation #16) As with the earlier dispatch interval featuring SNOWTWN1, this unit here also ‘fails to start’.  It’s given a target to rise from 0MW to 50MW, but is unable to get off the ground – hence the +50MW under-performance in Raw Off-Target value.

(B1f)  At 16:40 the SA dispatch price drops to –$1,000/MWh

Fifteen minutes after this, and in the next Trading Period, we see the dispatch price drop to the Market Price Floor for the 4th time that day in South Australia.

Observation #16)   In this dispatch interval, of the 19 DUIDs in South Australia, we saw considerable churn requested by NEMDE:

#16a)   We saw request that three units (BNGSF1 and BNGSF2 and LKBONNY2) significantly reduce their output.  All three of these units did reduce, with the exception that LKBONNY2 appears to have again tripped in response to the large negative price (Raw Off-Target = +12MW).

#16b)   Additionally we saw requests that four units (LGAPWF1, SNOWNTH1, SNOWSTH1, SNOWTWN1) significantly increase their output.  Three seemed to manage this well,  but again SNOWTWN1 appears to have ‘failed to start’.

(B1g)  At 16:50 the Aggregate ROT was +256MW

The 8th dispatch interval highlighted in this 12-hour period saw the second largest aggregate deviation from Target

Observation #17)  There was a total under-performance of +256MW:

#17a)  In aggregate +208MW of this total was contributed by just 3 x DUIDs – DAYDSF1, BANN1 and MUWAWF1;

#17b)  Meaning the remaining 90 DUIDs in aggregate net terms accounted for +48MW

1)  Daydream Solar Farm in QLD (+43MW at 16:50)

Here’s DAYDSF1:

Observation #18)  We see here that DAYDSF1 has begun declining through the afternoon, but at 16:50 (as at 16:00 beforehand as well) experienced an unforecast drop in output.  I’m guessing that this was some passing cloud cover that was not accounted for in the Availability forecast?

.

2)  Bannerton Solar Farm in VIC (+37MW at 16:50)

Here’s BANN1:

Observation #19)  We see here that BANN1 a repeat of the ‘unforecast trip’ also shown in the 16:25 dispatch interval above… increasing my desire to understand what the cause was?

3)  Murra Warra Wind Farm in VIC (+129MW at 16:50)

Here’s MUWAWF1:

Observation #20)  We see here that Murra Warra Wind Farm also displayed (similar to Snowtown 1 Wind Farm in a couple cases above) a ‘fail to start’ response to a Dispatch Target increase … in this case to 143MW (managing only 14MW by the end of the dispatch interval):

#20a)  We see here that the unit had been bid ‘out of the money’ in order to avoid being dispatched for a number of hours – since the 10:30 dispatch interval, it turns out;

#20b)  Rebid at 16:50 puts it back ‘in the money’ so receives a large Target because of its 45MW/min ROC UP rate unchanged in the bid.

#20c)  Target is capped at the AvailGen for the plant (143MW), but it only gets to 14MW – hence the +129MW Raw Off-Target

.

(B2)  Specific DUIDs with high individual results

Returning to the 12-hour view, but flipping the representation, we see another interesting representation of the DUIDs where the ‘evil twin phantom generators’ have been working overtime to consistently deliver significant deviations from target:

This article is already quite long, and some of these have already been flagged above, but I wanted to specifically highlight a couple below:

(B2a)  What happened to Macarthur Wind Farm

In part because it’s become like an ‘old friend’ in all these Case Studies of extreme DIs, and also because I was asked by others ‘what’s happened to Macarthur WF?’  it was noticeable that Macarthur WF was missing in action in the day above.

Observation #21)  We can see in this trend above from NEMreview v7 that it had been offline since the morning of Tuesday 25th August, and that it’s still offline  at Sunday evening 6th September:

#21a)  That’s a significant outage (12 days and counting)!

#22a)  Unfortunately, requirements for reporting of data are different for Semi-Scheduled plant (both in terms of the participant to the AEMO, and also in terms of AEMO to the market) as a result of which we’re unable to really dig further to understand what the problem is with the plant.  As we noted in Theme 14 within Part 2 of the 180-page analytical component in the GRC2018, this seems to be an increasing anachronism:

i.  The following is a ‘Bid Details’ view in ez2view showing the bid that took effect on 12:15 on 25th August, when Macarthur set its ‘MaxAvail’ parameter in the bid to 0MW from the 11:00 trading period:

ii.  The ‘Compare Bids’ function in ez2view shows that the Registered Participant (AGL in this instance) took the plant out of service in the bid registered at AEMO at 10:45 on Tuesday 25th August:

… however the rebid reason used then was ‘10:23~P~AVAILABILITY STATEMENT (10:35~SEMI-SCHED AS ADVISED BY OPS)’ does not really provide much help to understanding what’s been going on!

iii.  The ‘Bids and Offers’ widget in ez2view shows that AGL has continued to submit bids, with the MaxAvail continues to set to 0MW in all of them, but nothing really helpful in any of the bid reasons to provide an indication of what the issue is, or when the plant might return:

Deficiency 1 = At least in this case, AGL chose to change the MAXAVAIL in the bid to 0MW.  Because AWEFS overwrites this, some Semi-Scheduled participants don’t even bother to change this, meaning we’d be totally flying blind.

Deficiency 2 = However as it is, the repetitive Bid Reason format does not shed any light on what’s actually going on!  Particularly in the light of the ‘rise of the auto-bidder’ (not in use here, but topical because the format is often even less useful from a practical point of view) there is plenty of food for thought on how the AER might update their Rebidding Guidelines in order to  ensure that bids are as useful as possible in terms of really informing the market

Deficiency 3 = Pity that the new MT PASA unit-level data does not cover Semi-Scheduled generation!  We really are still quite blind as to what’s going on at MACARTH1 in this instance!

(B2a)  Lincoln Gap Wind Farm in South Australia

As a final curiosity, I’ve added back in the trend view, but having first removed the other 92 units so we can quite clearly see a very peculiar phenomenon, being the ongoing periodic significant over-performance at LGAPWF1 through many of the trading periods in the day:

I don’t have time to delve into this here, but would be interested to hear from someone who could shed more light?

.

(C)  A comment about Scheduled Generators (and Loads) on Saturday

Readers who have stayed with me this far to the bottom should remember that there’s probably plenty of stories about, and analysis into, the behaviour of the Scheduled Assets (both old thermal & hydro units and newer batteries) in response to these negative prices.  The fact that I have not focused on them here should not be taken as any sign that:
1)  They are not important; or
2)  We are not interested in them.

This could be something we assess in more detail in the upcoming Generator Statistical Digest 2020 (release Jan 2021), and the follow-on Generator Report Card update (release perhaps April 2021).