As noted last week, our number crunching has begun for the purposes of compiling our Generator Report Card 2018.  Further information about the Report Card is provided online – including the chance to secure a discounted price when ordering before the report has been finalised.

With the report card, we are focused on answering a long list of questions about various aspects of performance of all generators, no matter what flavour.

(A)  Is the AEMO’s job becoming more difficult?

One of the headline questions we are curious to delve into is whether the AEMO’s job is becoming more difficult in dispatch of the market as the years progress and this energy transition continues?

There are a number of different aspects of the challenge of balancing supply with demand from one dispatch interval to the next – one of these is the extent to which AEMO’s expectations of where each individual unit will land (at the end of each dispatch interval) matches what actually happens.  In simple terms, whether units are “Off-Target”.

Whether or not units are Off-Target is something that can be measured for all Scheduled and Semi-Scheduled units, by comparing their Target values with the Initial MW values for the subsequent dispatch interval (i.e. their expected output with their actual).  So we have calculate this for every dispatch interval, for every DUID in the entire history of the NEM (less a few dispatch intervals we’re missing from the start).

It’s the same comparison that is the root of the AEMO’s “Aggregate Dispatch Error” calculation, however I was curious to look at the numbers a different way for the entire history of the NEM.  Hence for this preliminary analysis, we have:

Step 1 = calculated an “Off target” measure for every DUID in a given dispatch interval
Step 2 = taken the absolute value of that Off Target “error” (i.e. some will be above 0MW and some are below 0MW, but we’re only interested in the scale at this point; and
Step 3 = Aggregated the numbers up for every DUID in that DI up to derive a single measure for each Dispatch Interval.

We call this “Aggregate Absolute Measure of Off-Target”.  It’s probably not exactly what a formal statistician would define (they would probably prefer something like Root Mean Square Error) but it’s useful enough for our purpose here.

The three charts that have included here shows a trended distribution of a metric calculated for every single dispatch interval over the 20-year history of the NEM (minus a few orphan intervals in 1998):

Chart 1)  Low numbers

In the first chart included here, we see that almost all DIs see an “Aggregate Absolute Off-Target” being more than 100MW, and that the incidence for the measure being above 200MW seems to be steadily increasing from the early years – with an aberration in the early years that has not been explored.

I’ve noted on the chart a suspicion that this trended increase for >200MW might just be due (at least in part) to a growing number of DUIDs involved in the dispatch process.

Chart 2)  Mid-range numbers

In the second chart, we see that “Aggregate Absolute Off-Target” appears choppier in the mid-range incidence levels.

There appears a notable decline from around 2009 through to 2017 (perhaps linked to the decline in demand?) before a sharp spike upwards for 2018 (for reasons that have not been investigated).

Chart 3)  Low incidence and High “Error” numbers

In the final chart, we look at the upper range of Aggregate OffTarget error:

Each specific Dispatch Interval could be individually investigated (if we had enough time, which we clearly don’t).  However out of curiosity I did track down the all-time high point being 10:50 on 2nd July 2009 – when a trip of all 4 x Bayswater units contributed over half of the total “error” of 4,031MW.

In these charts there does not seem to be a clear pattern emerging (though there is a marked reversal of trend in 2018 for the mid-range incidences that we would investigate further if we had time).  It might be that this has been due to the commissioning of many new solar farms in 2018 (i.e. whilst the ASEFS model is still being trained up to correctly forecast their output – however that’s entirely speculation on my part).

(B)  Different reasons why generators might not follow their targets

Readers (especially Villains no 3 shouting from either end of the Emotion-o-meter) need to be very clear that there are several significant reasons why an individual generation unit might see an actual output vary from what the AEMO expects it will be, or tells it to be (in a dispatch target).

In compiling our Generator Report Card 2018, we will be exploring all of the significant reasons – but for starters let’s just flag a few in no particular order:

Reason B1)  Units sometime trip

Reminding readers of what guest author Hugh Saddler noted (that Unit Trips are normal occurrences, but system blackouts are not) we start with the fact that units occasionally trip.  This can happen for a range of different reasons – and can happen for all plant types (though the larger the unit, the larger the single effect a trip will have on balancing supply and demand).

As one example of a unit trip (of which we will find an increasing number more recently, if the #coalfail meme is to be believed), here’s a short post about what happened with Mt Piper unit 2 in the 23:45 dispatch interval on 24th October 2018.

In the preparation of our Generator Report Card 2018, we’re having a detailed look at what’s been happening with respect to unit trips (one form of units being Off-Target).

Reason B2)  Units sometimes fail to start

The reverse can also happen sometimes – “fast start” units might be sent a dispatch target instructing them to start, but fail to start (for whatever reason).

As an example of a unit that did not start, here’s a short post about what happened with Braemar A unit 3 on 29th August 2018 at 17:00.

In the preparation of our Generator Report Card 2018, we’re having a detailed look at what’s been happening with respect to unit failing to start (another form of units being Off-Target).

Reason B3)  Sometimes forecast output of Semi-Scheduled units are not as good as we’d like them to be

For the majority of time (different for different units, depending on the incidence of constraints – which we are also looking into) Semi-Scheduled wind and solar plant don’t have to follow dispatch targets from the AEMO.  On these occasions, they just produce what the sun or wind allows them to produce in that interval.

That does not mean that the AEMO is not interested in having a good understanding about what the unit will be producing at the end of a dispatch interval.  Currently, the responsibility for the forecasts rests with the  AEMO’s AWEFS forecasting system (for wind) or ASEFS (for solar).

As an example of forecast that appeared to prove more difficult than AEMO would have wanted, let’s look at this case for Macarthur Wind Farm on 11th August 2018 at 18:35.

In the preparation of our Generator Report Card 2018, we’re having a detailed look at what’s been happening with respect to forecasts not being as accurate as desired (another form of units being Off-Target).  We’re particularly interested to understand if:
(a)  Some Semi-Scheduled plants are more difficult to forecast than others; and
(b)  Whether there is any trend in the pattern (i.e. forecasts getting better, or worse).

Reason B4)  Sometimes Unit behaviours are questionable

On occasions where the Semi-Dispatch Cap is flagged, the Semi-Scheduled plant are required to pay attention to the Dispatch Target they are assigned for that particular dispatch interval (as noted before, the “target” is strictly more like a “cap”).

I’ve previously posted this article about a particular case on 24th July 2018 for Lake Bonney 2 and Lake Bonney 3 wind farms, both operated by Infigen Energy, where the wind farms appear to have ignored dispatch targets instructing them to increase and instead appear to have manually tripped the units in reaction to a negative price that they appear not to have liked.  My sense is that this would have been to the consternation of both:
(a)  The AER, as it is responsible for ensuring all generator bids are not false or misleading; and
(b)  The AEMO, as it had set the request to increase outcome for system stability reasons in the south-east of South Australia.

In the preparation of our Generator Report Card 2018 (and in the light of conversations with other new entrants who have been thinking of adopting what would seem to be a similar approach), we’re having a detailed look at this particular type of behaviour to see how many examples there are, by different portfolios.

Here’s other samples that we flagged in our searching:

Example 1 seems a very similar case at Lake Bonney 2 only a few weeks earlier on 7th July 2018, but

Example 2 seems (based on further information provided by those associated with the plant) to further illustrate the difficulties of getting to the bottom of what happens-and-why when units are off-target based just on public data in the MMS.

Reason B5)  Supply of FCAS Services

As further development work continued on the GRC2018, we recognized that one reason we’d omitted from this initial list on 7th January is that a unit might be supplying FCAS services during that dispatch interval.

In other words, a unit might have received a dispatch target to go from A to B over that 5-minute period – but, during the period received updated instructions to provide Raise or Lower Services for the FCAS as a result of which that the unit ends up at point C at the end of that dispatch interval (not B, as initially intended).  In this case, a unit would still flag as being Off-Target in our query however this should not be misinterpreted as this unit making the AEMO’s job more difficult by not following instructions.

—————-

PS added Jan 2020 – note that, as part of our work in preparing the GSD2019, we refined the logic for calculating ‘Raw Off-Target’ to take account of any Raise/Lower Regulation Enablement levels that had been assigned to that DUID – hence (using this improved definition) it would not be possible for a unit to be ‘Off -Target’ as a result of FCAS Regulation Enablement.

Other Reasons?)  To be investigated further

As time permits, we will be interested to delve deeper into the data to understand if there are other significant reasons why DUIDs might be missing their targets by large margins.

Most importantly, we’re interested to understand if any of these reasons are becoming more pronounced or prolific.