Back on 3rd August 2018, in a group effort involving a number in our team we posted  this “Beginner’s Guide to How Dispatch Works, and how Prices are Set in the NEM”.

It’s critical that readers understand that (as noted here) the NEM works with price-based dispatch where the marginal unit(s) set the price received by all generators.  As guest author Allan O’Neil noted here on January 11th:

“Fundamental to marginally priced markets is that prices are set by the last supply or demand reduction offer dispatched in order to balance supply and demand, after (in principle) arranging the volumes offered in order of increasing price. So it is the pricing of offers that fall at around the total demand level on the cumulative bid stack that is critical, rather than the average level of pricing across all market offers. ”

… so Allan goes on to say that:

“With the current supply-demand balance in the NEM, gas and hydro generators are frequently the marginal price setting plant”

… you can review Allan’s article, which focused more specifically on why prices through 2018 remained stubbornly high.

——————–

In the 6 months that have passed since our Beginner’s Guide was published, we have received numerous queries about when we might be posting the next edition – the Intermediate Guide, if you like.

In this post, we endeavour to to part of this, in that:

1)  We hope we deliver, in the following, a clear enough explanation of why the NEM operates (in the “real world”) at a much more complex level than that covered in the Beginner’s Guide.

1a)  In other words, we hope you understand, through this post, that reality is much more complex than simple theory.
1b)  Unfortunately, much of the discussion we’ve seen (e.g. on social media) about Price Setter seems to exist only down at that Beginner’s Guide level.

2)  However note that it’s not intended in this post that we actually fully explain all the nuances in these complexities.  We don’t have time to do this currently, but hope to return later to provide more of a fuller Intermediate Guide at some point in the months ahead (certainly after we’ve completed our Generator Report Card 2018, and the work that will flow on from that into our products like ez2view and NEMreview).

In recent times we’ve been fielding a growing number of calls from various people asking to know more about “who’s been setting the price” in the electricity market.  In part these questions have been spurred by the AEMO’s inclusion of trends in various reports (such as Figure 5 and Figure 6 in their Q4 2018 Quarterly Energy Dynamics report*)

As an aside, we’d like to note that these Quarterly Reviews are an excellent initiative, in our view.  Thanks to AEMO for delivering something that had been missing beforehand!  It seems pitched at about the right level (given the types of questions we keep hearing from new entrants, spectators and commentators).

Various others (such as Allan here) have also been referring to stats focused answering the same basic question.   These various analysts and publishers all seem to be basing their trends on interpretation of the data published in AEMO’s Price Setter files.

(A)  An introduction to AEMO’s Price Setter file

The AEMO started publishing these Price Setter files a number of years ago in response to Registered Participant requests to understand more detail about how NEMDE was actually determining price – because the price setting process is more complex than the simple bid stack analogies in our Beginner’s Guide would suggest.

These Price Setter files do a reasonable* job at explaining the multi-dimensional complexities involved in setting 45 prices in every dispatch interval (i.e. price for 5 regions, for Energy and 8 FCAS commodities).   However they are not simple.

Noting (only?) a “reasonable” job is not meant as a criticism – just a reflection that it’s actually a whole lot more complex than we might initially think, if our thinking is at Beginner’s Guide level – as I hope this article will illustrate quantitatively…..

We’ve been including these Price Setter files in our ez2view software (used by wholesale participants and others who have significant exposure to the wholesale market in some form) for a number of years, enabling clients to drill into particular dispatch intervals forensically to understand the specifics of what went on in those dispatch intervals of high interest.

However in more recent times, we have been investigating how else we might use these Price Setter files in other ways to help expand what we do in making complexity understandable to our broad range of clients:

1)  For a start, we are calculating (and trending) a measure of Aggregated “RRP Contribution Factor” on an individual DUID basis for inclusion inside our Generator Report Card 2018.  This work is currently underway (but not yet completed) so I can’t be definitive about specifically what has been done.

2)  Following from the completion of the Generator Report Card 2018, we will be taking the lessons we’ve learnt in dealing with a complete history of Price Setter data files over a number of years and applying these lessons in the enhancement of our existing software products in a number of ways:

2a)  For our ez2view (higher-end) product, we will be enhancing in some considerable depth – making full use of these insights.
2b)  Reflecting the more generalised analysis completed by our NEMreview clients (who tend more to be analysts sitting on the outside of the wholesale market, looking in) we will be applying the learnings we have made, but also striving to retain a level of clarity reflecting the fact that these clients may never have been involved within the wholesale market directly, so quite unfamiliar with Price Setter (and the complexities noted in this preliminary Intermediate Guide).

In this post I hope we clarify some of our reasoning in what we are doing.

If you have specific questions about the increasing number of ways in which we’re utilizing the raw data published in AEMO’s Price Setter files, please call us (+61 7 3368 4064) or use this Feedback Form.

(B)  Categories of Complexity in Price Setter

In the process of assembling the historical series of price setter files for our Generator Report Card 2018, we* have defined the following categories for getting some statistical sense of the level of complexity involved in the price setting process.

*  As we have been the ones to define the categories, readers should be aware that it’s not an “official” categorisation in that sense, and that someone else might have a somewhat different division of outcomes per dispatch interval and region (as we might ourselves, on a different day).  The purpose of this categorisation exercise is merely just to help explain the scale of the complexity, in terms of numbers of dispatch intervals involved.

Readers should be aware that we’re intending these categories to be used only for ENERGY prices (not any FCAS commodity), for each dispatch in each separate region.

(C)  Examples of these Categories

Applying these categories above to the Price Setter files for all Regions for all Dispatch Intervals from 1st Jan 2018 through until sometime on 5th February 2019 reveals an informative distribution that is different for different regions, as follows:

A couple of things jump out at me that I’d like to highlight for a start:

1)  For a start, we see a significant percentage of time (roughly 20% in mainland regions, and a massive 45% in Tassie) where bids to supply FCAS services are involved in the price set for Energy.

2)  In Category 1 for NSW and VIC we see a higher percentage of time than is the case for QLD.  Without looking in detail at the data, which is still to be done for the Generator Report Card 2018, my suspicion is that this is because the Snowy Hydro stations are operating as single DUID stations.  Similar scenario for most of the hydro generators in Tasmania.

3)  We see in all regions that more than 58% of the time (80% of the time in SA), the Price in that region is set by contributions from more than one Station.  Or, to flip that around, it’s significantly less than 40% of the time in all of the mainland regions that the price is set in the “Simple” or “Very Simple” way we talk about in the Beginner’s Guide (and which seems to be the common, but over-simplistic, understanding* of the average “NEM Spectator and Commentator”).

*  That’s an example of Villain no 4 in action, and an example where (I hope) this article will go some way to helping alleviate

To illustrate each of the 5 categories above, we’ve picked a single Region/DI combination from recent weeks to provide an example of what we mean in the categories – and used Time Travel in ez2view to dig forensically as follows:

(C1)  Example of Category 1 – Very Simple

It’s sometimes noted that us Queenslanders like the simpler life – though I note that it’s only 3.7% of the time that we see a “Very Simple” outcome in terms of a single DUID in the Queensland region setting the Energy Price for Queensland.

I thought I would start by highlighting one of the relatively rare examples that have occurred over the past 14 months – focusing specifically on the 22:00 dispatch interval back on 4th February – highlighted in this snapshot from ez2view in Time Travel mode:
Click on image for a full-screen view.

In the case shown here, we see a case where a bid from Swanbank E sets the price directly in the Queensland region (Category 1 = Very Simple), and also propagates through to the other mainland regions to play the sole role (with dynamic inter-regional losses) in setting the price in NSW, VIC and SA (not Category 1 in those cases!)

(C2)  Example of Category 2 – Simple

Stepping up the complexity (though only slightly) to the “Simple” category – we expand the filter to allow multiple DUIDs at one station to set the price.  As an example of this we roll the clock forward a few hours to 03:00 on Tuesday 5th February 2019 in Victoria, when we see the following:
Click on image for a full-screen view.

As seen in the table above, there’s a reasonable number of instances (close to 40% in some regions) where Price Setter outcomes are at this level, or Very Simple.

However that leaves a large number of instances (the majority!) that are not….

(C3)  Example of Category 3 – Slightly Complex

Given that I already invested the time to dig into the reasons why the price spiked in NSW unexpectedly in the 16:15 dispatch interval on Thursday 31st January 2019, I’m saving myself some time by linking into this here.

In that article, I included this snapshot from ez2view that highlighted (on the Price Setter widget) how Upper Tumut and Torrens B (units 1 and 2) were both involved in setting the price in NSW for that dispatch interval.  That’s:
(a)  Two different fuel types both involved – Hydro, and Gas
(b)  Units in two different regions involved – NSW, and SA via the interconnected NEM.

For those who like to look at aggregated data for “how much the [FuelTypeIHateorLike] has set the price”, there’s an open question about how you would classify this one.  Remember that DI outcomes are in this category or worse for >58% of the time.  Genuinely happy to take Your Suggestions on that one!

That particular example is also a good one to highlight how two things can be simultaneously true:

True #1 = the Price Setter file nominates both Torrens and Upper Tumut as being marginal units, and hence their bids setting the price; however

Also True #2 = my earlier article also highlighted a number of other factors contributing to why the price was what it was  (demand up, intermittent supply down, imports dropped (due in part to bidding behaviour, one of the comments notes), reduced capacity at Mt Piper 2, and the list would go on…).

Readers should take this as a note of caution about reading too much into Price Setter in isolation to all the other factors that contribute to making the price outcome what it is on all occasions.

(C4)  Example of Category 4 – Complex

As shown in the table above, a staggering 44% of the time we see that price outcomes for Energy in TAS are in the Complex category (for instance, that FCAS bids are involved in the way the Energy price is set).  Hence we thought it would be useful to highlight one of those particular cases – being 23:05 on 16th January 2019 via Time Travel in ez2view:
Click on image for a full-screen view.

As per the annotation, there’s not enough time now to explain all of what went on within that dispatch interval for the price of Energy in Tassie, but three highlights are:

(a)  There’s no contribution in Price Setter from any ENERGY offers in Tassie (or Energy Supply anywhere!) to setting the Tassie Energy Price;

(b)  The bid by Hornsdale Power Reserve to suck power out of the grid as a load (in SA) plays a big role in setting the price for Energy in Tassie (and note that the contribution is more than the price ends up being).

(c) There are numerous FCAS bids (in Tassie and elsewhere) that are involved in making contributions to the Tassie Energy Price – some positive and some negative contributions, with the net effect being that it reduces the Total Band Cost back down towards the final RRP.

Yep, price setting can be Complex!  And it is so for close to 20% of all dispatch intervals on the mainland (44% of the time in TAS) …

(C5)  Example of Category 5 – Very Complex

… but in a (thankfully small) number of cases (<1% of the time in each region) it becomes even more complex than the above.

We see a total of 216 dispatch intervals for South Australia in the table above, so I have singled out one of these for the example here – being 15:20 on 29th January 2019 via Time Travel in ez2view:
Click on image for a full-screen view.

I certainly don’t have time to explain all of what’s happening in this Very Complex dispatch interval for Price Setter in the South Australian region.

(D)  “Who’s involved in setting the price?”  is larger than just Price Setter

Readers should exercise caution, however, in not misconstruing what the “Price Setter” files mean.

For instance, they do not mean that these were the only factors involved in setting the price where it was.  I alluded to this in my comments about Category 3 above, and the example of why the price spiked in NSW unexpectedly in the 16:15 dispatch interval on Thursday 31st January 2019.

Hence please take caution to not become myopic in an obsessive scrutiny of “Price Setter” files whilst neglecting the broader factors that can all contribute to why a price outcome is what it is!

———

We’ll certainly be trying to provide that broader perspective in our Generator Report Card 2018.  Now back to the analysis for that…