No, this isn’t a guide to the week’s events in a lovely part of regional New South Wales; it’s more a deep dive into some recent – and potentially ongoing – market events highlighting yet again how some parts of a transmission system and market structure built for the 20th century aren’t coping particularly well with the 21st century transition in sources of generation.
If you’ve been observing spot market behaviour in any detail over the past month or so, you might have noticed pretty frequent occurrences of situations like this one:
That’s an ez2view market overview screenshot from 10:45am on 3 November, showing positive prices north of the Murray and negative prices in the southern mainland. Despite this price gradient, energy is flowing strongly from the higher price north into the southern regions across the Victoria – New South Wales interconnection (VNI). Why isn’t it flowing from the regions where it’s cheap to those where it’s not?
If you’d kept watching for a few more intervals, you might have seen the picture evolve to something like this:
That seems more rational (if you ignore the flows between Victoria and South Australia travelling in opposite directions across the parallel interstate links – a story for another day).
But not long afterwards, we’re back to this:
At this point you might pull out a calculator and estimate that 800-odd megawatts flowing from NSW with a price of over $100/MWh into Victoria where the price is negative $375/MWh is losing money for someone at a rate of about $380,000 per hour. What’s going on?
I’ll use this post to unpick the drivers of these outcomes, which are important not so much because they seem counter-intuitive – plenty of things in energy markets and power systems fit that description – but because they seem to be becoming endemic, and symptomatic of wider issues the NEM is currently grappling with.
A day in the life of VNI
Let’s look across the whole 288 dispatch intervals on November 3rd 2023 to get an overview of how flows between Victoria and NSW (formally referred to as the VIC1-NSW1 interconnector) behaved, and what spot prices did.
Overnight and through the evening, we see flows from Victoria into NSW at various levels up to about 1,000 MW, with Victorian prices close to but a bit below those in NSW. That seems quite normal – the way prices and inter-regional flows are meant to work in the NEM, energy will flow from the cheaper region to the more expensive, but prices will stay linked, the difference reflecting just the marginal losses incurred in transferring additional power between regions. Only once the capability to securely transfer power from the cheaper region to the dearer one has been fully utilised do we expect to see substantial price separation.
However through the daytime, things go pear-shaped, with NSW prices consistently much higher than Victoria’s which go negative, but flows consistently going – or trying to – from the higher-priced region into the lower. There are three shortish intervals where the flow seems to be throttled down to zero, before resuming its previous (mis)behaviour.
Reading the Residues
The NEM provides a useful financial diagnostic on what these flow and price patterns add up to, known as “inter-regional settlement residues”, or IRSR. This is the progressive surplus or deficit earned on transferring energy between regions, taking into account power flows, price differences and the energy losses involved in these transfers. When inter-regional flows behave as expected, taking energy from lower-priced regions to higher, positive IRSR (surplus) is accumulated, usually relatively slowly, but growing at large rates if regional prices separate significantly after flows reach secure limits.
When we see counter-price flows, as in the daytime period above, a deficit or negative IRSR accumulates – the $380,000 per hour back-of-the-envelope calculation I sketched earlier.
To quantify overall outcomes on the day illustrated earlier, I’ve supplemented the same chart with separate estimates of the IRSR earned on the flows from Victoria into NSW, and that on the flows going in the opposite direction. Here the right-hand IRSR axis measures surpluses and deficits, not the direction of the underlying flow – it just happens that the NSW to Vic direction has consistently run up losses because of the counterprice flows:
- The losses – negative IRSR – on counterprice flows from NSW into Victoria are much larger in magnitude than the positive IRSR earned on flows going the other way: -$360,000 lost vs +$40,000 earned. That’s because of the large price separation on these counterprice flows.
- The three periods where the counterprice flows were throttled – now highlighted – obviously slowed the rate at which the negative IRSR grew. These are periods where a “negative residue management” (NRM) constraint automatically kicked in and modified the dispatch outcome to force a reduction in southward transfers and slow the buildup of negative IRSR. The trigger for NRM constraints is typically a growth of ~$100,000 in accumulated negative residues – but as we can see, they only slow down the amount of negative IRSR accumulated over a day, not prevent it happening altogether.
- Keen-eyed readers might notice some growth in the positive IRSR line for Victoria to NSW flows in these throttled periods and wonder how that happens – there doesn’t seem to be any flow northwards in the chart. Good pickup – the chart shows target flows scheduled every five minutes by the NEM dispatch algorithm. The real world being more complicated than an algorithm, actual physical transfers can vary from these targets, and it’s these actual flows that IRSR gets calculated on. I’ve charted the target flows because:
- They’re typically the values shown in real-time market overview screens.
- They are products of the same scheduling algorithm that determines spot prices, and reflect the intended outcome of that algorithm – whereas actual flows also reflect forecast inaccuracies, generator deviations from dispatch targets, and other noise.
- They more clearly highlight the “throttling” periods when an NRM constraint operated.
Just for completeness, here’s a comparison of the target and actual metered flows, showing the positive daytime actual flows mentioned in the last point:
Adding up to what?
With that deviation out of the way, let’s pull back to a wider view to look at flow and price patterns for the whole quarter to date, and how IRSR has trended for two flow directions:
More surpluses are being earned (on flows from Victoria to NSW) than losses on flows in the other direction – about $14.1 million vs -$8.6 million – but the size of the negative IRSR is concerning. Remember, this isn’t really what should be happening with inter-regional transfers: we should expect to see some positive earnings from NSW sending energy to Victoria when Victoria’s prices are higher, or at least far fewer instances of counterprice flows – but that’s not the case. Almost all the southward flows we see are counterprice. We can also see from the pattern of flows that this is a daytime phenomenon. More clearly in the following chart which shows quarterly average values for flows and prices by time-of-day:
Where does Wagga fit in?
So why is this happening? Physically, it’s about the configuration of the southern NSW transmission network:
- There’s now a lot of large-scale solar generation operating in southern NSW, and much of it (as well as Origin’s gas-fired Uranquinty station) connects into the transmission system around the Wagga area or further west – here’s a map of generator locations helpfully produced by the Australian Energy Regulator
- This part of the transmission system was built with multiple high capacity 330 kilovolt (kV) lines connecting the large Snowy scheme generators at Murray, Upper Tumut and Lower Tumut into both the New South Wales and Victorian grids. This is also the main “highway” across which transfers between the two regions flow. Here’s the configuration of Transgrid’s network in southern NSW, with multiple 330kV lines (shown in orange) interconnecting the Snowy generators (and Victoria) into NSW via Yass and Canberra
- Between Wagga and those parts of NSW to its east, there’s a single 330kV line running to Lower Tumut – known as the 051 line – and a series of lower capacity 132kV links (shown in red) running towards Yass. Connecting Wagga to Victoria there is a 330kV link via Jindera, Wodonga and Dederang. When built, the 330kV loop via Wagga provided additional capacity and redundancy between NSW and Victoria, while the 132kV circuits were likely more about supply to regional NSW.
- The rapid growth of solar generation connected around Wagga (and further west), means that both the 330kV links and the 132kV network between Wagga and Yass now carry much more daytime flow than would originally have been envisaged.
- The risk in this situation is that if the relatively high capacity 051 line trips for any reason, loadings on the surrounding 132kV system will jump, potentially overloading lines or transformers on that sub-network.
- This risk grows as overall transfers from Victoria northward into NSW increase (since some of the transfer will be carried on the Jindera-Wagga-Lower Tumut route, and some fraction of that on the 132kV lines). Conversely the risk is eased if transfers are scheduled from New South Wales towards Victoria, as loadings east of Wagga will tend to reduce, and line 62 from Wagga to Jindera maintains a 330kV pathway for southward flows even if the 051 line trips.
As a consequence of all this, market dispatch has to be scheduled so that overall flows eastwards and northwards from Wagga remain within limits that are secure against the risk of a trip of the 051 line, which could overload the surrounding lower capacity network. When available low-priced generation output in the area is high, this means either (or both of):
- Curtailing some of that local generation output, or
- Reducing northward transfers out of Victoria, or even reversing these flows – which amounts to curtailing generation somewhere in Victoria instead
As with almost all transmission security risks, this is managed through constraint equations which are overlaid on the NEM’s economic dispatch algorithm, specifying levels and combinations of dispatch variables (in this case, relevant generator and inter-regional transfer targets) which are consistent with limiting the underlying flows at risk.
Culprit constraints
With a bit of digging into AEMO’s data, we can confirm that nearly all periods of counterprice flow between NSW and Victoria this quarter have been related to constraints on this specific section of the NSW transmission network. Here’s a listing of the identity of the binding constraints impacting flows during these counterprice periods, ordered by how frequently they were the limiting constraints and showing the average size of flows and difference between NSW and Victorian prices when binding:
AEMO’s constraint naming conventions may be arcane, but it’s clear enough that the 051 line features in the top four constraints in this listing as well as the sixth, accounting for over 80% of overall hours involving counterprice flows and negative IRSR.
A further point about these constraints is that most of them are so-called “system normal” or “nil outage” – they don’t involve a prior outage on some part of the transmission network which degrades its usual capability. The only exception is “N>>YSTX_TX_051” which is only invoked on outage of a 132kV/330kV transformer at the Yass switching station. An important implication of this is that these constraints will continue to regularly impact the market until the transmission capability in this area is upgraded, or something else is changed.
As far as I’m aware, there are no immediate plans afoot to upgrade this section of the NSW grid. In particular, the Energy Connect project linking South Australia to NSW will terminate at Wagga and not strengthen connections further east.
What’s to be done?
So are we likely to see these counterprice flows and build-up of significant negative settlement residues (which are the tip of a deeper iceberg – capability to transfer power east and north out of south western NSW regularly reaching its secure physical limits) continuing for years into the future? Other than physically upgrading the network, what else might change?
Market structure?
For reasons I’ve explained in other posts, inter-regional transfers are almost always impacted before local generators when both are caught in the same constraint(s). It’s important to understand that the underlying reason for this isn’t a fundamental problem with the NEM’s basic design, but reflects a structural choice overlaid on that design. This choice was to set and apply spot prices on a wide-area basis, so that all generators in a given region are paid the same spot price (adjusted only for fixed marginal loss factors) regardless of whether their output can contribute to additional supply at the margin, or whether they are caught behind one or more transmission constraints which limits their contribution to meeting demand in their home region.
With this structure, and when facing a constraint, generators with very low operational costs – such as renewables – have clear incentives to maximise their dispatch when the regional spot price is well above those costs, by offering their output at low bid prices in an attempt to be fully dispatched. This can lead to “disorderly” or “race to the floor” bidding where all potentially constrained generators end up offering at the market floor price of -$1,000/MWh.
With an alternative structure such as “nodal pricing”, spot prices would be set on a more granular, locational basis and directly reflect the value of energy at different locations on the network. Generators and loads behind a binding constraint would usually face nodal prices reflecting competing offer prices from neighbouring generators and bidding incentives would be very different. If all these generators offered at -$1,000/MWh, that’s where their nodal price would land.
We can see the effect of spot price incentives in the examples I’ve shown by observing that when these constraints bind, forcing imports into Victoria (or simply reducing northward flows), this means that marginal generators in Victoria are losing volume. They could respond by lowering their offer prices, but this would just lead to the Victorian spot price falling ever lower as generators competed for volume.
The Wagga-area generators in these examples don’t face the same risk because their low bids are not setting the NSW spot price – that is being done by generators elsewhere in NSW or Queensland who are not caught in the constraint and who can therefore contribute to meeting NSW demand at the margin.
Short of nodal pricing being introduced, something market participants have broadly and repeatedly rejected in numerous reviews instigated by AEMC, ESB and others, it’s hard to see tweaks to market structure or design which might alleviate this particular issue.
Clever controls?
A physical alternative to building more transmission might be the introduction of a “runback scheme” for some of the generators impacting the Wagga-Yass network. Recall that the underlying limitation is overloading of the lower capacity 132kV network if the 051 line trips during periods of high local generation. A runback scheme might set up the capability to rapidly and automatically curtail generation from key solar farms connected in the area, on detection of a line 051 trip. Similar schemes exist in many different parts of the NEM to deal with this form of contingency, and can be effective if the risk is of a type which allows a short time after the trip to deal with the overload – which is the case with the thermal constraints on the 132kV network. In other situations (eg voltage collapse risks), schemes triggered after the event would usually be too slow to mitigate the risk.
Let’s hope the network planners somewhere are looking hard at something along these lines, or a better alternative. Otherwise we’ll continue to see red ink in the residues.
=================================================================================================
About our Guest Author
Allan O’Neil has worked in Australia’s wholesale energy markets since their creation in the mid-1990’s, in trading, risk management, forecasting and analytical roles with major NEM electricity and gas retail and generation companies.
He is now an independent energy markets consultant, working with clients on projects across a spectrum of wholesale, retail, electricity and gas issues. You can view Allan’s LinkedIn profile here. Allan will be occasionally reviewing market events here on WattClarity Allan has also begun providing an on-site educational service covering how spot prices are set in the NEM, and other important aspects of the physical electricity market – further details here. |
As a less drastic solution than reconfiguring the NEM into a nodal system, how about enacting a higher market price floor for renewable (or inverter-based, or non-slow start…) generators? Snowy Hydro has lodged a rule change request with the AEMC to limit semi-scheduled plant to a floor of $-100: https://www.aemc.gov.au/rule-changes/addressing-access-risks-dispatchable-resources
I’m not sure if that proposal is progressing or what discussions are occurring around it, but at a glance it seems a reasonable approach to limit the impact of out-of-order bidding on interregional transfers.
In a system without any or many slow-start generators it would certainly be preferable to have a significantly higher MFP than -$1,000/MWh. Variations in offer price floors to deal with priority access are also something that ESB / AEMC has been considering – not sure where that has got to, perhaps the Snowy request has got wrapped up in that process. Others have suggested changes to threshholds for interconnectors going into the LHS of constraints. There’s a slightly ad-hoc feel to all of it.
Still waiting for somebody to explain how massively expanding wind and solar capacity, and the transmission to connect it all, is expected to reduce consumer electricity costs.
Allan’s detailed look at settlement residues and constraints is a great explainer of just one largely hidden additional cost imposed on consumers directly from the renewables push, but looking ahead 5yrs it is clear the situation must get far worse.
Surely there’s an explainer somewhere by now…
This is a great article. When I read the nsw energy infrastructure report I gained the impression that this part of the network would be upgraded via:
“South West REZ transfer capacity is delivered through ISP projects. The cost has not been included here as this capacity will be delivered
through Project EnergyConnect, HumeLink and the Wagga Wagga to Dinawan section of VNI West.”
In your opinion Allan, will these projects relieve that constraint? I note via EIS filings that there are a number of large prospective wind projects located East of Balranald that will connect to Energy Connect. Surely there must be a way for that power to get to the load?
Thanks David. As far as projects relieving the constraints, I only considered Energy Connect, as that is actually being built. Humelink and VNI West still seem way too far from breaking ground to think about as providing relief anytime soon. It’s possible EnergyConnect will worsen these specific constraints as it will bring in more flows from west of Wagga and as far as I can see do nothing to improve capability into the rest of NSW.
Thank Allan.
I personally see Humelink as a project that will be built and more or less on time. Around $600 m of early works capital is already either committed or spent and approved by the AER. In my opinion it’s something that developers and existing stakeholders will find it profitable to take a view on. If you don’t think it will happen on time there are lots of implications. If you do think it will happen (and the target date is 2026) then that’s 2.5 GW of more power that can get to Sydney/Newcastle.
You’re probably closer to it than I am then David. I just hear about lots of landowners and others arguing for it to be built anywhere but on their land or underground or not at all and wonder how it’s possibly going to happen on time, even without the construction delays and cost blowouts that seem to beset just about any major engineering project these days.
At some point in Australia’s energy future, reality and common sense will kick in and everyone of us will at that stage realise we have been saddled with the most glorious of stuff ups in our nation’s history.
That is, we will have destroyed a fully functional, sturdy, secure, reliable, and cheap means of meeting our fundamental need for huge amounts of electricity.
It’s all been predicated on the theory that man made climate change is destroying our Earth and the only way to address this concern is by purchasing and installing horrendous amounts of costly short lifespan “renewables” and batteries. We now have a privatised national electricity grid, in which hundreds of entities all expect to make fat profits via a huge and bewildering accounting system – extensively outlined above – that few people could comprehend.
How can it be that we are destroying healthy coal fired generating plant with great urgency, whilst elsewhere in the world, new long life coal fired plant is being built at a far greater rate than that at which we demolish our own.
A sensible nation (if it truly must reduce its emissions) should have been doing so by building large scale nuclear powered generators at the various sites, where coal fired generation is reaching end-of-life, and operational transmission lines and already exist.
We would be seeking to build small modular reactors at these sites where growing demand for minerals processing and other heavy industry development could be encouraged by cheap electricity prices and the minimised need for new and lengthy transmission lines.
Of course, such a national electricity grid would have been best provided by full government ownership and run by the best qualified commissioners and electrical and mechanical engineers and executives the nation could enlist.
We must presume that such a coterie of altruistic Australian professionals will always be waiting patiently in the wings.
If not, we can only shudder at the potential disaster now looming ever closer.
Interesting comments Rob. I find it bemusing that you seem to have enormous faith in technologies and technical subject matter with which you’re evidently personally comfortable (“healthy (!) coal fired generation”, large scale nuclear, small modular reactors), but prepared to dismiss in such offhand terms other technologies and equally well-grounded areas of scientific and technical expertise (“horrendous amounts of costly short lifespan “renewables” and batteries”; “theory that man made climate change is destroying the earth”; “huge and bewildering accounting system”).
I think Rob’s concerned that the efforts (by state and federal governments) to chase emissions reduction targets using our electricity system is not going to achieve any measurable emissions reduction, but will certainly deindustrialise our economy while it industrialises the countryside.
The only conceivable benefit for doing this at all is if we end up with very low cost electricity and the associated benefits.
I’d very much appreciate some insights into how you think all this is meant to result in low cost electricity for consumers. I think that’s a fair enough question.
This is only going to get worse with the introduction of the EnergyConect loop constraint
https://www.transgrid.com.au/media-publications/news-articles/more-than-meets-the-eye-to-complex-transformer-convoy
Might not be such a problem after all. Five phase shifting transformers to go in at Buronga, will allow adjustment of power flow on energyconnect independent of dispatch.
If I may reply to the above response to my earlier contribution:
My reference to “healthy coal fired generation plant” was in reference to coal fired plant (that still has many years of valuable service life), being demolished prematurely. Perhaps a bad choice of words, but we taxpayers are spending billions of dollars on renewables and the essential collateral equipment required to connect them to the grid, while we should be dragging the last bit of useful service life out of our existing plant.
Furthermore, can there be any doubt that horrendous amounts of taxpayer cash are required to facilitate the introduction of intermittent renewable generation into an already functional system and then deal with the reality that this “well grounded area of scientific expertise” comes with short life spans and more horrendous expense to cyclically rebuild it all every 20 years or so.
As for the porridge of cost and profit sharing amongst the hundreds of corporate entities now enmeshed in the grid, I defy anybody to explain how the incredibly complicated financial processes are working for the benefit of all consumers.
Similar problems exist with NSW to Victoria flows. During hot weather a few years ago, I was shocked to see ~1000MW flowing into NSW against the price. In this case, part of the problem was Melbourne’s HV network (constrained at South Morang). I suspect we need to add phase shifters or some other means of regulating real power on those 132kV parallel paths, possibly the same on the EnergyConnect loop.
As to comments that these problems are new, they are not. The supply-side issues are from the design of VNI by then state and federal governments. The big change on the demand side is population growth in the urban heat island of Western Sydney.