On the 15th December 2021, (following a major effort across a both Global-Roam and Greenview Strategic Consulting, and with assistance of others) we were pleased to officially release GenInsights21.
… in Part 2 of that 622-page report, we included 22 x Key Observations, some of which have already been discussed in articles subsequently on WattClarity®.
Almost 4 years later:
- we attended (and I spoke on 8th October at) the NEMdev conference in Brisbane, and it was a good experience.
- this was in parallel with some considerations about the draft report from the Nelson Review.
Tying both together, on Thursday 16th October 2025, we posted ‘Some suggestions for the Nelson Review panel to consider (with respect to Forecasting) following review of the Draft Report’., and in that mentioned Key Observation #6 (titled ‘What next for market modelling?’) from within GenInsights21.
So here it is … as a belated and back-dated article…
There are other types of tools used to model the electrical network in detail (the realm of PSSE and PSCAD etc) and the industry has experienced a suite of challenges.
Here, however, we focus on a different type of technology – that of the market model – which is used both:
- by individual project proponents to support major investment decisions, but also
- by market bodies and governments and other explore (and/or to support) policy changes.
We are concerned that the pace of evolution of the NEM has outpaced the ability of the evolving market models to compute genuinely valid answers:
- First and foremost, in terms of the underlying physical supply and demand balance for Energy; but also
- In adequate understanding of fuel supply limitations – be they:
- Coal pricing, and availability, as existing contracts roll off;
- Gas supply pricing and availability – including network limitations; and
- Weather (and different climate) impacts on the potential generation for hydro, wind and solar plant;
- With the plethora of ‘Keeping the Lights on Services’ that are being splintered out of the ‘Energy only’ market structure adding to the number of services that all need to be co-optimised in the modelling process.
- Not forgetting a mix of other regulatory and market overlays.
These are just some examples!
6.1 Reducing the time step in the market model
The best placed estimates seem to indicate that we collectively invested between $500M and $1B in implementing Five Minute Settlement (see Appendix 28) because of increasing demands for high response plant to receive the correct market signal.
Much was made of the way in which this faster price signal would incentivise the more responsive plant that are already being required in increasing shares (see Appendix 15).
So, it may be a shock to many readers to learn that much market modelling performed to support individual project business cases, and to produce high profile industry reports, are not done with a 5-minute timestep.
- Some are half-hourly, and
- Some (even worse) use an hourly granularity of time step.
- Others might be even more coarse than that.
How can such models hope to correctly deal with the ramp rate constraints and other transient issues that are the driver of much of the price volatility we see in the market?
The industry has collectively invested so much money to transition into Five Minute Settlement, and investors are adding millions more in new assets in the market, but (to the best knowledge of the authors) much of the modelling done in the NEM relies on 20th century, slow moving timesteps.
6.2 Co-optimisation
How many forecasting and market models do you know of that now include FCAS optimisation (which includes enablement levels/ramp rates etc)?
That’s not even factoring in the next level of complication already being observed (as described in Appendix 24 on Curtailment) including:
- Inertia and system strength (even at a very basic level); and
- Energy unserved, or foregone at different pricing levels, such as:
- At the extreme, $35,000/MWh as the Value of Customer Reliability (VCR), but
- Also $500/MWh or $1,000/MWh which might be typical of demand response (which has its own ramp rate constraints!)
Modelling the co-optimisation of Energy and FCAS, along with the requirements to concurrently model the other ‘Keeping the Lights on Services’ that are increasingly being splintered out of this Energy-Only design via this permanent schism will be the keys to understanding the complexities of the future.
6.3 More ‘roll of the dice’ iterations, because of greater uncertainty
Through Parts 3, 4 and 5 we have explored the lower aggregate level of availability (‘true availability’ or energy-constrained) across the changing generation fleet supplying the NEM.
Coal unit availability has been declining since January 2017 (Appendix 18) – a phenomenon which, due to broader commercial and society pressures, seems unlikely to be rectified. There are also concerns (not explored directly in GenInsights21) about similar issues emerging with gas fired generators that may have been relied on, historically, to run much of the time.
With either type of plant (VRE or thermal), a lower level of availability means that:
- A broader range of outcomes is possible with a different ‘roll of the dice’ even if the underlying state of the plant is the same.
- Summer 2020/21 was a mild one with a low level of system stress – because of the weather pattern that ensued.
- Summer 2019/20 was a stressful one, however, just because of a different weather pattern.
- We need not be talking about weather patterns in terms of the dice – a lower level of availability for any plant type makes it increasingly likely that there will be coincident outages across those plant type.
There are two significant implications for this, in terms of Monte-Carlo modelling that’s the basis of most sophisticated market models:
- When seeking to obtain a view of the ‘average’ outcomes, more spins of the Monte-Carlo simulation tool* will be inevitably required. When in past years 100 iterations might have been sufficient, in future years it might require 10,000 or more iterations;
>> * The article ‘ConfUSEd by the ESOO? You’re not alone’ published on 23rd August 2019 helps to explain some of the workings of Monte-Carlo modelling:
- Therefore, a lower aggregate level of firm availability means that it makes it more likely that ‘in the real world’ there is a higher chance that the dice might land on the particularly high-stress ‘High Impact Low Probability’ extreme scenarios.
- the outliers generally cause the greatest stress (and prices) on the market and power system alike.
- this is further discussed in Observation #7 immediately below.
6.4 How many reference years?
In the GRC2018 we noted that ‘The NEM is becoming increasingly dependent on the weather’ (Theme 6 in Part 2).
Questions readers should ask of their chosen modellers are around what is done to model the increasingly important weather patterns:
- A single fixed year is not suitable anymore for market modelling purposes.
- Nor is just several years (which might have been sufficient, historically, for different types of demand traces).
For instance, in Appendix 27 we discovered that even a 16-year history of wind speed proved nowhere near sufficient to rigorously model the impacts of ‘Dunkelflaute’ as a High Impact, Low Probability event that:
- Is (statistically speaking) virtually certain to occur at various points in time in the NEM out to 2050 and beyond – though we just can’t predict exactly when; but
- When it does occur, will have a massive impact on the supply/demand balance (including, but not limited to, price).
This is further discussed in Observations #y & #8 below.
6.5 Modelling ‘System Normal’ is not enough
What about modelling of constraints?
We are regularly surprised to see modelling done of the NEM on the basis that it’s only 5 nodes (and, to make matters worse, constraints on the interconnectors are only with respect to thermal limits).
- How can this be promoted as credible?
- In Appendix 27, for instance, we were very clear up-front of the great number of simplifying assumptions made in performing a preliminary study of wind diversity (which took it well away from a ‘real world’ model).
Even just factoring in dynamic inter-regional limitations, whilst a step in the right direction, is not sufficient.
The ‘best’ existing models might take a full suite of ‘System Normal’ constraint conditions (and hence better reflect curtailment of various generation plant to bring the modelled results one step closer to the ‘real world’ outcomes discussed in Appendix and Appendix 24).
But in Appendix 12 we reported just how significant ‘Outage’ constraints have been for the market – and for participants’ output, hence financial outcomes.
- If your chosen consultant is telling you ‘your preferred project won’t be constrained’ – then we suggest that you look for another consultant!
- If you are pressuring your chosen consultant to come to a similar conclusion:
- Then you only have yourself to blame (certainly not ‘Red Tape’!)
- Remember that ‘Garbage in = Garbage out’.
If you’re an investor looking to invest in a prospective project and the proponent has a similar rose-coloured glass view, or come to you with a spreadsheet model, you might want to invest your money more wisely elsewhere.
‘Real World’ reality will always highlight the holes in a quarterly cashflow spreadsheet model!
6.6 Interconnector reliability
What about the more vexed issue about interconnector reliability?
The recently released draft 2022 Integrated System Plan promotes the necessity of in excess of $10 billion of ‘urgent’ network investment to unlock the promises of wind and solar resource diversity (notwithstanding the HILP challenges of Dunkelflaute).
Most models (to our knowledge) still assume 99.98% (or even 100%) reliability of these links.
However, through the NEM’s history we have already experienced a concerning number of instances where these links have tripped and been offline:
- For a variety of different reasons
- For a variety of different durations;
- But almost always at significant cost to energy users (in terms of dollars, and/or in terms of unserved energy).
In this table below we list just some specific examples (refer to the Chronological record in Part 6 for more details of each of these events) for further details:
QLD to NSW | NSW to VIC | VIC to SA | VIC to TAS |
---|---|---|---|
25 August 2018
|
16 January 2007 (via Snowy)
7 & 8 February 2009
|
In 2005
In 2006
|
17 April 2010
|
Is it possible that the benefit of reducing (even not entirely) the risk of a region being totally islanded would justify additional interconnection?
Just remember that interconnectors, also, are not a ‘magic wand’.
6.7 Investment Decision Drivers
Given all the variability and risks we have been describing above, it was refreshing to see a LinkedIn post* … with increasing uncertainty comes the need to skew decision making in terms of increasing option value, potentially resulting in:
- in fewer multi-decade commitments and
- increasingly preference for information technology solutions (see Observation #22) over large-scale, costly ‘traditional asset’ solutions.
>>> * See Ben Skinner’s comment here, and the overall article.
What does this mean for investors in terms of all of the following:
- The way the market makes, and changes, market rules
- The way the market operates now and into the future (is the past a good indicator of the future?)
- The way we build, operate, and retire assets?
- The information systems we need to support this?
Be the first to comment on "‘What next for market modelling?’ … Key Observation #6 from GenInsights21"