Not as simple as it appears – estimating curtailment of renewable generation

We’ve been asked by clients, and at the CEC’s large-scale solar webinar on 21 May this year “Maximising Value in the NEM“, how to estimate the amount and value of curtailment at wind and solar farms, and to attribute the curtailment between causes such as constraints and economic curtailment. AEMO’s Quarterly Energy Dynamics (QED) for Q1 2020 publishes estimates on curtailment but doesn’t explain these in detail, because (thanks to Jonathan Myrtle at AEMO for the explanation) the detail is quite complex, and the calculations require AEMO to make a bunch of assumptions and estimates based on their judgement to capture enough of what’s important.

The key word in estimating curtailment is “estimate”. It can only ever be an estimate, as we’ll never know exactly what would have happened had there been no curtailment, and the data isn’t always available to make the calculation. Valuing this curtailment is feasible for a plant but complex for the market, and attributing curtailment to causes is mostly straightforward, except when it isn’t!

Let’s see what we can do …

Basic concept – what is curtailment?

“Curtailment” is where a wind or solar farm’s output is reduced from the maximum it could produce in the prevailing weather conditions. This is interesting to project owners and developers (how much of my production am I losing?), and to markets as a whole (how much cheap energy are we spilling?).

The key causes of curtailment:

• Network constraints – such as a transmission line out of service, thermal or stability constraints on the network, or system strength limitations applying to specific units or types of generators.
• Economic reasons – including the spot price being negative due to oversupply, or the cost of generation too high (e.g. In Feb 2020, due to extreme FCAS contingency costs that are paid by generators pro-rata on generation)
• Commissioning – new generators must progress through “hold points”, limiting their output until AEMO is satisfied they’re ready to generate more.

A simple curtailment calculation

At its simplest, curtailment could be thought of as “what was possible to generate” less “what was actually generated”, but this isn’t quite fair. What was actually generated includes where the wind or sun dropped off of its own accord without any curtailment instruction, so may overestimate curtailment. A more defensible formulation is to look at how much curtailment was instructed – so taking what AEMO thought the plant could generate less what AEMO allowed it to generate. As a bonus this information is publicly available for semi-scheduled generators.

There’s more information on AEMO’s dispatch process (which sets the generators’ targets) here, and more detail on the semi-scheduled category of generator (which includes all recent large wind and solar farms) here. In short, every five minutes AEMO conducts the “dispatch” process, taking the bids and offers from market participants, forecasts of demand and intermittent generation, plus network constraints, and solves to find the lowest-cost dispatch solution. For a semi-scheduled generator the key numbers are:

• Availability: The forecast in MW made just before the 5-minute dispatch interval of what the plant could generate if it wasn’t subject to curtailment from AEMO. This is calculated by either AWEFS/ASEFS (more info here), taking into account current MW generation, turbines/inverters available and on-site measurement of wind/insolation, or a self-forecast provided by the generator (more info here).
• Dispatch target: In the market data, “TOTALCLEARED” – the MW that AEMO says the generator can produce in the next 5 minutes. A semi-scheduled generator only needs to comply with this if the semi-dispatch cap is set.
• Semi-dispatch cap: For a semi-scheduled generator, when its cap is 1 the generator must stay under the dispatch target. If the semi-dispatch cap isn’t 1, the dispatch process isn’t causing curtailment.

So as a starting point, calculating for each 5 minute dispatch interval when SEMIDISPATCHCAP = 1, the AVAILABILITY minus the TOTALCLEARED, gives an estimate of the curtailment imposed by the dispatch process. I discuss further down some additional sources of curtailment outside the dispatch process, generally less material than those visible in dispatch.

The calculation can be simplified by aggregating AVAILABILITY and TOTALCLEARED over a time period (such as months), and ignoring the SEMIDISPATCHCAP, as any time the TOTALCLEARED is less than AVAILABILITY (and therefore making a non-zero contribution to the curtailment calculation), the SEMIDISPATCHCAP will be 1. If valuing the lost generation, a shorter aggregation (such as a trading interval) may be needed to line up with prices.

Finding the data

The “market data” is AEMO’s EMMS, which includes the detailed results of the dispatch calculations. If you have a mirror of the MMS the table is DISPATCHLOAD. You can also look it up on NemWeb though it’s a lot of work to process these files. When aggregating, remember that each 5 minute period is 1/12 of an hour.

Users of Global-Roam’s NEM-Review and ez2view have convenient access to this data, by generator and by region.

Example – a wind farm

Here’s a screenshot example trend showing daily curtailment totals at Snowtown South wind farm, estimated in ez2view Trend Editor.

Licensed users of ez2view can download this example trend here, and NEM-review v7 users here.

Valuing the curtailed output is straightforward for a single generator. For a spot-exposed generator, take the MWh of curtailment, multiply by MLF and the RRP (per half-hour), or under a PPA substitute RRP with the PPA fixed price. The value of LGCs foregone must also be considered – including whether to value at a fixed rate or at the LGC spot price.

See the caveat below on the number-of-inverters solar constraints, where this calculation isn’t the whole story for some generators.

Regional curtailment estimates

Estimating curtailment in a region is easy enough to do if you’re happy to take just the semi-scheduled generators. The non-scheduled generation may also be material especially if there is a lot of negative pricing, and the impact of inverter-limited solar constraints also needs to be considered, as discussed further below.

NEM-Review v6, v7 and ez2view Trend Editor allow grouping by region and fuel type and will output the totals for Available and Target generation, and in NEM-Review v7 and ez2view Trend Editor you can enter calculated fields to directly calculate the curtailment.

In NEM-Review v6:

A similar grouping can be done in the Trend Editor in ez2view and NEM-Review v7.

Valuation of the spilled energy for a region is a complex question. It’s easy for a single generator to analyse foregone revenue, but if you want a regional market impact instead you need to value the substitution of low-cost wind or solar for whatever the marginal generator was. That’s far too complex a question for this article – have a look at a RIT-T if you want to see how this can be done.

How good is the data?

The estimated curtailment calculated above relies heavily on the accuracy of the forecast availability (the AVAILABILITY field). There are many reasons why this isn’t quite as it seems:

• The availability forecasts are just that – forecasts. Things can change in 5 minutes, and the forecasts are just estimates based on a model, which in some wind speed ranges can have significant uncertainty. You can hope that the unders and overs here average out. An alternative is to look at a “Possible Power” or similarly named SCADA point at the farm, which gives a “now-cast” of what the farm may have been capable of at the time, however this is still an algorithmic estimate (and not publicly available).

• Non-scheduled generators (including several large wind farms in SA and VIC) aren’t curtailed through the dispatch process by AEMO. Sometimes they are curtailed by direction from AEMO (such as during the SA-VIC separation in February 2020), by direction from the TNSP, or choose to reduce their output during periods of negative pricing. There is little data available to know what they could have done. In some QED curtailment calculations AEMO has estimated the non-scheduled curtailment using wind speed information.

• Some constraints have a curtailment effect that isn’t visible in the market systems, most notably the recent system strength constraints on the number of inverters online at some solar farms in NSW and VIC. The constraints are configured such that a farm has a limit on the number of inverters it may have in service, and if this is exceeded, the constraint will force the output to zero. To stay online the generator will, outside of the dispatch process, disable the required number of inverters. This means AEMO’s calculation of dispatch availability, which depends on the number of inverters available, underestimates the true plant availability and therefore curtailment of the farm. In the recent QED report AEMO has estimated this curtailment in an additional calculation.

• Curtailment during commissioning, where the generator’s output is limited according to AEMO hold-points, may show up as curtailment in the AVAILABILITY – TOTALCLEARED calculation, but you need to consider what this means in the context of your question – was there really “free” energy being spilt because of the time taken to carry out commissioning, or was this no different to delays in building the farm in the first place? Given the number of new farms coming online at the moment this may be a material consideration if looking at regional curtailment.

Attributing to causes is hard!

Finally, the really tough question, and one where the rabbit-holes get deeper the further you dig – what was the cause of the curtailment?

You may want to take another look at this explainer here about how the dispatch process works, to understand the complexity. A key feature of the linear-program approach used by NEMDE is that if you ask one question (what’s the optimal dispatch), you get one answer – and from this answer you can’t easily run a bunch of “what-ifs”. You’ll know how expensive a constraint was for the last MWh it constrained, but not much else.

One “simple” approach, requiring querying constraints in your MMS database, is to see for each dispatch interval if there was a constraint binding (on each generator!) and if so, label that interval as “constraint” curtailment, perhaps even in buckets by which constraint. Sometimes there’s more than one constraint binding, and splitting these out is hard, and the “what-if” outcome can only be guessed at.

The next challenge is to identify economic curtailment (when generator bidding sees them curtailed due to low prices, or to avoid high FCAS contingency costs as seen in SA in Feb 2020). One “simple” approach is to take all the intervals where there was some curtailment but no constraints binding and see if prices are unfavourable, and call this economic curtailment. This is good enough for most purposes. If you really want to know the cause of the curtailment (for example to evaluate the effectiveness of a bidding strategy against a base-case), you’ll have to consider the actual bids and their impact in each interval distinct from the impact of constraints, plus make some consideration of ramping effects, and the more you look the more complexity and anomalies you’ll find. Trust me on that one.