What inputs and processes determine a semi-scheduled unit’s availability

The semi-scheduled generating unit’s availability, in megawatts, is the upper limit for its power output used in dispatch.

It is indispensable to understand how it gets produced to optimally manage the critical inputs and comprehend dispatch outcomes.

We capture the key inputs and processes in the upcoming diagram, focusing on the dispatch timeframe.

Background concepts

In order to most effectively digest the diagram we will quickly note three fundamental concepts:

a)      A unit’s availability limits the dispatch target

A unit’s maximum output that can be delivered, at the end of an interval, is the availability.

The dispatch engine (NEMDE) requires this information to prevent setting output targets that can’t be met.  This applies to all Scheduled and Semi-Scheduled units.

b)      A variable renewable resource means variable output

It’s no secret that winds and solar irradiances are variable and these impact the unit’s power output. Because of this, the upper limit of output (the energy availability) is not firm at a predetermined level – it needs to be forecast.

In the NEM the forecast is called the Unconstrained Intermittent Generation Forecast (UIGF).

And because the UIGF represents the upper limit of output it can be adopted as the unit’s availability. Availabilities are needed for each interval in each timeframe that the market operator calculates or projects dispatch.

It’s important these UIGFs are as accurate as possible as any net shortfall or overestimate in unit availability can lead to a mismatch between demand and supply and costs to correct the difference.

c)      The dispatch timeframe is where the rubber hits the road

The dispatch timeframe is the time between now and the end of the very next 5-minute dispatch interval.

The dispatch run, for the dispatch timeframe, produces the actual dispatch schedule. This where each generating unit’s target is their actual instruction for generating into the power system. It’s for these reasons, and the role participants can play, that this article focuses on availability in the dispatch timeframe.

Yet longer outlook timeframes require UIGFs and unit availability:

• The 5-minute predispatch (5MPD … also known as P5) timeframe extends from now, out 12 5-minute intervals into the future.
• The predispatch timeframe (PD … also known as P30) extends from now to the end of the next trading day (set at midday) and is at 30-minute granularity.
• The ST-PASA timeframe (ST) extends out 7 days and refers to the ‘Short Term Projected Assessment of System Adequacy’ at 30-minute granularity.

This article primarily relates to the dispatch timeframe … there are some differences to the diagram below for the longer time-frames.

What inputs and processes determine a semi-scheduled unit’s availability?

For the dispatch timeframe, the following diagram summarises the inputs and processes that determine the availability as used by NEMDE, the dispatch engine.

What Inputs and Processes Determine a Semi-Scheduled Unit’s Availability (click image for higher resolution diagram).

The UIGF is the availability, unless…

Jumping straight to the end of the diagram (section 4), we can see the UIGF is the availability used by the dispatch engine unless the participant has issued a bid with a lower MaxAvail.  This change occurred very recently … on Monday 7th August 2023 (as noted here at the time).

The bid MaxAvail for semi-scheduled units can be thought of as representing maximum commercial availability.

And where does the UIGF come from?

The start of the diagram begins to answer this from the perspective of data inputs (section 1).

The data feeds into UIGF processes to produce UIGFs.

Data inputs used to produce the UIGF, and determine availability, are site-specific

At the top of the diagram, at section 1, we summarise the data inputs into three groups:

• Site information (1.1): This is used by downstream UIGF processes.
• Additional inputs (1.2): These may be used by a Participant’s UIGF process in conjunction with a self-forecast provider. May be more than one provider. More on the participant’s UIGF process later.
• Market inputs (1.3): The energy bid. Not specifically for the UIGF, but for capping availability to reflect commercial availability.

Site information is the primary input for the dispatch timeframe

Weather information for UIGFs in the dispatch (and 5MPD) timeframe comes from site measurements, sent as SCADA signals. These site measurements include wind speeds and solar irradiances.

• In PD and ST timeframes weather information used to produce the forecast comes from one or more numerical weather forecast providers.

Availability information, sent as SCADA signals, captures physical aspects including how many turbines or inverters are available, active power output, turbines in extreme wind cut-out state.

• In PD and ST timeframes turbines/inverters available and upper MW limit information is used and is provided by the participant through the Intermittent Generation Availability Information Portal.
• In the 5MPD timeframe the PD forecast is used as an input into the 5MPD forecast so, indirectly, availability information from the portal is applied to the 5MPD forecast.

Site standing data is used to build forecast models. This includes the data that forms the energy conversion model (ECM) – it defines how the intermittent input energy source (such as wind or solar irradiance) is converted by the unit into electrical power output.

Local Limits and set points are needed as inputs to the forecast process. Network outages or distribution network limits that are not managed in central dispatch (i.e., through constraints) should be reflected in the Local Limit signal to ensure dispatch does not exceed those limits.

• In PD and ST timeframes the upper MW limit submitted through the Intermittent Generation Availability Information Portal can be used to reflect local limits.
• In the 5MPD timeframe the PD forecast is used as an input into the 5MPD forecast so, indirectly, upper MW limit information from the portal is applied to the 5MPD forecast.

Further information on AEMO requirements for data inputs to the AEMO UIGF process is available in the policy and process documentation webpage for forecasting at AEMO.

Additional inputs

Different weather or additional information might be acquired for use in a Participant UIGF process and can be sourced from different providers. These may include different on-site weather measurements that support improved forecasts.

• Self-forecasting is not offered for 5MPD, PD and ST timeframes.

Further information on participant self-forecasting is available from the Participant forecasting webpage at AEMO.

Market inputs

The Market Input that is used for dispatch availability is the MaxAvail value from the energy bid, which plays a role as an input to NEMDE (diagram section 3.4).

• In 5MPD, PD and ST timeframes bid Max Avail is also used to cap the UIGF for availability in NEMDE.

The UIGF Process is not only AEMO in the dispatch timeframe

AEMO is obliged to produce the UIGF forecast in accordance with the Rules (diagram section 2.1).

In 2019 AEMO introduced the opportunity for participants to provide their own forecast, the “self-forecast”, following a successful trial in 2018.

This created a second channel (diagram section 2.2) through which a UIGF may be produced.

The AEMO UIGF Process

The AEMO UIGF process (diagram section 2.1) will run at least once per dispatch interval, usually more than once, to produce many forecasts per interval.

The forecast approach used depends on model readiness and whether the input data is of sufficient quality. The SCADA signals have quality flags that indicate if the signal is good or bad.

There are, presently, four ‘origins’ used to denote the source of the UIGF:

1. The primary forecast approach is AWEFS (for wind) or ASEFS (for solar).
2. For the dispatch timeframe simplified forecasts will be produced under the ‘FCST’ origin label if there haven’t yet been enough data points to adequately train the AWEFS/ASEFS model.
3. A persistence-based forecast may be used when more sophisticated approaches are not possible (e.g. missing critical SCADA signals). The power output at the start of the interval (from SCADA) may be used as the interval-end estimate and labelled ‘SCADA’.
4. In other circumstances the last target (energy dispatch target) may need to be used as the next UIGF.

The Participant UIGF process

The Participant UIGF process (diagram section 2.2) if enabled produces “self-forecasts”.

There are multiple self-forecasting providers operating and more than one may be used by a participant for their unit. A priority is provided with each forecast to differentiate the preferred one.

Similarly, there are multiple weather data providers that may provide inputs to the forecasting process, or these weather services may be integrated into the forecast provider’s service.

The participant may decide to suppress its forecast if it should not be considered for use in dispatch.

Over the years WattClarity has covered many aspects of self-forecasting in prior articles. Here’s a sample:

• Insights on self-forecasting from analysis of Q4 2022
• The increasing role of self-forecasting and auto-bidding in the modern day NEM
• And there’s many more collated under the self-forecasting topic

NEMDE Inputs require a UIGF to be selected from many in addition to the bid MaxAvail

There may be multiple UIGFs made in the leadup to the start of the dispatch interval:

• Earlier forecasts might be made to ensure a forecast exists.
• Later forecasts aim to capture more recent information.
• There may be self-forecasts (diagram section 3.2) and AEMO forecasts (diagram section 3.1).
• Self-forecasts may be suppressed by AEMO (not to be used in dispatch).

The UIGF sent to NEMDE (diagram section 3.3) is determined from many candidate forecasts.

Self-forecasts can be eligible for use if they are not Participant-suppressed and if they are not AEMO-suppressed (diagram section 3.2).

AEMO suppression refers to the process AEMO applies to ensure the quality of the self-forecast is equal-to or better than AEMO’s. We discuss self-forecasting performance in the article ‘Delving Deeper into Dispatch Availability Self-Forecasting Performance‘ in April 2023.

Preference is given to the self-forecast for use in dispatch:

• AEMO applies a set of principles to determine which of participant’s self-forecasts should be used. This set includes the priority number, suppression state and whether it is received before gate closure.
• If there is no eligible participant self-forecast the AEMO forecast is used after applying the same set of principles on the AEMO forecasts candidate.

The energy bid’s MaxAvail (diagram section 3.4) is also provided to NEMDE.

Some data is published, but not always in real-time

At this point in the diagram (diagram section 3, and also section 1) we see yellow boxes indicating data that is published in the electricity market management system database (MMS).

Access depends on whether it is public or private to a Participant.

All the UIGF, bid MaxAvail and availability data at unit-level, for the dispatch timeframe, is “private, next-day public”.

This means the data for the trading day 4am-4am is published at the start of the next trading day, for public access. It is available in real-time to Participants with private access (the data relevant to their own units).

Where to visualise: ez2view’s Unit Dashboard [this article contains examples] captures much of the unit-level availability information. Aggregate level information can be viewed through ez2view user-defined Trends either using ez2iew (installed application) or ez2view online.

Nonetheless, if public data is what you use, at NEM-Region level the aggregate semi-scheduled wind and solar UIGF and availability is public in real-time.

The availability used by NEMDE accounts for the bid

As we discussed at the start of the article, the UIGF is adopted as the availability unless the bid MaxAvail is lower. In that case the availability is capped to the MaxAvail level.

Use of bid MaxAvail to cap availability was introduced on 7 August 2023.

• Prior to this, the UIGF was adopted as-is.
• We covered further details on this change in several articles, including:
  • ‘An Imminent improvement to the bidding and dispatch process for Semi-Scheduled units‘, and
  • ‘AEMO’s MMS 5.2, ez2view v9.7, Semi-Scheduled MaxAvail in the Bid, and a self-forecasting example‘.

Recap

Acknowledging different readers will have diverse applications for the information, here are four key takeaways that are likely worth consideration:

1. The energy bid MaxAvail now acts as a cap on the UIGF sent to NEMDE for a semi-scheduled unit’s availability.
2. The UIGF (and there could be many per interval) can come from two channels;
   • AEMO or
   • one or more participant self-forecast providers (but self-forecasting is only offered for the dispatch timeframe).
3. Site-based measurements are key inputs to the UIGF process in the dispatch timeframe. Numerical weather forecasts from weather forecast providers are used in the for PD and ST timeframes.
4. Plant limits or local limits are sent through as SCADA signals for the UIGF in the dispatch timeframe. These need to be included in the UIGF alongside the weather resource because they aren’t accounted-for elsewhere in central dispatch.

PS1

As of 25 August 2023 we’ve made the “AEMO dispatch UIGF” (probability of exceedance 50%) available as a Trend series (customers  of ez2view, NEMreview online). This concerns section 3.3* in the diagram.  The value is selected as the most recent, and then highest priority, UIGF from the many AEMO UIGF process-forecasts issued each dispatch interval. In cases where the self-forecast is the one sent to NEMDE (section 3.3 of the diagram) this “AEMO Dispatch UIGF” provides the alternate and independent view of the UIGF.