Semi-scheduled generation (large-scale wind and solar) is “intermittent” by nature. Its energy availability is variable and is forecast for the dispatch, pre-dispatch and PASA processes in the NEM. Critical to an effective energy availability forecast is accurate provision of plant availability information – as first you need to know any limitations on the equipment before you can consider the wind or solar conditions. Earlier this year I wrote an article which includes some background on the relevant terminology and the forecasting processes.

Forecasting energy availability is a complex task, including getting the right information to be able to make the best forecasts, which requires co-operation between AEMO and the wind and solar farm operators. AEMO is keeping the industry informed via forums – most recently on November 30 and December 7. There’s a link to all the forum presentations on AEMO’s website. Some of the commentary in this article is responding to questions asked by AEMO at the November 30 forum.

So, what does the market need from a forecasting system, and in particular from wind and solar farm operators? First is that the plant availability information be consistent and reliable. Then, a basic requirement common to many aspects of the NEM is for transparency and traceability of this information. At Global-Roam (the owners of the WattClarity blog) we like to get right into the data to understand why things happened and to identify emerging patterns. If the data’s not there we can’t do that! In the longer term, detailed information on plant availability will help the market understand the long-term performance of wind and solar, to improve forward planning and investment decision making.

How to get consistently reliable information?

The best way to get the right answer to a question is to make sure that everyone knows what the question is and how to answer it. Participants need to know what to put where and when, including in more challenging situations.

The dispatch and short-term forward (pre-dispatch/ST-PASA) timeframes are managed very differently in AWEFS and ASEFS, which causes some confusion among participants.

• Dispatch forecasting is essential to maintaining the real-time security of the power system, including in maintaining frequency stability and reducing the call on regulation FCAS services. From the participant’s view, it’s important to get this accurate as it affects their dispatch outcome and causer-pays (regulation FCAS) costs.
• Forward forecasting of intermittent generation availability is critical to maintaining power system security across a day by ensuring enough generation self-commits or is directed to fill the gaps, and also to market efficiency by giving good price signals to slow-ramping plant and batteries.

In recent years there has been a lot of confusion among participants about how the dispatch and forward (PD/ST-PASA) forecasts are calculated and where they need to put the plant availability information. The AWEFS/ASEFS system is how it is for historical reasons, but perhaps there’s a way to unify these. I’ll come back to that in a minute. In short, this is how it works now: the dispatch forecast is based on SCADA information (element availability, wind/solar measurements, active power, local limit) or a self-forecast, and the forward (PD/ST-PASA) forecasts take plant availability information from the Intermittent Generation portal (elements unavailable, upper MW limit).

An aside: for the PD/ST-PASA information it’s good to see AEMO is working on useability improvements to the intermittent generation portal. This system is currently very tedious to use which limits how frequently a participant can update the data. In the interim, participants may like to explore the use of spreadsheets (which can be uploaded to the portal) to create their forward availability submissions. Have a look at page 23 of the Guide to Intermittent Generation. An additional improvement would be to remove the need for participants to enter data in the pre-production as well as production portals during commissioning while the AWEFS/ASEFS forecast model is being prepared, so that data does not need to be entered twice in a clunky system at the time when updates are most frequently needed.

Simple situations

In simple situations, the automatic SCADA (real-time data feed, configured at commissioning) provides the correct data for the dispatch forecast. Alternatively, a self-forecast can be provided by the participant – another whole topic of its own! The participant then (generally manually) updates the Intermittent Generation portal with planned maintenance and unplanned outages of turbines or solar inverters, and limits to the whole site such as planned substation outages, transformer issues or reactive power equipment limitations to the active power.

Challenging situations

For more complex situations it’s not so easy for the participant to get the information right. Some of these complex situations include extreme weather conditions (high wind, high temperatures, and smoke), distribution network limits and runbacks not modelled by AEMO, and the recent inverter-limit constraints.

Extreme weather

I commented on the impact of extreme weather on forecasting in an article following the previous AEMO intermittent generation forum. AEMO discussed these issues again in the late-2020 forum, making it clear that AEMO sees it as the participant’s responsibility to update its plant availability during extreme weather as the participant is better placed to know the effects on their plant. Many participants will need to develop complex internal processes and tools to meet this expectation, but I don’t think it’s currently clear to participants what’s expected of them, nor how to best enter the information. While AEMO has offered to talk through the particular complexities with each participant individually (contact the Operational Forecasting team), it would be valuable for AEMO to publish specific examples of extreme weather with their expectation for participant submissions, particularly in how the information would look in the SCADA and the intermittent generation portal. It’s not productive to have participants trying to reverse-engineer what AWEFS/ASEFS will do in each situation, and there are some situations where the elements available / plant limit model doesn’t seem to fit. The particular situations of interest are high-wind (affecting wind farms), high-temperature effects (affecting wind through turbine cut-out and de-rating, and solar through a reduction in efficiency and inverter cut-out), and smoke effects (affecting solar). There are many other complexities including wind sector management, where the operator shuts down some turbines to improve the overall outcome in certain wind conditions.

When the impact on the site is either a limit to total output (e.g. the transformer total output is reduced), or a number of turbines/inverters are anticipated to shut down, this is straightforward to enter. It’s more complex to accurately convey a de-rating effect or a reduction in efficiency, where some parameters of the situation that don’t neatly fit the elements available / plant limit model need to be shoehorned in.

• De-rating is where the maximum output of a part of the system is reduced. This could apply to transformers or to individual turbines/inverters. In the case where all turbine/inverter elements are available the maximum possible output from the site can be easily calculated as the de-rating limit times the number of elements, and this limit set on the Local Limit SCADA and the Upper MW Limit in the portal. This doesn’t work so well when there are also a number of elements out of service or becoming unavailable due to the weather conditions. A complex technical solution could be to calculate the limit (potentially automatically in the case of the Local Limit SCADA) to account for the number of elements available and their de-rating limit.
• A reduction in plant efficiency, such as for solar panels on extreme heat days or when conditions are smoky, is difficult to fit into the elements available / plant limit model. What’s really happening is that the power curve has shifted down. A similar result could be achieved by pretending some inverters are out of service, and in the middle of the day scaling down the plant limit would work, but this is starting to muddle up plant and energy availability.

Distribution limits

AEMO has recently resolved a legal question about whether distribution network limits that aren’t modelled by AEMO can be included in the AWEFS/ASEFS forecasts (I believe this concerned NER 3.7B(c)(6)), which now gives clarity to AEMO and participants that distribution limits can explicitly be included in the availability information provided, a big step forward. There remains some complexity for participants in knowing which limits are modelled by AEMO and which aren’t, and if the limit isn’t a simple MW level then it’s difficult for participants to forecast this in their Intermittent Generation portal submissions. Runback schemes can also be quite complex. If it’s difficult (or impossible) for the participant to forecast what the distribution limit or runback might do then there could be inaccuracy in the PD/ST-PASA forecasts, which could be a problem in the aggregate if multiple sites have similar limitations.

Inverter limit constraints

In the last year or so some new constraint formulations have popped up from AEMO with limits on the number of inverters at a site that can be in service in certain conditions. These constraints affect the system not by binding as a typical constraint would, but by driving the operator to switch the inverters off (making them appear physically unavailable) when the constraint is invoked. This is fine in dispatch for AWEFS/ASEFS as the elements available SCADA count reflects what’s available at the time. In PD/STPASA this appears to be more complex. Is the participant expected to anticipate when the constraints will be invoked and correspondingly reduce the inverter count in the Intermittent Generation portal for PD/STPASA to prevent AWEFS/ASEFS from over-estimating the energy availability? One way to manage this could be to have a PD/ST-PASA version of the constraint that was a MW limit instead of an inverter count, but this would only be correct at maximum wind/solar conditions. There are often many sites affected by similar constraints at the same time so this could add up to a big offset in the forward forecasts if not handled well.

Bidding of maximum availability

One improvement to the system that would make it easier for participants to provide accurate information in challenging situations is to make use of the MAXAVAIL field in the bid file, which scheduled generators use to indicate their maximum MW availability for that interval. The AWEFS system was designed to override the MAXAVAIL field with its forecast of energy availability. Participants can enter a value in the MAXAVAIL field but it’s ignored. At the recent Intermittent Generation Forum AEMO raised again the question of whether there was value in enabling the MAXAVAIL field as a lower limit on the availability of the generator.

There are several good reasons why this should be done, at least in dispatch (I’ll look later at other timescales):

• To provide an easy place for the participant to communicate a limit on plant availability to AEMO, particularly in unusual circumstances. At the recent forum AEMO discussed a number of situations where the participant should communicate their capability for dispatch by updating their Local Limit SCADA signal, which has the effect of capping the AWEFS/ASEFS dispatch forecast. The Local Limit SCADA signal is part of an industrial control system, designed for automatic responses – such as if a transformer trips, or reactive plant fails, or other known limits apply that can be coded into the SCADA signal logic. What it’s not as good for is coding unforeseen or less well-understood limits such as high-temperature de-rating where human intervention is likely required. Compared to manually entering a limit in the MAXAVAIL (which involves updating the bid in the AEMO bid portal), manually updating the Local Limit SCADA requires logging into a vendor-specific interface, possibly in a segregated part of the company’s IT/OT network, to change the value of the signal so it can be fed back into AWEFS/ASEFS (through the industrial SCADA system via the TNSP), to be applied by AWEFS/ASEFS to limit the dispatch forecast in the bid – quite a roundabout route just to provide one human-created number to AEMO. It’s also not a trivial matter for most participants with outsourced site operations and maintenance to update the automated calculation of the Local Limit SCADA signal as new situations come to light.
• A rebid of MAXAVAIL would require a rebid reason, which is good for commercial transparency. Back in 2016 when the Local Limit SCADA was first introduced the AER noted in its responses to the Energy Conversion Model consultation that it saw benefit in the increased commercial transparency that would come from a bid MAXAVAIL.
• Finally, a timely benefit with the requirement for participants to communicate high-temperature effects to AEMO is that participants could enter in one bid submission an estimated maximum availability profile for several hours ahead. This would be of particular benefit to operators of multiple sites, who could put in estimated plant limits for each site and then monitor these and correct as needed, compared to the Local Limit SCADA approach which would require logging into each SCADA system to instantaneously change the limit for each site, potentially numerous times across a hot afternoon. As well as streamlining human intervention, entry of such limits via the bid file would enable the use of automated bidding systems to manage high-temperature days, and the implementation of improved algorithms for predicting the high-temperature effects.

Perhaps there’s a different way?

The idea of entering a profile for MAXAVAIL across a day raises another question – what effect would the MAXAVAIL field have in the pre-dispatch and ST-PASA timeframes?  Consistency of the NEMDE calculations would suggest the MAXAVAIL be applied as a limit in PD and ST-PASA as well, which begs the question of what the Upper MW Limit field in the Intermittent Generation portal would then be for.

Perhaps there’s a different way. What if the bid file had an extra column for “elements available”, and the Intermittent Generation portal (the system where currently the PD and ST-PASA timeframe plant availability is entered) was taken out of the process? The SCADA values could still be incorporated in real-time.

There would be several benefits from such an approach:

• Ending the need to submit the PD/ST-PASA availability information into a separate (currently clunky) system.
• Ending the confusion by participants on which values apply to dispatch and which to PD/ST-PASA.
• Motivating participants to fix up the MAXAVAIL if it changed as real-time approached. Currently, apart from potential compliance issues, participants have no motivation (other than a call from the AEMO control room seeking to correct a pre-dispatch error) to fix up an incorrect Upper MW Limit submission in the Intermittent Generation portal as it has no effect in dispatch.

Transparency and traceability

Why is transparency of information (and the related traceability, knowing where a value came from) important?

Short to medium term

In the short to medium term, transparency of information used to prepare energy availability forecasts allows participants to see what other participants are doing (increasing understanding and motivating compliance, and increasing market efficiency), and allows external analysis (such as in the Generator Statistical Digest and Generator Report Card) to assess behaviour, compare performance between generators, and identify emerging patterns.

AEMO has announced that it will be publishing next-day the PD/ST-PASA plant availability submissions from May 2021. Here at Global-Roam we’ll be taking a close look at this data, as we expect it’ll greatly improve our analysis of events where generation dropped away or rose unexpectedly. In the MT-PASA timeframe a recent rule change means for scheduled units that their plant availability is public 3 years out (ending the efforts of many to reverse-engineer the aggregated data), but unfortunately semi-scheduled were omitted.

Traceability is also important to ensure the information is understood, such as when there are multiple factors involved that may mask each other. A great example of this is in the publication of the dispatch self-forecast data, where AEMO publishes next-day the AWEFS/ASEFS forecast, all of the self-forecasts submitted, and which forecast was used in NEMDE and why.

At the recent Intermittent Generator forum AEMO sought feedback on the publication of actual plant availability from SCADA.

I see the publication of SCADA plant availability data as the natural complement to the publication of the PD/ST-PASA availability data. Its publication would allow increased understanding of the derivation of the dispatch availability and would allow assessment of the accuracy of the PD/ST-PASA plant availability submissions.

In response to AEMO’s questions:

• While the exact values for the SCADA measures used within AWEFS/ASEFS may not be visible (and so the effect of validation applied by AWEFS/ASEFS may be hidden), a very adequate measure to understand how the forecast was made would be to publish the average value of the SCADA signals taken over the third minute of the dispatch interval as is input to AWEFS/ASEFS from the PI system. A further measure that would be useful in understanding the actual generation would be a snapshot of the SCADA signals taken at a similar time to the INITIALMW snapshot recorded in the MMS. The SCADA quality indication (good/bad etc, as recorded in PI) would also be of value for cleaning up the data during analysis.
• What the participant’s self-forecasting system does with plant availability is its own concern, but even with self-forecasting the participant is required to provide correct Turbines/Inverters Available and Local Limit SCADA signals to AEMO. This is what AEMO should publish. Inconsistency identified between these SCADA signals and the self-forecast would indicate the fall-back AWEFS/ASEFS system is being fed incorrect data.
• While the implementation of Turbines/Inverters Available and Local Limit SCADA signals may differ slightly across OEMs, the application of these signals in AWEFS and ASEFS remains consistent, and so the data should be useable for analysis. There may be some variations, such as whether an all-off outage is shown as all turbines/inverters unavailable, a zero MW plant limit, or both. While creating some complexity for analysis, this shouldn’t materially reduce the useability of the data. Other variations may arise particularly in the challenging situations discussed earlier in this article, where the variation in implementation will be of interest in itself.

Long term

In the long term, transparent plant availability information tells the market how the plants are ageing, allowing analysis of failure rates and extrapolation of long-term generation capacity. Publication of plant availability data serves this purpose best if the data is as close to the physical plant availability as possible. It won’t serve this purpose if a) physical plant availability is masked by other issues such as distribution network limits or inverter-count constraints and b) these reasons aren’t communicated (e.g. by a rebid reason). One related example of this masking is the impact of the inverter-count constraints on the calculation of the apparently simple measure of curtailment based on energy availability, as I explored in an earlier article.