ANALYTECON

The subject for the next few blogs will be prices and wind generation in South Australia. South Australia may foreshadow issues for all NEM regions that are planning to significantly increase wind generation capacity. At the same time, it is important not to just focus on the intermittency of wind generation in South Australia, wind needs to be place into the context of the electricity system.

The relationship between the volatility of wind generation and wholesale electricity prices is hypothesised to revolve about three key factors. These are the level of wind generation capacity relative to:

·      Electricity demand

·      Local fast start generation capacity, and

·      Constraints on imports and exports of power between South Australia and     Victoria. 

The first point is taken up later in this blog. However, it is the difference between electricity demand and wind generation, referred to as residual demand, that needs to be managed, for the most part, through the normal dispatch system. Fast start generation provides the dispatch system the flexibility to respond changes in and generation and load within a short time frame. Open cycle generally has the shortest start up times and fastest ramp rates (the rate at which output can be adjusted). South Australia’s open cycle gas turbine capacity is currently about 870MW, representing a bit less than 60 per cent of average demand and 30 per cent of peak demand.

Third, interconnections allow AEMO to dispatch power from other NEM regions to South Australia.  The main link between South Australia and Victoria is the Heywood interconnector, which was sequentially upgraded in 2016 from 460MW to 650MW and energised in July 2016.  The constraint on the exchange of electricity between regions is a system level issue, and often reflects constraints elsewhere in the network or system security considerations (rather than the capacity of an interconnector per se). However, there are two key points from a South Austrian perspective when imports of energy from Victoria are constrained: 

·      Variation in residual demand needs to be managed through the dispatch of local generation; and  

·      The market becomes separated and local prices can diverge from Victoria and other NEM regions.

After the South Australian blackout in September 2016, AEMO have reduced transfer limits across Heywood. Presumably these constraints have been imposed to increase system security. However, in principle, this will lead to the South Austrian market becoming separated more frequently adding to the task of managing variability.  However, when the South Austrian market becomes separated it creates the opportunity to explore the impact of having a high proportion of demand met by local wind generation.

Share of Operational Demand Met by Wind

The metric we are going to look is the share of half hourly operational demand met by wind generation from 1 September 2015 to 16 January 2017 (the period of the 2016 blackout has again been excluded).  The smoothed trend in the share of operational demand met by wind is shown in Figure 20. The most striking point is the sharp increase in the share of wind in the second quarter of 2016. This is due to several factors including seasonal variation in wind generation, the closure of the Northern coal fired station and additional wind generation capacity being brought on line.

Figure 20 The smoothed trend in the share of operational demand met by wind in South Australia: 1 September 2015 to 16 January 2017

 What is more interesting is the reliability wind in meeting this increased proportion of demand, particularly when demand is high, as for example peaks in summer and winter. What would like to see is how the volatility of the share of operational demand met by wind is changing over time, at time scale that matches up with the problem of managing intermittent wind generation. A bit of work needs to be done.

First, the standard deviation of the change in the share of operational demand met by wind is calculated at successively longer time scales; over a half hour, over an hour and so forth.  Longer time scales can be created by taking the difference of averages, as for example, the difference in hourly, daily and weekly averages. However, this dampens temporal correlation and variability.  Longer time scales can be created by taking successively longer differences in the share of operational demand met by wind without averaging, say from 9:00 to 9:30, 9:00 to 10:00, 9:00 to 11:00 and so forth. This preserves temporal correlation and it is temporal correlation that gives rise to large swings in wind generation and demand. This raw variability, which is still tied to a five-minute interval, is a measure of how much things can change over a given length of time.    

The change variability over longer time scales is shown in Figure 21.  The variability in the share of operational demand met by wind increases but at a decreasing rate. For instance, there is a sharp increase in variability over the first four hours but this progressively slows and eventually plateaus at 18 hours. After 18 hours, the temporal correlation in the change in share of operational demand met by wind is essential gone.  We can use this measure of variability to calibrate a useful picture of how volatility has been changing over time.

Again, a two-stage smooth is used to extract the trend in volatility over time, this time calibrating the smooths to approximate the volatility to a time scale of about two hours. This is done by matching standard deviation of smoothed volatility to about 10 per cent which corresponds to about two hours in Figure 21. The motivation is to try and get a picture of how the scope of problem of managing intermittent generation has changed in South Australia. 

Figure 21. The standard deviation in the share of operational demand met by wind generation over successive time periods in South Australia: 1 September 2015 to 16 January 2017

 The first stage smooth, which takes into account the expected pattern of diurnal variation, can be interpreted as the mean or expected share at a point in time. The deviations from trend represent volatility or uncertainty about the expected level demand met by wind. This uncertainty is managed through capacity to adjust the output of dispatch from generators at the same time scale as the volatility.  The trend in volatility is obtained by taking successively finer smooths of the deviations from trend until their variability matches the variability of the share of operational demand met by wind over two hours (as shown in Figure 21).  Over this time span the flexibility to manage variability comes from generators that are online that can adjust output, fast start new generators that can be brought on line and the ability to import and export energy to Victoria. 

 The result, which is intended to be illustrative, is shown in Figure 22. The dark orange bands show expected diurnal pattern in the share of operational demand met by wind. The pattern within the band is constant over each 24 hour period and is repeated so frequently that it fills the band with a solid colour that has been made translucent. The edges of the band are the upper and lower bounds of the expected diurnal variation. The overall pattern jumps sharply toward the end of the first half of 2016 as was seen in figure 20.

 The light orange line is the deviation about the diurnal pattern and is the volatility or the uncertainty that needs to be managed. For the most part, volatility falls within the band of diurnal variation but there are spikes that lie well outside these bounds and they occur quite frequently. These spikes are ramp like events are created, for the most part, by irregular cycles in wind speed that were shown in the previous blog. The time from peak to trough would be the order of four hours. The escalation of uncertainty in the second half of 2016 is quite clear, particularly from late June into September, where the magnitude and the frequency of the peaks and troughs are sharply elevated.   

Figure 22 The expected diurnal pattern and the volatility of the share of operational demand met by wind: 1 September 2016 to 16 January 2017

The light orange line is the deviation about the diurnal pattern and is the volatility or the uncertainty that needs to be managed. For the most part, volatility falls within the band of diurnal variation but there are spikes that lie well outside these bounds and they occur quite frequently. These spikes are ramp like events are created, for the most part, by irregular cycles in wind speed that were shown in the previous blog. The time from peak to trough would be the order of four hours. The escalation of uncertainty in the second half of 2016 is quite clear, particularly from late June into September, where the magnitude and the frequency of the peaks and troughs are sharply elevated.   

Wind generation in South Australia was a primary driver of the volatility of residual demand over the winter month in 2016. However, the relationship between demand and wind also contributed.  Demand and wind generation output were negatively correlated. That is, higher than average demand is associated with lower than average wind generation. Over the entire period this correlation was only about 12 per cent but this jumped to 25 per cent in July and August of 2016.

Meeting demand with a high proportion of wind generation is, as hypothesis at start of this blog suggested, one factor that is likely to be a source of increased price volatility in South Australia.  Other factors may be a prerequisite or of equal or greater importance.  Infrastructure is brought into the picture in the next blog

We start by looking at some general characteristics of wind generation in New South Wales, South Australia and Victoria. The AEMO data are actual generation for semi-scheduled wind farms in each state at five minute intervals from September 1 2015 to 16 January 2017. Data for the 28^th through the 30^th of September 2016 were removed due to the South Australian blackout.

The data are then averaged over half hourly intervals and the standard deviation in wind generation was calculated for each interval. The standard deviation over six five minute intervals is being used as a measure of variability that occurred over half an hour, as opposed to a statistical measure of the expected dispersion of possible outcomes within that half hour. The data were then smoothed to show how wind generation evolves over time.

The smooth of wind generation is shown in Figure 14. The time span over which AEMO wind generation is available online is limited, so the smooth is over the full data range.  Consequently, the beginning and the end of the smooths should be regarded with caution. Nevertheless, the similarity of the seasonal pattern in each state is very clear. The differences in wind generation in each region are due, in large part, to installed capacity. Further, one additional generator came on line in South Australia in late June 2016, adding 102 MW of capacity which is also evident in the figure.

The smoothed standard deviation of wind generation in each half hour interval is shown in Figure 15. Again, the seasonal pattern is similar in each region. The greater variability of five-minute wind generation in South Australia is consistent with the greater level of wind generation and a positive correlation in output from wind farms. The average correlation across South Austrian wind farms over the period was 58 per cent with a range of 18 to 94 per cent. Although wind has a strong seasonal component, the increase in the volatility of wind generation the second half of 2016 in South Australia may again be consistent with the increase in capacity that occurred over this time.

These smoothed trends provide a picture of how the profiles of wind generation have changed over time and help to explain why wind generation, at five minute intervals, is correlated across New South Wales, South Australia and Victoria, as shown below:

·      SA – NSW 29%

·      SA – VIC   54%

·      NSW-VIC  48%

The variability of wind generation, which may change over half hourly and longer time scales, can only be anticipated and managed in a probabilistic way. From a non-technical perspective, there are two elements that shape the management problem.  First, there is less flexibility to respond the shorter the time scale.  Second, most the deviations about what is expected may be small and managed at a low cost while more extreme outcomes will be infrequent but costlier to accommodate. To get a sense of this we should look at wind generation from a distributional perspective. While interest is in volatility will start with the distribution of the level of generation in each region.

Figure 14. The trend in half hourly wind generation in New South Wales, South Australia and Victoria: September 1 2015 to 16 January 2017

Figure 15. The standard deviation of half hourly wind generation in New South Wales, South Australia and Victoria: five minute intervals: September 1 2015 to 16 January 2017.

The distribution of wind generation in each state is shown in Figure 16. The distributions are all strongly skewed to the right with a fat tail. So, while low and even zero levels of output quite common there still are relatively frequent high levels of generation. South Australia has a longer and fatter tail. The former is due to greater capacity but the fatter tail is likely to reflect a difference the wind resource. New South Wales has a substantial number of zero generation intervals and Victoria more low generation intervals when compared to South Australia.

Figure 16. The distribution of the change wind generation in New South Wales, South Australia and Victoria: September 1 2015 to 16 January 2017.

Turning to time scale of volatility, we can compare the in the change in wind generation between five and 30 minute intervals.  The distribution of the change between five minute intervals is shown in Figure 17 and for 30 minute intervals in Figure 18. The spread of both the distributions is considerably wider (less peaked) in South Australia compared to Victoria which in turn is substantially wider than in New South Wales. While the differences may be mostly due to installed capacity, the wind resource also seem to play a role particularly between New South Wales and Victoria.

Figure 17. The distribution of the change in wind generation, delta, between five minute intervals New South Wales, South Australia and Victoria: September 1 2015 to 16 January 2017.

In moving from 5 to 30 minute intervals the change in the distributions is also quite striking. The range roughly doubles and the tails due fatten, especially in South Austria. This is due to the positive temporal correlation in the changes in generation through the half hour intervals. In other words, the five-minute changes tend to be more strongly negative or positive as opposed to random with the half hour.

The implications are a little easier to see in Table 3 in which the upper percentiles of the distributions are shown (the distributions are quite symmetric so the lower percentiles will roughly mirrored, the 1^st per centile the opposite in sign of the 99th). In South Australia 10 per cent of the variation between five minute intervals was over 15MW. This more than triples to 48 MW between 30 minute intervals. This threefold increase is consistent through the upper percentiles in South Australia and there is a similar though smaller pattern of increase in the other regions.

The NEM maintains ancillary generation services to continuously manage very short term fluctuations in generation and load, referred to as regulation frequency control. Regulation is required because there is not sufficient time to adjust output through the dispatch of price and energy bids from generators. However, a lower level of variability can make this essential service more manageable and less costly to deliver. The converse is also true, as the level of reliability of the service needs to maintained.

The median supply of regulation frequency control in South Australia was 60MW which is more than the 99.5 per cent of variation in half hourly wind generation. The supplies of these services can be adjusted to meet anticipated increases in variability in generation and load. For example, in 20 per cent of the same period, the supply of regulation frequency control was over 100MW.

Figure 18. The distribution of the change in wind generation, delta, between 30 minute intervals New South Wales, South Australia and Victoria: September 1 2015 to 16 January 2017.

The greater level of variability in half hourly generation can be managed though the dispatch of higher and lower priced energy bids from generators. These prices and offers form the spot market for electricity generation.  Consequently, variability in wind generation can impact on the variability of spot market prices. How big this impact will depend on the current state of the market. For example, four per cent of the variation in half hour prices in South Austria was between roughly 70MW and 120MW (the difference between the 95^th and 99^th percentiles). If over the corresponding times, there were available energy bids for this range of power close to the current prices, the impact would still be small. One can construct examples in which the impact would be large. The extent to which conditions promote high levels of price variability frame the empirical question of interest here.

However, the temporal correlation in wind generation can create different levels variability in output at longer and often irregular time scales. We can get a sense of this by looking at the deviation of wind generation about the trend in average wind generation (as shown in Figure 14). To do so we are going to use a much finer smooth of the deviations about trend as opposed to the trend itself as we are interested in capturing relatively short term patterns due to changes in wind velocity. Further, the regular diurnal pattern in wind generation over the period has been accounted for in the smooth to focus more on changing weather patterns. The results are shown in Figure 19.

The figures illustrate how temporal correlation can generate short, sharp and irregular cycles, referred to a ramp events when they occur in real time. The general pattern is again similar in each region, all showing a marked increase in volatility in 2016.  However, South Australia clearly stands out not only in terms of the magnitude of the swings but the frequency with which generation moves from well above to well below trend.

The time scale of these swings is considerable longer the half hourly but their speed, severity and irregular nature may be more difficult to manage, at least in terms of cost, though the dispatch process. Picking the extremes or turning points in such conditions may be on the edge of impossible. The system just needs to be sufficiently resilient and that may, at times, prove costly.

At the same time, pattern of these large swings in wind generation in South Australia fit visually with the shift in the changing distribution of prices over time presented in the very first blog.  While not a sound basis for drawing conclusions, the dynamics wind generation and prices in South Austria look to be worth investigating.

Figure 19. The trend and deviation about trend in wind generation in New South Wales, South Australia and Victoria: September 1 2015 to 16 January 2017.

While an increase in the variability of demand in South Australia may be a very unlikely source of the observed increase in the variability and consequent increase in prices in South Australia, there are other candidates. These include:

·      the natural variability in dispatched wind generation in South Australia;

·      transmission constraints in and out of South Australia; and

·      variability in market conditions in other NEM regions that also affect South Australia.

Before looking  look directly at the variability of dispatched wind generation in South Australia it is interesting to look at what has been happening in other states for a few reasons.

First we can see whether the problem is unique to or predominantly occurs within South Australia, or whether it is symptomatic of a wider problem across the Eastern Seaboard. This approach goes beyond addressing the third possible source. Wind generation accounts for a much larger proportion of dispatch in South Australia than it does in the other regions, so if the variability of wind generation is a driver of price variability, it should predominantly manifest in South Australia. Second, if price volatility is unique to South Australia, then it would be reasonably clear that whatever its source, the increased price variability it is not being passes through to the other regions. Third, if there is a similar pattern of prices and prices variability in other regions, this raises a critical question whether the problem affects the Eastern Seaboard as a whole, or whether what is happening in South Australia is impacting on other states.

Smoothed prices for the other NEM regions (excluding Tasmania) are shown in Figure 9, while corresponding levels of price volatility are shown in Figure 10. Queensland is shown in a separate panel as it is clear that different patterns in prices and price volatility prevail there. While there is a strong relationship between average prices and price volatility, as measured by the mean absolute price deviation about trend, we will keep the focus here on NSW, South Australia and Victoria.

From the two figures it is clear that the escalation of average prices and price volatility in 2016 has predominantly occurred in South Australia. However, there is a similar but dampened pattern in New South Wales and Victoria. This suggests that what is occurring in South Australia is either occurring in the other regions to a lesser degree or that what is occurring in South Australia is impacting across the system.

Figure 9. Smoothed prices in NSW, Queensland, South Australia and Victoria: 15 October 2012 through 15 October 2016.

Figure 10. Price volatility in NSW, Queensland, South Australia and Victoria: 15 October 2012 through 15 October 2016.

We can take a closer look at what has been happening to the price distributions in NSW and Victoria using the same quantile graph as was done for South Australia. The evolution of the distribution for NSW is shown in Figure 11 and in Figure 12 for Victoria. The same pattern emerged in 2016 in these regions, but on a different scale. This includes the shift of the 80^th percentile which has not generally occurred in the past during periods of elevated price volatility. It is immediately apparent that the evolution of the price distributions in NSW and Victoria are very similar.

Figure 11. Quantiles of the electricity price distribution for NSW: 15 October 2012 through 15 October 2016.

Figure 12. Quantiles of the electricity price distribution for Victoria: 15 October 2012 through 15 October 2016.

 This does not resolve the question of whether increased price volatility is predominantly due  to a system wide issue or is localised in South Australia. However there are two clear differences in the generation profiles of NSW, South Australia and Queensland as can been seen in  Figure 13. South Australia has relatively high proportion of wind generation which is a debated potential source of price volatility. Two, South Australia represents a relative small proportion of total generation and as such might not have a substantive impact on the rest of the grid.  This brings to the forefront the question of causality.

Figure 13: Installed and committed scheduled and semi-scheduled generation capacity