series_decompose()

  • Article

Applies to: ✅ Azure Data Explorer ✅ Azure Monitor ✅ Microsoft Sentinel

Applies a decomposition transformation on a series.

Takes an expression containing a series (dynamic numerical array) as input and decomposes it to seasonal, trend, and residual components.

Syntax

series_decompose(Series , [ Seasonality, Trend, Test_points, Seasonality_threshold ])

Learn more about syntax conventions.

Parameters

Name Type Required Description
Series dynamic ✔️ An array of numeric values, typically the resulting output of make-series or make_list operators.
Seasonality int Controls the seasonal analysis. The possible values are:

- -1: Autodetect seasonality using series_periods_detect. This is the default value.
- Period: A positive integer specifying the expected period in number of bins. For example, if the series is in 1 - h bins, a weekly period is 168 bins.
- 0: No seasonality, so skip extracting this component.
Trend string Controls the trend analysis. The possible values are:

- avg: Define trend component as average(x). This is the default.
- linefit: Extract trend component using linear regression.
- none: No trend, so skip extracting this component.
Test_points int A positive integer specifying the number of points at the end of the series to exclude from the learning, or regression, process. This parameter should be set for forecasting purposes. The default value is 0.
Seasonality_threshold real The threshold for seasonality score when Seasonality is set to autodetect. The default score threshold is 0.6.

For more information, see series_periods_detect.

Returns

The function returns the following respective series:

  • baseline: the predicted value of the series (sum of seasonal and trend components, see below).
  • seasonal: the series of the seasonal component:
    • if the period isn't detected or is explicitly set to 0: constant 0.
    • if detected or set to positive integer: median of the series points in the same phase
  • trend: the series of the trend component.
  • residual: the series of the residual component (that is, x - baseline).

Note

  • Component execution order:
  1. Extract the seasonal series
  2. Subtract it from x, generating the deseasonal series
  3. Extract the trend component from the deseasonal series
  4. Create the baseline = seasonal + trend
  5. Create the residual = x - baseline
  • Either seasonality and, or trend should be enabled. Otherwise, the function is redundant, and just returns baseline = 0 and residual = x.

More about series decomposition

This method is usually applied to time series of metrics expected to manifest periodic and/or trend behavior. You can use the method to forecast future metric values and/or detect anomalous values. The implicit assumption of this regression process is that apart from seasonal and trend behavior, the time series is stochastic and randomly distributed. Forecast future metric values from the seasonal and trend components while ignoring the residual part. Detect anomalous values based on outlier detection only on the residual part only. Further details can be found in the Time Series Decomposition chapter.

Examples

Weekly seasonality

In the following example, we generate a series with weekly seasonality and without trend, we then add some outliers to it. series_decompose finds and automatically detects the seasonality, and generates a baseline that is almost identical to the seasonal component. The outliers we added can be clearly seen in the residuals component.

let ts=range t from 1 to 24*7*5 step 1 
| extend Timestamp = datetime(2018-03-01 05:00) + 1h * t 
| extend y = 2*rand() + iff((t/24)%7>=5, 10.0, 15.0) - (((t%24)/10)*((t%24)/10)) // generate a series with weekly seasonality
| extend y=iff(t==150 or t==200 or t==780, y-8.0, y) // add some dip outliers
| extend y=iff(t==300 or t==400 or t==600, y+8.0, y) // add some spike outliers
| summarize Timestamp=make_list(Timestamp, 10000),y=make_list(y, 10000);
ts 
| extend series_decompose(y)
| render timechart

Series decompose 1.

Weekly seasonality with trend

In this example, we add a trend to the series from the previous example. First, we run series_decompose with the default parameters. The trend avg default value only takes the average and doesn't compute the trend. The generated baseline doesn't contain the trend. When observing the trend in the residuals, it becomes apparent that this example is less accurate than the previous example.

let ts=range t from 1 to 24*7*5 step 1 
| extend Timestamp = datetime(2018-03-01 05:00) + 1h * t 
| extend y = 2*rand() + iff((t/24)%7>=5, 5.0, 15.0) - (((t%24)/10)*((t%24)/10)) + t/72.0 // generate a series with weekly seasonality and ongoing trend
| extend y=iff(t==150 or t==200 or t==780, y-8.0, y) // add some dip outliers
| extend y=iff(t==300 or t==400 or t==600, y+8.0, y) // add some spike outliers
| summarize Timestamp=make_list(Timestamp, 10000),y=make_list(y, 10000);
ts 
| extend series_decompose(y)
| render timechart

Series decompose 2.

Next, we rerun the same example. Since we're expecting a trend in the series, we specify linefit in the trend parameter. We can see that the positive trend is detected and the baseline is much closer to the input series. The residuals are close to zero, and only the outliers stand out. We can see all the components on the series in the chart.

let ts=range t from 1 to 24*7*5 step 1 
| extend Timestamp = datetime(2018-03-01 05:00) + 1h * t 
| extend y = 2*rand() + iff((t/24)%7>=5, 5.0, 15.0) - (((t%24)/10)*((t%24)/10)) + t/72.0 // generate a series with weekly seasonality and ongoing trend
| extend y=iff(t==150 or t==200 or t==780, y-8.0, y) // add some dip outliers
| extend y=iff(t==300 or t==400 or t==600, y+8.0, y) // add some spike outliers
| summarize Timestamp=make_list(Timestamp, 10000),y=make_list(y, 10000);
ts 
| extend series_decompose(y, -1, 'linefit')
| render timechart

Series decompose 3.

Related content