Manage cluster vertical scaling (scale up) in Azure Data Explorer to accommodate changing demand

  • Article

Sizing a cluster appropriately is critical to the performance of Azure Data Explorer. A static cluster size can lead to under-utilization or over-utilization, neither of which is ideal.

Since demand on a cluster can't be predicted with absolute accuracy, a better approach is to scale a cluster, adding and removing capacity and CPU resources with changing demand.

There are two workflows for scaling an Azure Data Explorer cluster:

  • Horizontal scaling, also called scaling in and out.
  • Vertical scaling, also called scaling up and down.

This article explains the vertical scaling workflow:

Configure vertical scaling

  1. In the Azure portal, go to your Azure Data Explorer cluster resource. Under Settings, select Scale up.

  2. In the Scale up window, you'll see available SKUs for your cluster. For example, in the following figure, there are eight recommended SKUs available. Expand the Storage optimized, Compute optimized, and Dev/test dropdowns to see more options.

    Scale up.

    The SKUs are disabled because they're the current SKU, or they aren't available in the region where the cluster is located.

  3. To change your SKU, select a new SKU and then select Apply.

    Note

    • During the vertical scaling process, while a new cluster's resources are prepared the old cluster's resources continuing to provide service. This process may take tens of minutes. Only when the new cluster's resources are ready, switchover is performed to the new cluster. The parallel process makes the SKU migration experience relatively seamless, with minimal service disruption during the switchover process that takes about one to three minutes to complete. Query performance may be impacted during SKU migration. The impact may vary due to usage patterns.
    • We recommend enabling Optimized Autoscale to allow the cluster to scale-in following migration.
    • Clusters with Virtual Network configuration may experience longer service disruptions.
    • The price is an estimate of the cluster's virtual machines and Azure Data Explorer service costs. Other costs are not included. For an estimate, see the Azure Data Explorer cost estimator. For full pricing, see the Azure Data Explorer pricing page.

You've now configured vertical scaling for your Azure Data Explorer cluster. Add another rule for a horizontal scaling. If you need assistance with cluster-scaling issues, open a support request in the Azure portal.

Related content