Change workspace storage redundancy options

  • Article

When creating a workspace, the default redundancy for the managed storage account (Databricks filesystem (DBFS) root) is set as Geo-redundant storage (GRS). In this article, you will learn how to change the Azure Databricks workspace storage (DBFS) account redundancy settings.

Important

Starting on March 6, 2023, new Azure Databricks workspaces use Azure Data Lake Storage Gen2 storage accounts for the DBFS root. Workspaces created before March 6, 2023 used Blob Storage.

Storage redundancy options

To see a list of supported SKUs for the DBFS storage account type used in your workspace, see the supported redundancy options for each storage account type.

Review the replication change table for migration requirements. Confirm that the replication type is available in your workspace's region. See region availability documentation. If switching to a new replication type requires performing a conversion, see the related Azure documentation to learn how.

Note

Confirm that your region supports ZRS or GZRS storage.

Change the managed workspace storage redundancy option

Note

Updating workspace storage accounts is supported from API versions greater than 2022-04-01-preview.

For existing workspaces, change the replication type of DBFS in the Azure Portal, or use an ARM template.

Change the Replication type in the Azure Portal:

  1. Select the workspace to update from the Azure Databricks Portal Blade.

  2. Click Export Template from the left-hand menu. The Portal loads the current workspace definition in JSON format.

  3. Click Deploy on top of the screen.

  4. Click Edit Template, search for "storageAccountSkuName", and modify the replication type. For example:

     "storageAccountSkuName": {
          "type": "String",
          "value": "Standard_LRS"
       }

  5. Click Save, and confirm that the Subscription, Resource group Name, Region, and Workspace name haven't changed.

  6. Click Review + Create to deploy the change.

FAQ

Will there be any downtime during this change?

No. During a conversion, you can access data in your storage account with no loss of durability or availability. The Azure Storage SLA is maintained during the migration process, and there is no data loss associated with a conversion.

What changed on March 6, 2023?

Starting on March 6, 2023, any new workspace created will have an Azure Data Lake Storage Gen2 Storage Account as its DBFS root.

What replication types are supported for workspaces created before March 6, 2023?

Three replication types are supported: LRS, GRS and RA-GRS. Workspaces created before March 6, 2023 cannot be upgraded to ADLS Gen 2. If ADLS Gen 2 is required for DBFS, create a new workspace.

How do I migrate an existing workspace storage (DBFS) account to Zone Redundant Storage (ZRS)?

Please contact the Azure support team for assistance migrating to ZRS.