September 2021

Article
10/14/2024

These features and Azure Databricks platform improvements were released in September 2021.

Note

The release date and content listed below only corresponds to actual deployment of the Azure Public Cloud in most case.

It provide the evolution history of Azure Databricks service on Azure Public Cloud for your reference that may not be suitable for Azure operated by 21Vianet.

Note

Releases are staged. Your Azure Databricks account may not be updated until a week or more after the initial release date.

Databricks Runtime 10.0 (Beta)

September 29, 2021

Databricks Runtime 10.0, 10.0 Photon, and 10.0 ML are now available as Beta releases.

See the full release notes at Databricks Runtime 10.0 (EoS) and Databricks Runtime 10.0 for ML (EoS).

Databricks ODBC driver 2.6.18

September 28, 2021

We have released version 2.6.18 of the Databricks ODBC driver (download). This release prepares for the Unity Catalog by incorporating three-level namespace support and enabling management across multiple catalogs.

Use customer managed keys for encryption with existing workspaces

September 24, 2021

You can now update an existing workspace for encryption using a customer managed key. Previously you could enable encryption with customer managed keys on new workspaces only. This helps you to comply with business and security requirements and provides you with more control over workspace objects. For more information, see Enable customer-managed keys for managed services.

Databricks Runtime 9.1 LTS and 9.1 LTS ML are GA; 9.1 LTS Photon is Public Preview

September 23, 2021

Databricks Runtime 9.1 LTS and 9.1 LTS ML are now generally available. 9.1 LTS Photon is in Public Preview. These images will be supported through September 2023.

Note

LTS means this version is under long-term support. See Databricks Runtime LTS version lifecycle.

See the full release notes at Databricks Runtime 9.1 LTS and Databricks Runtime 9.1 LTS for Machine Learning.

Databricks Runtime 8.1 series support ends

September 22, 2021

Support for Databricks Runtime 8.1 and Databricks Runtime 8.1 for Machine Learning ended on September 22. See Databricks support lifecycles.

Databricks JDBC driver 2.6.19

September 22, 2021

We have released version 2.6.19 of the Databricks JDBC driver (download). This release adds support for Cloud Fetch, which is a mechanism for fetching data in parallel through cloud storage to bring the data faster to BI tools. This release also reduces latency by removing unnecessary roundtrips between the client and the Databricks Runtime. See How We Achieved High-bandwidth Connectivity With BI Tools.

September 20-27, 2021: Version 3.55

Feature Store now supports sharing feature tables across multiple workspaces. For example, from your own workspace, you can create, write to, or read from a feature table in a centralized feature store. This is useful when multiple teams share access to feature tables or when your organization has multiple workspaces to handle different stages of development.

For details, see Share feature tables across workspaces (legacy).

General Availability of several Azure GPU instance types

September 20-27, 2021: Version 3.55

Support for clusters using the following Azure GPU instance types is now generally available:

Azure NC v2 instances: Standard_NC6s_v2, Standard_NC12s_v2, Standard_NC24s_v2, Standard_NC24rs_v2
Azure NC T4 v3 instances: Standard_NC4as_T4_v3, Standard_NC8as_T4_v3, Standard_NC16as_T4_v3, Standard_NC64as_T4_v3

See GPU-enabled compute for the full list of supported Azure GPU instance types.

Repos now supports `.gitignore`

September 20-27, 2021: Version 3.55

Databricks Repos now supports .gitignore, allowing you to specify that certain files not be tracked by Git. If you add a file to your repo and do not want it to be tracked by Git, create a .gitignore file or use one cloned from your remote repository and add the filename, including the extension.

.gitignore works only for files that are not already tracked by Git. If you add a file that is already tracked by Git to a .gitignore file, the file is still tracked by Git.

For more information about Databricks Repos, see Git integration for Databricks Git folders.

Enhanced jobs UI is now standard for all workspaces

September 20-27, 2021: Version 3.55

An enhanced jobs UI featuring a Job details side panel providing access to common jobs settings is now standard for all workspaces. The classic UI is no longer supported, and the Switch back to the Classic UI button is removed from all workspaces. See Schedule and orchestrate workflows for more information on using the new jobs UI.

Databricks Runtime 9.1 (Beta)

September 14, 2021

Databricks Runtime 9.1, 9.1 Photon, and 9.1 ML are now available as Beta releases.

See the full release notes at Databricks Runtime 9.1 LTS and Databricks Runtime 9.1 LTS for Machine Learning.

Streamlined management of settings for Azure Databricks jobs

September 8, 2021

In the new Job details side panel in the jobs UI, you can now manage common settings for your Azure Databricks jobs, such as the job schedule, cluster settings, maximum concurrent runs, and job alerts.

Databricks SQL Public Preview available in all workspaces

September 7, 2021

Databricks SQL, an intuitive environment for running ad-hoc queries and creating dashboards on data stored in your data lake, has started rolling out in Public Preview to all workspaces. For more information, see the Databricks SQL release notes.