December 2021
These features and Azure Databricks platform improvements were released in December 2021.
Note
The release date and content listed below only corresponds to actual deployment of the Azure Public Cloud in most case.
It provide the evolution history of Azure Databricks service on Azure Public Cloud for your reference that may not be suitable for Azure operated by 21Vianet.
Note
Releases are staged. Your Azure Databricks account may not be updated until a week or more after the initial release date.
Databricks Runtime 6.4 Extended Support series end-of-support date extended
December 31, 2021
Support for the Databricks Runtime 6.4 Extended Support series has been extended to June 30, 2022. A previous announcement stated that support would end on December 31, 2021. See All supported Databricks Runtime releases.
Databricks Runtime 5.5 Extended Support series reaches end of support
December 31, 2021
Support for the Databricks Runtime 5.5 Extended Support series ended on December 31. See Databricks support lifecycles.
Databricks JDBC driver 2.6.21
December 24, 2021
We have released version 2.6.21 of the Databricks JDBC driver (download). This release includes an upgrade of the Log4j library to version 2.17.0.
Databricks Connector for Tableau 2021.4
** December 23, 2021**
We have released a new version of the Databricks connector in Tableau 2021.4. This release adds support for navigating a three-level namespace in the Unity Catalog.
Databricks Runtime 10.2 and 10.2 ML are GA; 10.2 Photon is Public Preview
December 22, 2021
Databricks Runtime 10.2 and Databricks Runtime 10.2 ML are now generally available. Databricks Runtime 10.2 Photon is in Public Preview.
See Databricks Runtime 10.2 (EoS) and Databricks Runtime 10.2 for ML (EoS).
Switzerland West region now available
December 20, 2021
You can now create workspaces in the Switzerland West region. This is a special region that requires an Azure region access request for your subscription.
Updated Markdown parser
December 13, 2021 - January 6, 2022: Version 3.61
The Markdown parser used in Azure Databricks notebooks has been updated. The new version is faster when rendering. Some behavior has changed. The new behavior is closer to the commonmark spec.
The primary changes are:
- You must now put a space between the hashtags and the text of a heading.
- When you mix HTML and Markdown code in a cell, you must put a newline after each HTML start tag.
For example, the Markdown code to produce this result:
is as shown:
%md
## Heading
<del>
*example text*
</del>
User interface improvements for Delta Live Tables
December 13, 2021 - January 6, 2022: Version 3.61
This release includes the following enhancements to the Delta Live Tables UI:
- When you create a pipeline, you can directly edit the JSON specification in the Create Pipeline dialog by clicking the JSON button.
- You can view tooltips for data quality metrics by hovering over data quality values in the processing graph view for your pipeline.
Additional metastores coming soon to multiple regions
December 9, 2021
The following metastores are coming soon to the following regions:
centralusc2:
consolidated-centralusc2-prod-metastore-1.mysql.database.azure.comconsolidated-centralusc2-prod-metastore-2.mysql.database.azure.comconsolidated-centralusc2-prod-metastore-3.mysql.database.azure.com
eastusc3:
consolidated-eastusc3-prod-metastore-1.mysql.database.azure.comconsolidated-eastusc3-prod-metastore-2.mysql.database.azure.comconsolidated-eastusc3-prod-metastore-3.mysql.database.azure.com
eastus2c2:
consolidated-eastus2c2-prod-metastore-1.mysql.database.azure.comconsolidated-eastus2c2-prod-metastore-2.mysql.database.azure.comconsolidated-eastus2c2-prod-metastore-3.mysql.database.azure.com
northeuropec2:
consolidated-northeuropec2-prod-metastore-1.mysql.database.azure.comconsolidated-northeuropec2-prod-metastore-2.mysql.database.azure.comconsolidated-northeuropec2-prod-metastore-3.mysql.database.azure.com
westeuropec2:
consolidated-westeuropec2-prod-metastore-1.mysql.database.azure.comconsolidated-westeuropec2-prod-metastore-2.mysql.database.azure.comconsolidated-westeuropec2-prod-metastore-3.mysql.database.azure.com
You should always include all relevant hostnames for each region you use in your user-defined routes (UDRs). See User-defined route settings for Azure Databricks.
Update SCC relay for Australia Southeast region
If you use the Australia Southeast region with secure cluster connectivity (SCC), the SCC relay address has been updated to tunnel.australiaeast.azuredatabricks.net. Review and update your user-defined routes (UDRs). When possible, Databricks recommends that UDRs use hostnames, rather than IP addresses. See User-defined route settings for Azure Databricks.
Databricks Runtime 8.3 series support extended
December 8, 2021
Support for Databricks Runtime 8.3 and Databricks Runtime 8.3 for Machine Learning has been extended from December 8, 2021 to January 20, 2022. See All supported Databricks Runtime releases.
Databricks Runtime 10.2 (Beta)
December 6, 2021
Databricks Runtime 10.2, 10.2 Photon, and 10.2 ML are now available as Beta releases.
See Databricks Runtime 10.2 (EoS) and Databricks Runtime 10.2 for ML (EoS).
Revert of recent breaking change that removed escaping and quotes from $ in environment variable values for cluster creation
December 3, 2021
The breaking change to environment variable behavior that was introduced in some workspaces last month will be reverted on December 3, 2021 from 01:00-03:00 UTC.