November 2022
These features and Azure Databricks platform improvements were released in November 2022.
Note
The release date and content listed below only corresponds to actual deployment of the Azure Public Cloud in most case.
It provide the evolution history of Azure Databricks service on Azure Public Cloud for your reference that may not be suitable for Azure operated by 21Vianet.
Note
Releases are staged. Your Azure Databricks account might not be updated until a week or more after the initial release date.
Enhanced notifications for your Azure Databricks jobs (Public Preview)
November 30 - Dec 7, 2022
You can now send notifications for important job run events using webhooks or native Slack notifications. This feature adds to the existing email notification support for job run events. For more information, see Add email and system notifications for job events.
Databricks Runtime 12.0 (Beta)
November 28, 2022
Databricks Runtime 12.0 is now available as a Beta release.
See Databricks Runtime 12.0 (EoS) and Databricks Runtime 12.0 for Machine Learning (EoS).
Upload data UI can now be disabled via admin settings
November 17, 2022
The workspace administrator setting to disable the upload data UI now applies to the new upload data UI as well as the legacy DBFS file upload UI. This setting applies to the Data Science & Engineering, Databricks Machine Learning, and Databricks SQL personas.
Partner Connect support for Unity Catalog is GA
November 14, 2022
You can now connect select partner solutions to data managed by Unity Catalog using Partner Connect. For a list of supported partners, see Azure Databricks Partner Connect partners.
Work with large repositories with Sparse Checkout
November 14, 2022
Sparse Checkout support in Repos enables you to clone and work with a subset of the remote repository's directories in Azure Databricks. This is useful if you are working with a monorepo or your repository's size is beyond the Databricks supported limits.
For more information, see Configure Sparse checkout mode in Repos.
Databricks Terraform provider updated to version 1.6.5
November 11, 2022
Version 1.6.5 adds a query_plan parameter to the databricks_sql_visualization resource, uses a new name filter to search for jobs by name in the databricks_job data source, and more. For details, see the changelogs for version 1.6.5.
Create Serverless SQL warehouses in West US region
November 10, 2022
Serverless SQL warehouses are now available in West US region.
Databricks Terraform provider updated to versions 1.6.3 and 1.6.4
November 7-9, 2022
Versions 1.6.3 and 1.6.4 add the warehouse_type parameter to the databricks_sql_endpoint resource to support additional Databricks SQL warehouse types, and more. For details, see the changelogs for versions 1.6.3 and 1.6.4.
Specify a cloud storage location for Unity Catalog managed tables at the catalog and schema levels
November 18, 2022
Unity Catalog now lets you specify a cloud storage location for managed tables at the catalog and schema levels using Catalog Explorer. You'll be able to do the same using SQL statements soon. See Create catalogs and Create schemas.
Access recent objects from the search field in the top bar of your workspace
November 8-14, 2022
When you click the Search field in the top bar of your Azure Databricks workspace, your recent files, notebooks, queries, alerts, and dashboards are now listed under Recents, sorted by the last opened date. Recent objects in the list are filtered to match your search criteria. For more information, see Search results page.
Create or modify table from file upload page now supports multiple files
November 8, 2022
You can now use the Create or modify table from file upload page to load up to 10 files into a Delta table simultaneously. See Create or modify a table using file upload.
Create or modify table from file upload page now supports overwrite
November 8, 2022
You can now use the Create or modify table from file upload page to create or overwrite managed Delta tables. See Create or modify a table using file upload.
Search for jobs by name with the Jobs API 2.1
November 3, 2022
You can now filter by job name with the List all jobs operation (GET /jobs/list) in the Jobs API.
Databricks Terraform provider updated to version 1.6.2
November 2, 2022
Version 1.6.2 adds runtime_engine to the databricks_cluster resource, validation for path in the databricks_repo resource, auto-detection of AWS CodeCommit URLs in the databricks_repo resource, and much more. For details, see the changelog.
Search for tables in Unity Catalog is GA
November 1, 2022
You can now search for tables that are registered in Unity Catalog using the search dialog located at the top of every page in the Azure Databricks workspace UI. You can search on table names, table comments, column names, and column comments. Only tables that you have permission to see appear in search results. There is no longer a separate table search feature in Catalog Explorer.
For details, see Search for workspace objects.