April 2018
Releases are staged. Your Azure Databricks account may not be updated until a week after the initial release date.
Note
The release date and content listed below only corresponds to actual deployment of the Azure Public Cloud in most case.
It provide the evolution history of Azure Databricks service on Azure Public Cloud for your reference that may not be suitable for Azure operated by 21Vianet.
Note
We are now providing Databricks Runtime deprecation notices in Databricks Runtime release notes versions and compatibility.
Secrets CLI
April 26, 20180
Databricks CLI version 0.7.0 gives you the power to manage secrets from the command line. The secrets documentation now shows how to use the secrets CLI commands to create and manage secrets.
See Secret management.
Deep Learning guides
April 24, 2018
We have added documentation for Deep Learning on Azure Databricks using CPU clusters.
See Deep learning.
Secrets API update for Create Secret Scope
April 25 - May 1, 2018: Version 2.70
The Create Secret Scope endpoint (2.0/preview/secret/scopes/create) now deprecates the field initial_manage_acl and uses initial_manage_principal instead. The new field provides the same functionality but better semantics.
See Secrets API.
Spark error tips
April 24 - May 1, 2018: Version 2.70
Azure Databricks now provides tips to help you interpret and troubleshoot many of the errors you might see when you run Spark commands. And we'll keep adding more.
Databricks CLI 0.7.0
April 24, 2018
Databricks CLI 0.7.0 includes bug fixes.
It also provides a command-line interface to the Secrets API.
Increase init script output truncation limit
April 24 - May 1, 2018: Version 2.70
We have increased the output truncation limit for init scripts to 500,000 characters.
Clusters API: added UPSIZE_COMPLETED event type
April 24 - May 1, 2018: Version 2.70
The new UPSIZE_COMPLETED cluster event type indicates that nodes have finished being added to a cluster.
See Clusters API in the Clusters API reference.
Command autocomplete
April 10 - 17, 2018: Version 2.69
Azure Databricks now supports two types of autocomplete in your notebooks: local and server. Local autocomplete completes words that exist in the notebook. Server autocomplete is more powerful because it accesses the cluster for defined types, classes, and objects, as well as SQL database and table names. To activate server autocomplete you must attach your notebook to a running cluster and run all cells that define completable objects.
Serverless pools upgraded to Databricks Runtime 4.0
April 10, 2018
The serverless pools runtime version has been upgraded from Databricks Runtime 3.5 (which includes Apache Spark 2.2.1) to Databricks Runtime 4.0 (which includes Apache Spark 2.3.0). You must restart your clusters to pick up this change.
The upgrade represents a minor Apache Spark version update and is backwards compatible.