通过

用于机器学习的 Databricks Runtime 7.6 (EoS)

  • 项目

注意

对此 Databricks Runtime 版本的支持已结束。 有关终止支持日期，请参阅终止支持历史记录。 有关所有受支持的 Databricks Runtime 版本，请参阅 Databricks Runtime 发行说明版本和兼容性。

Databricks 于 2021 年 2 月发布了此版本。

用于机器学习的 Databricks Runtime 7.6 基于 Databricks Runtime 7.6 (EoS)，为机器学习和数据科学提供了随时可用的环境。 Databricks Runtime ML 包含许多常用的机器学习库，包括 TensorFlow、PyTorch 和 XGBoost。 它还支持使用 Horovod 进行分布式深度学习训练。

有关详细信息，包括有关如何创建 Databricks Runtime ML 群集的说明，请参阅 Databricks 上的 AI 和机器学习。

有关从 Databricks Runtime 6.x 进行迁移的帮助信息，请参阅 Databricks Runtime 7.x 迁移指南 (EoS)。

新增功能和主要更改

Databricks Runtime 7.6 ML 基于 Databricks Runtime 7.6 构建。 若要了解 Databricks Runtime 7.6 中的新增功能，包括 Apache Spark MLlib 和 SparkR，请参阅 Databricks Runtime 7.6 (EoS) 发行说明。

弃用功能

  • 即将发布的 Databricks Runtime 主版本将不支持 Tensoflow 1.x。
  • 以下 CUDA 包被弃用，在即将发布的 Databricks Runtime 主版本中将会删除该包：
    • cuda-command-line-tools
    • cuda-compiler
    • cuda-cudart-dev
    • cuda-cufft
    • cuda-cufft-dev
    • cuda-cuobjdump
    • cuda-cupti
    • cuda-curand
    • cuda-curand-dev
    • cuda-cusolver
    • cuda-cusolver-dev
    • cuda-cusparse
    • cuda-cusparse-dev
    • cuda-documentation
    • cuda-driver-dev
    • cuda-gdb
    • cuda-gpu-library-advisor
    • cuda-libraries-dev
    • cuda-license
    • cuda-memcheck
    • cuda-minimal-build
    • cuda-misc-headers
    • cuda-npp
    • cuda-npp-dev
    • cuda-nsight
    • cuda-nvcc
    • cuda-nvdisasm
    • cuda-nvgraph
    • cuda-nvgraph-dev
    • cuda-nvjpeg
    • cuda-nvjpeg-dev
    • cuda-nvml-dev
    • cuda-nvprune
    • cuda-nvrtc-dev
    • cuda-nvvp
    • cuda-samples
    • cuda-sanitizer-api
    • cuda-toolkit
    • cuda-tools
    • cuda-visual-tools
    • freeglut3
    • libcublas-dev
    • libcudnn7-dev
    • libdrm-dev
    • libegl1
    • libegm-mesa0
    • libgbl1-mesa-dev
    • libgbm1
    • libgles1
    • libgles2
    • libglu1-mesa
    • libglu1-mesa-dev
    • libnccl-dev
    • libnvinfer-dev
    • libnvinfer-plugin-dev
    • libopengl0
    • libwayland-server0
    • libx11-xcb-dev
    • libxcb-dri2-0-dev
    • libxcb-dri3-dev
    • libxcb-glx0-dev
    • libxcb-present-dev
    • libxcb-randr0
    • libxcb-randr0-dev
    • libxcb-render0-dev
    • libxcb-shape0-dev
    • libxcb-sync-dev
    • libxcb-xfixes0
    • libxcb-xfixes0-dev
    • libxdamage-dev
    • libxext-dev
    • libxfixes-dev
    • libxi-dev
    • libxmu-dev
    • libxmu-headers
    • libxshmfence-dev
    • libxxf86vm-dev
    • mesa-common-dev
    • nsight-compute
    • nsight-systems
    • x11proto-damage-dev
    • x11proto-fixes-dev
    • x11proto-input-dev
    • x11proto-xext-dev
    • x11proto-xf86vidmode-dev

Databricks Runtime ML Python 环境的主要更改

请参阅 Databricks Runtime 7.6 (EoS) 以了解 Databricks Runtime Python 环境的主要更改。 如需查看已安装的 Python 包及其版本的完整列表，请参阅 Python 库。

升级的 Python 包

  • databricks-cli 0.14.0 -> 0.14.1
  • koalas 1.4.0 -> 1.5.0
  • lightgbm 2.3.0 -> 3.1.1
  • mlflow 1.12.1 -> 1.13.1
  • plotly 4.12.0 -> 4.14.1
  • pytorch 1.7.0 -> 1.7.1
  • torchvision 0.8.1 -> 0.8.2
  • xgboost 1.2.1 -> 1.3.1

改进

XGBoost 的 PySpark 集成（公共预览版）

改进了 XGBoost 与 PySpark 的集成。 包 sparkdl 2.1.0-db5 包括两个新的 PySpark ML 估算器（XgboostRegressor 和 XgboostClassifier），用户可以使用它们在 PySpark ML 管道中训练 XGBoost 模型。

在此版本之前，XGBoost 没有与 PySpark 集成。 用户必须使用 Scala 中的 xgboost4j-spark 或中断 PySpark ML 管道，将驱动程序上的 Spark 数据帧作为 pandas 数据帧收集，并使用 Python 包 xgboost。 有关更多详细信息，请参阅 sparkdl API 文档和在 Azure Databricks 上使用 XGBoost。

系统环境

Databricks Runtime 7.6 ML 中的系统环境在以下方面不同于 Databricks Runtime 7.6：

库

以下部分列出了 Databricks Runtime 7.6 ML 中包含的库，这些库不同于 Databricks Runtime 7.6 中包含的库。

本节内容：

顶层库

Databricks Runtime 7.6 ML 包含以下顶层库：

Python 库

Databricks Runtime 7.6 ML 使用 Conda 进行 Python 包管理，并且包含许多常用的 ML 包。

除了在以下部分的 Conda 环境中指定的包外，Databricks Runtime 7.6 ML 还安装以下包：

  • hyperopt 0.2.5.db1
  • sparkdl 2.1.0-db5

CPU 群集上的 Python 库

name: databricks-ml
channels:
  - pytorch
  - defaults
dependencies:
  - _libgcc_mutex=0.1=main
  - absl-py=0.9.0=py37_0
  - asn1crypto=1.3.0=py37_1
  - astor=0.8.0=py37_0
  - backcall=0.1.0=py37_0
  - backports=1.0=pyhd3eb1b0_2
  - bcrypt=3.2.0=py37h7b6447c_0
  - blas=1.0=mkl
  - blinker=1.4=py37_0
  - boto3=1.12.0=py_0
  - botocore=1.15.0=py_0
  - c-ares=1.17.1=h27cfd23_0
  - ca-certificates=2021.1.19=h06a4308_1 # (updated from h06a4308_0 in May 26, 2021 maintenance update)
  - cachetools=4.2.0=pyhd3eb1b0_0
  - certifi=2020.12.5=py37h06a4308_0
  - cffi=1.14.0=py37he30daa8_1 # (updated from py37h2e261b9_0 in May 26, 2021 maintenance update)
  - chardet=3.0.4=py37h06a4308_1003
  - click=7.0=py37_0
  - cloudpickle=1.4.1=py_0
  - configparser=3.7.4=py37_0
  - cpuonly=1.0=0
  - cryptography=2.8=py37h1ba5d50_0
  - cycler=0.10.0=py37_0
  - cython=0.29.15=py37he6710b0_0
  - decorator=4.4.1=py_0
  - dill=0.3.1.1=py37_1
  - docutils=0.15.2=py37_0
  - entrypoints=0.3=py37_0
  - flask=1.1.1=py_1
  - freetype=2.9.1=h8a8886c_1
  - future=0.18.2=py37_1
  - gast=0.3.3=py_0
  - gitdb=4.0.5=py_0
  - gitpython=3.1.0=py_0
  - google-auth=1.11.2=py_0
  - google-auth-oauthlib=0.4.1=py_2
  - google-pasta=0.2.0=py_0
  - grpcio=1.27.2=py37hf8bcb03_0
  - gunicorn=20.0.4=py37_0
  - h5py=2.10.0=py37h7918eee_0
  - hdf5=1.10.4=hb1b8bf9_0
  - icu=58.2=he6710b0_3
  - idna=2.8=py37_0
  - intel-openmp=2020.0=166
  - ipykernel=5.1.4=py37h39e3cac_0
  - ipython=7.12.0=py37h5ca1d4c_0
  - ipython_genutils=0.2.0=pyhd3eb1b0_1
  - isodate=0.6.0=py_1
  - itsdangerous=1.1.0=py37_0
  - jedi=0.17.2=py37h06a4308_1
  - jinja2=2.11.1=py_0
  - jmespath=0.10.0=py_0
  - joblib=0.14.1=py_0
  - jpeg=9b=h024ee3a_2
  - jupyter_client=5.3.4=py37_0
  - jupyter_core=4.6.1=py37_0
  - kiwisolver=1.1.0=py37he6710b0_0
  - krb5=1.17.1=h173b8e3_0 # (updated from 1.16.4 in May 26, 2021 maintenance update)
  - ld_impl_linux-64=2.33.1=h53a641e_7
  - libedit=3.1.20181209=hc058e9b_0
  - libffi=3.3=he6710b0_2 # (updated from 3.2.1 in May 26, 2021 maintenance update)
  - libgcc-ng=9.1.0=hdf63c60_0
  - libgfortran-ng=7.3.0=hdf63c60_0
  - libpng=1.6.37=hbc83047_0
  - libpq=12.2=h20c2e04_0 # (updated from 11.2 in May 26, 2021 maintenance update)
  - libprotobuf=3.11.4=hd408876_0
  - libsodium=1.0.16=h1bed415_0
  - libstdcxx-ng=9.1.0=hdf63c60_0
  - libtiff=4.1.0=h2733197_0
  - libuv=1.40.0=h7b6447c_0
  - lightgbm=3.1.1=py37h2531618_0
  - lz4-c=1.8.1.2=h14c3975_0
  - mako=1.1.2=py_0
  - markdown=3.1.1=py37_0
  - markupsafe=1.1.1=py37h14c3975_1
  - matplotlib-base=3.1.3=py37hef1b27d_0
  - mkl=2020.0=166
  - mkl-service=2.3.0=py37he8ac12f_0
  - mkl_fft=1.0.15=py37ha843d7b_0
  - mkl_random=1.1.0=py37hd6b4f25_0
  - ncurses=6.2=he6710b0_1
  - networkx=2.4=py_1
  - ninja=1.10.2=py37hff7bd54_0
  - nltk=3.4.5=py37_0
  - numpy=1.18.1=py37h4f9e942_0
  - numpy-base=1.18.1=py37hde5b4d6_1
  - oauthlib=3.1.0=py_0
  - olefile=0.46=py37_0
  - openssl=1.1.1k=h27cfd23_0 # (updated from 1.1.1i in May 26, 2021 maintenance update)
  - packaging=20.1=py_0
  - pandas=1.0.1=py37h0573a6f_0
  - paramiko=2.7.1=py_0
  - parso=0.7.0=py_0
  - patsy=0.5.1=py37_0
  - pexpect=4.8.0=pyhd3eb1b0_3
  - pickleshare=0.7.5=pyhd3eb1b0_1003
  - pillow=7.0.0=py37hb39fc2d_0
  - pip=20.0.2=py37_3
  - plotly=4.14.1=pyhd3eb1b0_0
  - prompt_toolkit=3.0.3=py_0
  - protobuf=3.11.4=py37he6710b0_0
  - psutil=5.6.7=py37h7b6447c_0
  - psycopg2=2.8.6=py37h3c74f83_1 # (updated from 2.8.4 in May 26, 2021 maintenance update)
  - ptyprocess=0.6.0=pyhd3eb1b0_2
  - pyasn1=0.4.8=py_0
  - pyasn1-modules=0.2.8=py_0
  - pycparser=2.19=py37_0
  - pygments=2.5.2=py_0
  - pyjwt=2.0.1=py37h06a4308_0
  - pynacl=1.3.0=py37h7b6447c_0
  - pyodbc=4.0.30=py37he6710b0_0
  - pyopenssl=19.1.0=pyhd3eb1b0_1
  - pyparsing=2.4.6=py_0
  - pysocks=1.7.1=py37_1
  - python=3.7.10=hdb3f193_0 # (updated from 3.7.6 in May 26, 2021 maintenance update)
  - python-dateutil=2.8.1=py_0
  - python-editor=1.0.4=py_0
  - pytorch=1.7.1=py3.7_cpu_0
  - pytz=2019.3=py_0
  - pyzmq=18.1.1=py37he6710b0_0
  - readline=8.1=h27cfd23_0 # (updated from 7.0 in May 26, 2021 maintenance update)
  - requests=2.22.0=py37_1
  - requests-oauthlib=1.3.0=py_0
  - retrying=1.3.3=py37_2
  - rsa=4.0=py_0
  - s3transfer=0.3.4=pyhd3eb1b0_0
  - scikit-learn=0.22.1=py37hd81dba3_0
  - scipy=1.4.1=py37h0b6359f_0
  - setuptools=45.2.0=py37_0
  - simplejson=3.17.0=py37h7b6447c_0
  - six=1.14.0=py37h06a4308_0
  - smmap=3.0.4=py_0
  - sqlite=3.35.4=hdfb4753_0 # (updated from 3.31.1 in May 26, 2021 maintenance update)
  - sqlparse=0.4.1=py_0
  - statsmodels=0.11.0=py37h7b6447c_0
  - tabulate=0.8.3=py37_0
  - tk=8.6.10=hbc83047_0 # (updated from 8.6.8 in May 26, 2021 maintenance update)
  - torchvision=0.8.2=py37_cpu
  - tornado=6.0.3=py37h7b6447c_3
  - tqdm=4.42.1=py_0
  - traitlets=4.3.3=py37_0
  - typing_extensions=3.7.4.3=py_0
  - unixodbc=2.3.7=h14c3975_0
  - urllib3=1.25.8=py37_0
  - wcwidth=0.1.8=py_0
  - websocket-client=0.56.0=py37_0
  - werkzeug=1.0.0=py_0
  - wheel=0.34.2=py37_0
  - wrapt=1.11.2=py37h7b6447c_0
  - xz=5.2.5=h7b6447c_0 # (updated from 5.2.4 in May 26, 2021 maintenance update)
  - zeromq=4.3.1=he6710b0_3
  - zlib=1.2.11=h7b6447c_3
  - zstd=1.3.7=h0b5b093_0
  - pip:
    - astunparse==1.6.3
    - azure-core==1.10.0
    - azure-storage-blob==12.7.0
    - databricks-cli==0.14.1
    - diskcache==5.1.0
    - docker==4.4.1
    - gorilla==0.3.0
    - horovod==0.20.3
    - joblibspark==0.3.0
    - keras-preprocessing==1.1.2
    - koalas==1.5.0
    - mleap==0.16.1
    - mlflow==1.13.1
    - msrest==0.6.19
    - opt-einsum==3.3.0
    - petastorm==0.9.7
    - pyarrow==1.0.1
    - pyyaml==5.4
    - querystring-parser==1.2.4
    - seaborn==0.10.0
    - spark-tensorflow-distributor==0.1.0
    - tensorboard==2.3.0
    - tensorboard-plugin-wit==1.8.0
    - tensorflow-cpu==2.3.1
    - tensorflow-estimator==2.3.0
    - termcolor==1.1.0
    - xgboost==1.3.1
prefix: /databricks/conda/envs/databricks-ml

GPU 群集上的 Python 库

name: databricks-ml-gpu
channels:
  - pytorch
  - defaults
dependencies:
  - _libgcc_mutex=0.1=main
  - absl-py=0.9.0=py37_0
  - asn1crypto=1.3.0=py37_1
  - astor=0.8.0=py37_0
  - backcall=0.1.0=py37_0
  - backports=1.0=pyhd3eb1b0_2
  - bcrypt=3.2.0=py37h7b6447c_0
  - blas=1.0=mkl
  - blinker=1.4=py37_0
  - boto3=1.12.0=py_0
  - botocore=1.15.0=py_0
  - c-ares=1.17.1=h27cfd23_0
  - ca-certificates=2021.1.19=h06a4308_1 # (updated from h06a4308_0 in May 26, 2021 maintenance update)
  - cachetools=4.2.0=pyhd3eb1b0_0
  - certifi=2020.12.5=py37h06a4308_0
  - cffi=1.14.0=py37he30daa8_1 # (updated from py37h2e261b9_0 in May 26, 2021 maintenance update)
  - chardet=3.0.4=py37h06a4308_1003
  - click=7.0=py37_0
  - cloudpickle=1.4.1=py_0
  - configparser=3.7.4=py37_0
  - cryptography=2.8=py37h1ba5d50_0
  - cudatoolkit=10.1.243=h6bb024c_0
  - cycler=0.10.0=py37_0
  - cython=0.29.15=py37he6710b0_0
  - decorator=4.4.1=py_0
  - dill=0.3.1.1=py37_1
  - docutils=0.15.2=py37_0
  - entrypoints=0.3=py37_0
  - flask=1.1.1=py_1
  - freetype=2.9.1=h8a8886c_1
  - future=0.18.2=py37_1
  - gast=0.3.3=py_0
  - gitdb=4.0.5=py_0
  - gitpython=3.1.0=py_0
  - google-auth=1.11.2=py_0
  - google-auth-oauthlib=0.4.1=py_2
  - google-pasta=0.2.0=py_0
  - grpcio=1.27.2=py37hf8bcb03_0
  - gunicorn=20.0.4=py37_0
  - h5py=2.10.0=py37h7918eee_0
  - hdf5=1.10.4=hb1b8bf9_0
  - icu=58.2=he6710b0_3
  - idna=2.8=py37_0
  - intel-openmp=2020.0=166
  - ipykernel=5.1.4=py37h39e3cac_0
  - ipython=7.12.0=py37h5ca1d4c_0
  - ipython_genutils=0.2.0=pyhd3eb1b0_1
  - isodate=0.6.0=py_1
  - itsdangerous=1.1.0=py37_0
  - jedi=0.17.2=py37h06a4308_1
  - jinja2=2.11.1=py_0
  - jmespath=0.10.0=py_0
  - joblib=0.14.1=py_0
  - jpeg=9b=h024ee3a_2
  - jupyter_client=5.3.4=py37_0
  - jupyter_core=4.6.1=py37_0
  - kiwisolver=1.1.0=py37he6710b0_0
  - krb5=1.17.1=h173b8e3_0 # (updated from 1.16.4 in May 26, 2021 maintenance update)
  - ld_impl_linux-64=2.33.1=h53a641e_7
  - libedit=3.1.20181209=hc058e9b_0
  - libffi=3.3=he6710b0_2 # (updated from 3.2.1 in May 26, 2021 maintenance update)
  - libgcc-ng=9.1.0=hdf63c60_0
  - libgfortran-ng=7.3.0=hdf63c60_0
  - libpng=1.6.37=hbc83047_0
  - libpq=12.2=h20c2e04_0 # (updated from 11.2 in May 26, 2021 maintenance update)
  - libprotobuf=3.11.4=hd408876_0
  - libsodium=1.0.16=h1bed415_0
  - libstdcxx-ng=9.1.0=hdf63c60_0
  - libtiff=4.1.0=h2733197_0
  - libuv=1.40.0=h7b6447c_0
  - lightgbm=3.1.1=py37h2531618_0
  - lz4-c=1.8.1.2=h14c3975_0
  - mako=1.1.2=py_0
  - markdown=3.1.1=py37_0
  - markupsafe=1.1.1=py37h14c3975_1
  - matplotlib-base=3.1.3=py37hef1b27d_0
  - mkl=2020.0=166
  - mkl-service=2.3.0=py37he8ac12f_0
  - mkl_fft=1.0.15=py37ha843d7b_0
  - mkl_random=1.1.0=py37hd6b4f25_0
  - ncurses=6.2=he6710b0_1
  - networkx=2.4=py_1
  - ninja=1.10.2=py37hff7bd54_0
  - nltk=3.4.5=py37_0
  - numpy=1.18.1=py37h4f9e942_0
  - numpy-base=1.18.1=py37hde5b4d6_1
  - oauthlib=3.1.0=py_0
  - olefile=0.46=py37_0
  - openssl=1.1.1k=h27cfd23_0 # (updated from 1.1.1i in May 26, 2021 maintenance update)
  - packaging=20.1=py_0
  - pandas=1.0.1=py37h0573a6f_0
  - paramiko=2.7.1=py_0
  - parso=0.7.0=py_0
  - patsy=0.5.1=py37_0
  - pexpect=4.8.0=pyhd3eb1b0_3
  - pickleshare=0.7.5=pyhd3eb1b0_1003
  - pillow=7.0.0=py37hb39fc2d_0
  - pip=20.0.2=py37_3
  - plotly=4.14.1=pyhd3eb1b0_0
  - prompt_toolkit=3.0.3=py_0
  - protobuf=3.11.4=py37he6710b0_0
  - psutil=5.6.7=py37h7b6447c_0
  - psycopg2=2.8.6=py37h3c74f83_1 # (updated from 2.8.4 in May 26, 2021 maintenance update)
  - ptyprocess=0.6.0=pyhd3eb1b0_2
  - pyasn1=0.4.8=py_0
  - pyasn1-modules=0.2.8=py_0
  - pycparser=2.19=py37_0
  - pygments=2.5.2=py_0
  - pyjwt=2.0.1=py37h06a4308_0
  - pynacl=1.3.0=py37h7b6447c_0
  - pyodbc=4.0.30=py37he6710b0_0
  - pyopenssl=19.1.0=pyhd3eb1b0_1
  - pyparsing=2.4.6=py_0
  - pysocks=1.7.1=py37_1
  - python=3.7.10=hdb3f193_0 # (updated from 3.7.6 in May 26, 2021 maintenance update)
  - python-dateutil=2.8.1=py_0
  - python-editor=1.0.4=py_0
  - pytorch=1.7.1=py3.7_cuda10.1.243_cudnn7.6.3_0
  - pytz=2019.3=py_0
  - pyzmq=18.1.1=py37he6710b0_0
  - readline=8.1=h27cfd23_0 # (updated from 7.0 in May 26, 2021 maintenance update)
  - requests=2.22.0=py37_1
  - requests-oauthlib=1.3.0=py_0
  - retrying=1.3.3=py37_2
  - rsa=4.0=py_0
  - s3transfer=0.3.4=pyhd3eb1b0_0
  - scikit-learn=0.22.1=py37hd81dba3_0
  - scipy=1.4.1=py37h0b6359f_0
  - setuptools=45.2.0=py37_0
  - simplejson=3.17.0=py37h7b6447c_0
  - six=1.14.0=py37h06a4308_0
  - smmap=3.0.4=py_0
  - sqlite=3.35.4=hdfb4753_0 # (updated from 3.31.1 in May 26, 2021 maintenance update)
  - sqlparse=0.4.1=py_0
  - statsmodels=0.11.0=py37h7b6447c_0
  - tabulate=0.8.3=py37_0
  - tk=8.6.10=hbc83047_0 # (updated from 8.6.8 in May 26, 2021 maintenance update)
  - torchvision=0.8.2=py37_cu101
  - tornado=6.0.3=py37h7b6447c_3
  - tqdm=4.42.1=py_0
  - traitlets=4.3.3=py37_0
  - typing_extensions=3.7.4.3=py_0
  - unixodbc=2.3.7=h14c3975_0
  - urllib3=1.25.8=py37_0
  - wcwidth=0.1.8=py_0
  - websocket-client=0.56.0=py37_0
  - werkzeug=1.0.0=py_0
  - wheel=0.34.2=py37_0
  - wrapt=1.11.2=py37h7b6447c_0
  - xz=5.2.5=h7b6447c_0 # (updated from 5.2.4 in May 26, 2021 maintenance update)
  - zeromq=4.3.1=he6710b0_3
  - zlib=1.2.11=h7b6447c_3
  - zstd=1.3.7=h0b5b093_0
  - pip:
    - astunparse==1.6.3
    - azure-core==1.10.0
    - azure-storage-blob==12.7.0
    - databricks-cli==0.14.1
    - diskcache==5.1.0
    - docker==4.4.1
    - gorilla==0.3.0
    - horovod==0.20.3
    - joblibspark==0.3.0
    - keras-preprocessing==1.1.2
    - koalas==1.5.0
    - mleap==0.16.1
    - mlflow==1.13.1
    - msrest==0.6.19
    - opt-einsum==3.3.0
    - petastorm==0.9.7
    - pyarrow==1.0.1
    - pyyaml==5.4
    - querystring-parser==1.2.4
    - seaborn==0.10.0
    - spark-tensorflow-distributor==0.1.0
    - tensorboard==2.3.0
    - tensorboard-plugin-wit==1.8.0
    - tensorflow==2.3.1
    - tensorflow-estimator==2.3.0
    - termcolor==1.1.0
    - xgboost==1.3.1
prefix: /databricks/conda/envs/databricks-ml-gpu

包含 Python 模块的 Spark 包

Spark 包 Python 模块 版本
graphframes graphframes 0.8.1-db1-spark3.0

R 库

R 库与 Databricks Runtime 7.6 中的 R 库完全相同。

Java 库和 Scala 库（Scala 2.12 群集）

除了 Databricks Runtime 7.6 中的 Java 库和 Scala 库之外，Databricks Runtime 7.6 ML 还包含以下 JAR：

CPU 群集

组 ID 项目 ID 版本
com.typesafe.akka akka-actor_2.12 2.5.23
ml.combust.mleap mleap-databricks-runtime_2.12 0.17.3-4882dc3
ml.dmlc xgboost4j-spark_2.12 1.2.0
ml.dmlc xgboost4j_2.12 1.2.0
org.mlflow mlflow-client 1.13.1
org.scala-lang.modules scala-java8-compat_2.12 0.8.0
org.tensorflow spark-tensorflow-connector_2.12 1.15.0

GPU 群集

组 ID 项目 ID 版本
com.typesafe.akka akka-actor_2.12 2.5.23
ml.combust.mleap mleap-databricks-runtime_2.12 0.17.3-4882dc3
ml.dmlc xgboost4j-spark-gpu_2.12 1.2.0
ml.dmlc xgboost4j-gpu_2.12 1.2.0
org.mlflow mlflow-client 1.13.1
org.scala-lang.modules scala-java8-compat_2.12 0.8.0
org.tensorflow spark-tensorflow-connector_2.12 1.15.0