Use private Python packages with Azure Machine Learning

APPLIES TO: Python SDK azureml v1

In this article, learn how to use private Python packages securely within Azure Machine Learning. Use cases for private Python packages include:

• You've developed a private package that you don't want to share publicly.
• You want to use a curated repository of packages stored within an enterprise firewall.

The recommended approach depends on whether you have few packages for a single Azure Machine Learning workspace, or an entire repository of packages for all workspaces within an organization.

The private packages are used through Environment class. Within an environment, you declare which Python packages to use, including private ones. To learn about environment in Azure Machine Learning in general, see How to use environments.

Prerequisites

• The Azure Machine Learning SDK for Python
• An Azure Machine Learning workspace

Use small number of packages for development and testing

For a few private packages for a single workspace, use the static Environment.add_private_pip_wheel() method. This approach allows you to quickly add a private package to the workspace, and is well suited for development and testing purposes.

Point the file path argument to a local wheel file and run the add_private_pip_wheel command. The command returns a URL used to track the location of the package within your Workspace. Capture the storage URL and pass it the add_pip_package() method.

whl_url = Environment.add_private_pip_wheel(workspace=ws,file_path = "my-custom.whl")
myenv = Environment(name="myenv")
conda_dep = CondaDependencies()
conda_dep.add_pip_package(whl_url)
myenv.python.conda_dependencies=conda_dep

Internally, Azure Machine Learning service replaces the URL by secure SAS URL, so your wheel file is kept private and secure.

Use a repository of packages from private storage

You can consume packages from an Azure storage account within your organization's firewall. The storage account can hold a curated set of packages or an internal mirror of publicly available packages.

To set up such private storage, see Secure an Azure Machine Learning workspace and associated resources. You must also place the Azure Container Registry (ACR) behind the virtual network.

After completing these configurations, you can reference the packages in the Azure Machine Learning environment definition by their full URL in Azure blob storage.

Next steps

• Learn more about enterprise security in Azure Machine Learning