Package and deploy models outside Azure Machine Learning (preview)

Article
08/14/2024

You can deploy models outside of Azure Machine Learning for online serving by creating model packages (preview). Azure Machine Learning allows you to create a model package that collects all the dependencies required for deploying a machine learning model to a serving platform. You can move a model package across workspaces and even outside of Azure Machine Learning. To learn more about model packages, see Model packages for deployment (preview).

Important

This feature is currently in public preview. This preview version is provided without a service-level agreement, and we don't recommend it for production workloads. Certain features might not be supported or might have constrained capabilities.

For more information, see Supplemental Terms of Use for Azure Previews.

In this article, you learn how package a model and deploy it to an Azure App Service.

Prerequisites

Before following the steps in this article, make sure you have the following prerequisites:

An Azure subscription. If you don't have an Azure subscription, create a Trial before you begin. Try the Azure Machine Learning.
An Azure Machine Learning workspace. If you don't have one, use the steps in the How to manage workspaces article to create one.

Note

Private link enabled workspaces don't support packaging models for deployment outside of Azure Machine Learning.
Azure role-based access controls (Azure RBAC) are used to grant access to operations in Azure Machine Learning. To perform the steps in this article, your user account must be assigned the owner or contributor role for the Azure Machine Learning workspace, or a custom role. For more information, see Manage access to an Azure Machine Learning workspace.

Prepare your system

Follow these steps to prepare your system.

The example in this article is based on code samples contained in the azureml-examples repository. To run the commands locally without having to copy/paste YAML and other files, first clone the repo and then change directories to the folder:
- Azure CLI
- Python
```
git clone https://github.com/Azure/azureml-examples --depth 1
cd azureml-examples/cli
```
```
!git clone https://github.com/Azure/azureml-examples --depth 1
!cd azureml-examples/sdk/python
```
This article uses the example in the folder endpoints/online/deploy-with-packages/mlflow-model.
Connect to the Azure Machine Learning workspace where you'll do your work.
- Azure CLI
- Python
```
az account set --subscription <subscription>
az configure --defaults workspace=<workspace> group=<resource-group> location=<location>
```
The workspace is the top-level resource for Azure Machine Learning, providing a centralized place to work with all the artifacts you create when you use Azure Machine Learning. Here, you connect to the workspace in which you perform deployment tasks.

Important

To use this preview feature, the azure-ai-ml package must be upgraded to a minimum version 1.12.0. Run pip install azure-ai-ml==1.12.0 in your conda environment to ensure the latest SDK is installed.
1. Import the required libraries:
```
from azure.ai.ml import MLClient, Input
from azure.ai.ml.entities import ManagedOnlineEndpoint, ManagedOnlineDeployment, Model
from azure.ai.ml.constants import AssetTypes
from azure.identity import DefaultAzureCredential
```
2. If you're running in an Azure Machine Learning compute instance, create an MLClient as follows:
```
ml_client = MLClient.from_config(DefaultAzureCredential())
```
  Otherwise, configure workspace details and get a handle to the workspace:
```
subscription_id = "<subscription>"
resource_group = "<resource-group>"
workspace = "<workspace>"

ml_client = MLClient(DefaultAzureCredential(), subscription_id, resource_group, workspace)
```
Packages require the model to be registered in either your workspace or in an Azure Machine Learning registry. In this example, there's a local copy of the model in the repository, so you only need to publish the model to the registry in the workspace. You can skip this step if the model you're trying to deploy is already registered.

#<register_model>
MODEL_NAME='heart-classifier-mlflow'
MODEL_PATH='model'
az ml model create --name $MODEL_NAME --path $MODEL_PATH --type mlflow_model
#</register_model>

#<build_package>
az ml model package -n $MODEL_NAME -l latest --file package.yml
#</build_package>

#<endpoint_name>
ENDPOINT_NAME = "heart-classifier"
#</endpoint_name>

# The following code ensures the created deployment has a unique name
ENDPOINT_SUFIX=$(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w ${1:-5} | head -n 1)
ENDPOINT_NAME="$ENDPOINT_NAME-$ENDPOINT_SUFIX"

#<create_endpoint>
az ml online-endpoint create -n $ENDPOINT_NAME
#</create_endpoint>

#<create_deployment>
az ml online-deployment create -f deployment.yml -e $ENDPOINT_NAME
#</create_deployment>

#<test_deployment>
az ml online-endpoint invoke -n $ENDPOINT_NAME -d with-package -f sample-request.json
#</test_deployment>

#<create_deployment_inline>
az ml online-deployment create --with-package -f model-deployment.yml -e $ENDPOINT_NAME
#</create_deployment_inline>

#<delete_resources>
az ml online-endpoint delete -n $ENDPOINT_NAME --yes
#</delete_resources>

#<build_package_copy>
az ml model package -n $MODEL_NAME -l latest --file package-external.yml
#</build_package_copy>

model_name = "heart-classifier-mlflow"
model_path = "model"
model = ml_client.models.create_or_update(
	Model(name=model_name, path=model_path, type=AssetTypes.MLFLOW_MODEL)
)

Deploy a model package to the Azure App Service

In this section, you package the previously registered MLflow model and deploy it to the Azure App Service.

Deploying a model outside of Azure Machine Learning requires creating a package specification. To create a package that's completely disconnected from Azure Machine Learning, specify the copy mode in the model configuration. The copy mode tells the package to copy the artifacts inside of the package. The following code shows how to specify the copy mode for the model configuration:

Create a package YAML specification:

package-external.yml

$schema: http://azureml/sdk-2-0/ModelVersionPackage.json
target_environment_name: heart-classifier-mlflow-pkg
inferencing_server: 
    type: azureml_online
model_configuration:
    mode: copy

package_config = ModelPackage(
    target_environment_name="heart-classifier-mlflow-pkg",
    inferencing_server=AzureMLOnlineInferencingServer(),
    model_configuration=ModelConfiguration(
        mode="copy"
    )
)

Tip

When you specify the model configuration using copy for the mode property, you guarantee that all the model artifacts are copied inside the generated docker image instead of downloaded from the Azure Machine Learning model registry, thereby allowing true portability outside of Azure Machine Learning. For a full specification about all the options when creating packages see Create a package specification.

Start the package operation.

Azure CLI
Python

#<register_model>
MODEL_NAME='heart-classifier-mlflow'
MODEL_PATH='model'
az ml model create --name $MODEL_NAME --path $MODEL_PATH --type mlflow_model
#</register_model>

#<build_package>
az ml model package -n $MODEL_NAME -l latest --file package.yml
#</build_package>

#<endpoint_name>
ENDPOINT_NAME = "heart-classifier"
#</endpoint_name>

# The following code ensures the created deployment has a unique name
ENDPOINT_SUFIX=$(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w ${1:-5} | head -n 1)
ENDPOINT_NAME="$ENDPOINT_NAME-$ENDPOINT_SUFIX"

#<create_endpoint>
az ml online-endpoint create -n $ENDPOINT_NAME
#</create_endpoint>

#<create_deployment>
az ml online-deployment create -f deployment.yml -e $ENDPOINT_NAME
#</create_deployment>

#<test_deployment>
az ml online-endpoint invoke -n $ENDPOINT_NAME -d with-package -f sample-request.json
#</test_deployment>

#<create_deployment_inline>
az ml online-deployment create --with-package -f model-deployment.yml -e $ENDPOINT_NAME
#</create_deployment_inline>

#<delete_resources>
az ml online-endpoint delete -n $ENDPOINT_NAME --yes
#</delete_resources>

#<build_package_copy>
az ml model package -n $MODEL_NAME -l latest --file package-external.yml
#</build_package_copy>

model_package = ml_client.models.begin_package(model_name, model.version, package_config)

The result of the package operation is an environment in Azure Machine Learning. The advantage of having this environment is that each environment has a corresponding docker image that you can use in an external deployment. Images are hosted in the Azure Container Registry. The following steps show how you get the name of the generated image:
1. Go to the Azure Machine Learning studio.
2. Select the Environments section.
3. Select the Custom environments tab.
4. Look for the environment named heart-classifier-mlflow-package, which is the name of the package you just created.
5. Copy the value that's in the Azure container registry field.
Now, deploy this package in an App Service.
1. Go to the Azure portal and create a new App Service resource.
2. In the creation wizard, select the subscription and resource group you're using.
3. In the Instance details section, give the app a name.
4. For Publish, select Docker container.
5. For Operating System, select Linux.
6. Configure the rest of the page as needed and select Next.
7. Go to the Docker tab.
8. For Options, select Single Container.
9. For Image Source, select Azure Container Registry.
10. Configure the Azure container registry options as follows:
  1. For Registry, select the Azure Container Registry associated with the Azure Machine Learning workspace.
  2. For Image, select the image that you found in step 3(e) of this tutorial.
  3. For Tag, select latest.
11. Configure the rest of the wizard as needed.
12. Select Create. The model is now deployed in the App Service you created.
13. The way you invoke and get predictions depends on the inference server you used. In this example, you used the Azure Machine Learning inferencing server, which creates predictions under the route /score. For more information about the input formats and features, see the details of the package azureml-inference-server-http.
14. Prepare the request payload. The format for an MLflow model deployed with Azure Machine Learning inferencing server is as follows:
  
  sample-request.json
```
 {
	"input_data": {
    	"columns": [
        	"age", "sex", "cp", "trestbps", "chol", "fbs", "restecg", "thalach", "exang", "oldpeak", "slope", "ca", "thal"
    	],
    	"index": [1],
    	"data": [
        	[1, 1, 4, 145, 233, 1, 2, 150, 0, 2.3, 3, 0, 2]
    	]
	}
 }
```
1. Test the model deployment to see if it works.
```
cat -A sample-request.json | curl http://heart-classifier-mlflow-pkg.chinacloudsites.cn/score \
    --request POST \
    --header 'Content-Type: application/json' \
    --data-binary @-
```

Next step

Model packages for deployment (preview)

Package and deploy models outside Azure Machine Learning (preview)

Prerequisites

Prepare your system

Additional resources