Execute Azure Machine Learning pipelines in Azure Data Factory and Synapse Analytics

  • Article

APPLIES TO: Azure Data Factory Azure Synapse Analytics

Tip

Try out Data Factory in Microsoft Fabric, an all-in-one analytics solution for enterprises. Microsoft Fabric covers everything from data movement to data science, real-time analytics, business intelligence, and reporting. Learn how to start a new trial for free!

Run your Azure Machine Learning pipelines as a step in your Azure Data Factory and Synapse Analytics pipelines. The Machine Learning Execute Pipeline activity enables batch prediction scenarios such as identifying possible loan defaults, determining sentiment, and analyzing customer behavior patterns.

Syntax

{
    "name": "Machine Learning Execute Pipeline",
    "type": "AzureMLExecutePipeline",
    "linkedServiceName": {
        "referenceName": "AzureMLService",
        "type": "LinkedServiceReference"
    },
    "typeProperties": {
        "mlPipelineId": "machine learning pipeline ID",
        "experimentName": "experimentName",
        "mlPipelineParameters": {
            "mlParameterName": "mlParameterValue"
        }
    }
}

Type properties

Property Description Allowed values Required
name Name of the activity in the pipeline String Yes
type Type of activity is 'AzureMLExecutePipeline' String Yes
linkedServiceName Linked Service to Azure Machine Learning Linked service reference Yes
mlPipelineId ID of the published Azure Machine Learning pipeline String (or expression with resultType of string) Yes
experimentName Run history experiment name of the Machine Learning pipeline run String (or expression with resultType of string) No
mlPipelineParameters Key, Value pairs to be passed to the published Azure Machine Learning pipeline endpoint. Keys must match the names of pipeline parameters defined in the published Machine Learning pipeline Object with key value pairs (or Expression with resultType object) No
mlParentRunId The parent Azure Machine Learning pipeline run ID String (or expression with resultType of string) No
dataPathAssignments Dictionary used for changing datapaths in Azure Machine Learning. Enables the switching of datapaths Object with key value pairs No
continueOnStepFailure Whether to continue execution of other steps in the Machine Learning pipeline run if a step fails boolean No

Note

To populate the dropdown items in Machine Learning pipeline name and ID, the user needs to have permission to list ML pipelines. The UI calls AzureMLService APIs directly using the logged in user's credentials. The discovery time for the dropdown items would be much longer when using Private Endpoints.

Related content

See the following articles that explain how to transform data in other ways: