Deploy models for batch inference and prediction

This article describes what Databricks recommends for batch inference.

Batch inference using a Spark DataFrame

See Perform batch inference using a Spark DataFrame for a step-by-step guide through the model inference workflow using Spark.

For deep learning model inference examples see the following articles:

• Model inference using TensorFlow and TensorRT
• Model inference using PyTorch

Structured data extraction and batch inference using Spark UDF

The following example notebook demonstrates the development, logging, and evaluation of a simple agent for structured data extraction to transform raw, unstructured data into organized, useable information through automated extraction techniques. This approach demonstrates how to implement custom agents for batch inference using MLflow's PythonModel class and employ the logged agent model as a Spark User-Defined Function (UDF). This notebook also shows how to leverage Mosaic AI Agent Evaluation to evaluate the accuracy using ground truth data.

Structured data extraction and batch inference using Spark UDF

Get notebook

:::