Enable diagnostic logging for Azure AI services

  • Article

This guide provides step-by-step instructions to enable diagnostic logging for an Azure AI service. These logs provide rich, frequent data about the operation of a resource that are used for issue identification and debugging. Before you continue, you must have an Azure account with a subscription to at least one Azure AI service, such as Speech Services.

Prerequisites

To enable diagnostic logging, you'll need somewhere to store your log data. This tutorial uses Azure Storage and Log Analytics.

  • Azure storage - Retains diagnostic logs for policy audit, static analysis, or backup. The storage account does not have to be in the same subscription as the resource emitting logs as long as the user who configures the setting has appropriate Azure RBAC access to both subscriptions.
  • Log Analytics - A flexible log search and analytics tool that allows for analysis of raw logs generated by an Azure resource.

Note

Enable diagnostic log collection

Let's start by enabling diagnostic logging using the Azure portal.

Note

To enable this feature using PowerShell or the Azure CLI, use the instructions provided in Collect and consume log data from your Azure resources.

  1. Navigate to the Azure portal. Then locate and select an Azure AI services resource. For example, your subscription to Speech Services.
  2. Next, from the left-hand navigation menu, locate Monitoring and select Diagnostic settings. This screen contains all previously created diagnostic settings for this resource.
  3. If there is a previously created resource that you'd like to use, you can select it now. Otherwise, select + Add diagnostic setting.
  4. Enter a name for the setting. Then select Archive to a storage account and Send to log Analytics.
  5. When prompted to configure, select the storage account and OMS workspace that you'd like to use to store you diagnostic logs. Note: If you don't have a storage account or OMS workspace, follow the prompts to create one.
  6. Select Audit, RequestResponse, and AllMetrics. Then set the retention period for your diagnostic log data. If a retention policy is set to zero, events for that log category are stored indefinitely.
  7. Select Save.

It can take up to two hours before logging data is available to query and analyze. So don't worry if you don't see anything right away.

View and export diagnostic data from Azure Storage

Azure Storage is a robust object storage solution that is optimized for storing large amounts of unstructured data. In this section, you'll learn to query your storage account for total transactions over a 30-day timeframe and export the data to excel.

  1. From the Azure portal, locate the Azure Storage resource that you created in the last section.
  2. From the left-hand navigation menu, locate Monitoring and select Metrics.
  3. Use the available drop-downs to configure your query. For this example, let's set the time range to Last 30 days and the metric to Transaction.
  4. When the query is complete, you'll see a visualization of transaction over the last 30 days. To export this data, use the Export to Excel button located at the top of the page.

Learn more about what you can do with diagnostic data in Azure Storage.

View logs in Log Analytics

Follow these instructions to explore log analytics data for your resource.

  1. From the Azure portal, locate and select Log Analytics from the left-hand navigation menu.
  2. Locate and select the resource you created when enabling diagnostics.
  3. Under General, locate and select Logs. From this page, you can run queries against your logs.

Sample queries

Here are a few basic Kusto queries you can use to explore your log data.

Run this query for all diagnostic logs from Azure AI services for a specified time period:

AzureDiagnostics
| where ResourceProvider == "MICROSOFT.COGNITIVESERVICES"

Run this query to see the 10 most recent logs:

AzureDiagnostics
| where ResourceProvider == "MICROSOFT.COGNITIVESERVICES"
| take 10

Run this query to group operations by Resource:

AzureDiagnostics
| where ResourceProvider == "MICROSOFT.COGNITIVESERVICES" |
summarize count() by Resource

Run this query to find the average time it takes to perform an operation:

AzureDiagnostics
| where ResourceProvider == "MICROSOFT.COGNITIVESERVICES"
| summarize avg(DurationMs)
by OperationName

Run this query to view the volume of operations over time split by OperationName with counts binned for every 10s.

AzureDiagnostics
| where ResourceProvider == "MICROSOFT.COGNITIVESERVICES"
| summarize count()
by bin(TimeGenerated, 10s), OperationName
| render areachart kind=unstacked

Next steps