XML format in Azure Data Factory and Synapse Analytics pipelines

APPLIES TO: Azure Data Factory Azure Synapse Analytics

Follow this article when you want to parse the XML files.

XML format is supported for the following connectors: Amazon S3, Amazon S3 Compatible Storage, Azure Blob, Azure Data Lake Storage Gen2, Azure Files, File System, FTP, Google Cloud Storage, HDFS, HTTP, Oracle Cloud Storage and SFTP. It is supported as source but not sink.

Dataset properties

For a full list of sections and properties available for defining datasets, see the Datasets article. This section provides a list of properties supported by the XML dataset.

Below is an example of XML dataset on Azure Blob Storage:

{
    "name": "XMLDataset",
    "properties": {
        "type": "Xml",
        "linkedServiceName": {
            "referenceName": "<Azure Blob Storage linked service name>",
            "type": "LinkedServiceReference"
        },
        "typeProperties": {
            "location": {
                "type": "AzureBlobStorageLocation",
                "container": "containername",
                "folderPath": "folder/subfolder",
            },
            "compression": {
                "type": "ZipDeflate"
            }
        }
    }
}

Copy activity properties

For a full list of sections and properties available for defining activities, see the Pipelines article. This section provides a list of properties supported by the XML source.

Learn about how to map XML data and sink data store/format from schema mapping. When previewing XML files, data is shown with JSON hierarchy, and you use JSON path to point to the fields.

XML as source

The following properties are supported in the copy activity *source* section. Learn more from XML connector behavior.

Supported XML read settings under formatSettings:

Mapping data flow properties

In mapping data flows, you can read XML format in the following data stores: Azure Blob Storage, Azure Data Lake Storage Gen2. You can point to XML files either using XML dataset or using an inline dataset.

Source properties

The below table lists the properties supported by an XML source. You can edit these properties in the Source options tab. Learn more from XML connector behavior. When using inline dataset, you will see additional file settings, which are the same as the properties described in dataset properties section.

XML source script example

The below script is an example of an XML source configuration in mapping data flows using dataset mode.

source(allowSchemaDrift: true,
    validateSchema: false,
    validationMode: 'xsd',
    namespaces: true) ~> XMLSource

The below script is an example of an XML source configuration using inline dataset mode.

source(allowSchemaDrift: true,
    validateSchema: false,
    format: 'xml',
    fileSystem: 'filesystem',
    folderPath: 'folder',
    validationMode: 'xsd',
    namespaces: true) ~> XMLSource

XML connector behavior

Note the following when using XML as source.

• XML attributes:

  • Attributes of an element are parsed as the subfields of the element in the hierarchy.
  • The name of the attribute field follows the pattern @attributeName.

• XML schema validation:

  • You can choose to not validate schema, or validate schema using XSD or DTD.
  • When using XSD or DTD to validate XML files, the XSD/DTD must be referred inside the XML files through relative path.

• Namespace handling:

  • Namespace can be disabled when using data flow, in which case the attributes that defines the namespace will be parsed as normal attributes.
  • When namespace is enabled, the names of the element and attributes follow the pattern namespaceUri,elementName and namespaceUri,@attributeName by default. You can define namespace prefix for each namespace URI in source, in which case the names of the element and attributes follow the pattern definedPrefix:elementName or definedPrefix:@attributeName instead.

• Value column:

  • If an XML element has both simple text value and attributes/child elements, the simple text value is parsed as the value of a "value column" with built-in field name _value_. And it inherits the namespace of the element as well if applies.

Related content

• Copy activity overview
• Mapping data flow
• Lookup activity
• GetMetadata activity