Databricks makes native XML support GA for Auto Loader, read_files and COPY INTO

Databricks has made native XML file support generally available. According to the Databricks release notes entry dated October 29, 2026, teams can now ingest, query and parse XML files in batch or streaming workloads with Auto Loader, the read_files table-valued function and COPY INTO, and can parse XML strings stored in table columns with the from_xml and schema_of_xml SQL functions. The capability requires Databricks Runtime 14.3 or above and, as the documentation states, works without external jars.

For data teams that still receive XML from ERP systems, partner feeds, regulatory filings or older integration platforms, the GA status removes a long-standing reason to keep a separate parsing step outside the lakehouse. XML can now land in the same pipelines that already handle JSON and CSV, with the same schema inference, schema evolution and rescued data behavior that Auto Loader applies to those formats.

The practical change for analysts is downstream: XML sources that used to arrive in a reporting layer through custom scripts can be loaded into governed tables with SQL, then exposed to BI tools in the usual way. The rest of the work is configuration, and the most important setting is the row tag.

··········

⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣

WHAT DATABRICKS ANNOUNCED FOR NATIVE XML.

The release note is short and lists the supported surfaces precisely. XML support is generally available for batch and streaming reads through Auto Loader, read_files and COPY INTO, and the from_xml and schema_of_xml functions are available for XML stored as strings. The XML file format documentation adds the DataFrame API (spark.read with format("xml") and df.write for output) and SQL table definitions with CREATE TABLE ... USING XML.

The documentation also describes the parsing model. XML must be well-formed, and the rowTag option is required: the element named in rowTag becomes the top-level struct of each row, and its child elements become fields. A schema can be supplied or inferred. Databricks states that DTDs and external entities are filtered out because the parser only examines rowTag elements.

........

Ingestion path	How XML is selected	Typical use
Auto Loader	cloudFiles.format = xml	Incremental files in cloud storage
read_files	format => 'xml'	Ad hoc SQL queries on files
COPY INTO	FILEFORMAT = XML	Batch loads into a table
CREATE TABLE USING XML	OPTIONS with path and rowTag	SQL table over XML files
DataFrame API	format("xml")	Python or Scala notebooks and jobs
from_xml, schema_of_xml	SQL or PySpark functions	XML strings inside a column

........

The release note does not attach any price or separate charge to the feature. XML reads run on whichever cluster, SQL warehouse or serverless compute the job already uses, so the cost question is the usual one of compute consumption for that workload.

··········

HOW ROWTAG SHAPES THE DATABRICKS XML OUTPUT.

The choice of rowTag decides whether the result is one row per business record or one row containing an array. The documentation illustrates this with a reviews file: with rowTag set to reviews (the root element), the output is a single row with a review array column, each element a struct with the attribute and child fields; with rowTag set to review, the output has one row per review with top-level fields.

Attributes become fields with a prefix set by attributePrefix, and the documentation's examples use an underscore, so an id attribute appears as _id. Character data inside an element that also has attributes or children goes into the field named by valueTag, shown as _VALUE in the examples. The excludeAttribute option disables attribute handling entirely.

The official SQL example for one row per record looks like this:

SELECT * FROM read_files(
  '/Volumes/<catalog>/<schema>/<volume>/reviews.xml',
  format => 'xml',
  rowTag => 'review'
)

To check the logic on a small file before pointing a pipeline at production data, consider an illustrative orders feed with four order elements, each with an id attribute. One record carries a non-numeric amount and one carries an extra channel element that the target schema does not define.

<orders>
  <order id="A100"><customer>Northwind</customer><region>East</region><amount>1250.00</amount></order>
  <order id="A101"><customer>Contoso</customer><region>West</region><amount>980.50</amount></order>
  <order id="A102"><customer>Fabrikam</customer><region>East</region><amount>abc</amount></order>
  <order id="A103"><customer>Litware</customer><region>West</region><amount>2100.00</amount><channel>partner</channel></order>
</orders>

Read with rowTag set to order, a schema of _id string, customer string, region string and amount decimal(10,2), and a rescued data column enabled, the documented behavior gives the following expected result (computed locally in Python on the illustrative data):

........

_id	customer	region	amount	Goes to rescued data
A100	Northwind	East	1250.00	nothing
A101	Contoso	West	980.50	nothing
A102	Fabrikam	East	null	amount value abc
A103	Litware	West	2100.00	channel value partner

........

Summed by region on the parsed amounts, East totals 1,250.00 and West totals 3,080.50, for 4,330.50 across the file. The A102 amount is excluded from that total because it failed the type check, and a report built on this table would understate East unless someone reviews the rescued values.

··········

[[ADPLUS_300x250_1]]

SCHEMA INFERENCE AND RESCUED DATA IN DATABRICKS XML PIPELINES.

Auto Loader treats XML like JSON and CSV for inference: by default it infers every column, including nested fields, as a string. Setting cloudFiles.inferColumnTypes to true makes it infer data types. The DataFrameReader behaves differently and selects types from sample data, so the same file can produce different column types depending on the read path, a detail to settle before a BI model is built on top.

When Auto Loader detects a new column, the documentation says the stream stops with an UnknownFieldException. Before stopping, it infers the schema from the latest micro-batch and merges the new columns at the end of the schema, without changing the types of existing columns. The cloudFiles.schemaEvolutionMode option selects the evolution behavior, and schema hints can enforce known types with SQL schema syntax. The documented Auto Loader example combines these settings:

query = (spark.readStream
  .format("cloudFiles")
  .option("cloudFiles.format", "xml")
  .option("rowTag", "review")
  .option("cloudFiles.inferColumnTypes", True)
  .option("cloudFiles.schemaLocation", schemaPath)
  .option("cloudFiles.schemaEvolutionMode", "rescue")
  .load(inputPath)
  .writeStream
  .option("mergeSchema", "true")
  .option("checkpointLocation", checkPointPath)
  .trigger(availableNow=True)
  .toTable("reviews")
)

The rescued data column, named through rescuedDataColumn, keeps values that could not be parsed because the field is missing from the schema, has a mismatched type or differs in case from the schema's field names. It stores a JSON document with the rescued columns and the source file path; the file path can be removed with the spark.databricks.sql.rescuedDataColumn.filePath.enabled configuration set to false. With the rescued column enabled, type mismatches no longer drop records in DROPMALFORMED mode or fail them in FAILFAST mode. Only incomplete or malformed XML is dropped or fails. Fields whose case differs from the schema also go to the rescued column unless readerCaseSensitive is set to false.

Streaming XML feeds run on the same pipeline engine whose low-latency option is covered in Databricks real-time mode GA in Lakeflow pipelines.

··········

XSD VALIDATION AND NESTED XML PARSING IN DATABRICKS.

For feeds that come with an XML Schema Definition, the rowValidationXSDPath option validates each row-level record against the XSD. The documentation is explicit that the XSD does not change the provided or inferred schema. Records that fail validation are marked as corrupted and handled according to the corrupt record mode. A helper, XSDToSchema, can extract a Spark schema from an XSD, with support limited to simple, complex and sequence types and basic XSD functionality.

........

XSD type	Spark type
boolean	BooleanType
decimal	DecimalType
unsignedLong	DecimalType(38, 0)
double	DoubleType
date	DateType
dateTime	TimestampType
long, unsignedInt	LongType
Other types	StringType

........

XML that sits inside a string column, for example a payload stored by a message queue or an API log, is handled with schema_of_xml and from_xml. Both accept a single well-formed XML record, and rowTag does not apply to from_xml. In the default PERMISSIVE mode, malformed strings go into the field named by columnNameOfCorruptRecord and malformed fields are set to null; if that field is not in the schema, corrupt records are dropped. FAILFAST throws an exception instead.

SELECT from_xml('
  <review id="r001">
    <author>Alice</author>
    <rating>5</rating>
    <comment>Amazing stay, highly recommend!</comment>
  </review>',
  schema_of_xml('
  <review id="r001">
    <author>Alice</author>
    <rating>5</rating>
    <comment>Amazing stay, highly recommend!</comment>
  </review>')
);

Writing is supported as well. The DataFrame writer takes rootTag and rowTag, and arrayElementName (default item) names unnamed array elements. Databricks notes that an array whose elements are themselves arrays gains an extra nesting level when written from a DataFrame that did not originate from XML; files read from XML and written back keep their structure.

··········

⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁤

WHAT BI AND DATA TEAMS SHOULD CHECK BEFORE MOVING XML FEEDS.

The first check is the runtime. Clusters and jobs below Databricks Runtime 14.3 do not get native XML support, and pipelines built on the open-source spark-xml library, which Databricks documents separately in its archive, need to be tested against the native reader before a switch, since option names and inference behavior should be confirmed on real files.

The second check is the row tag and schema contract. Teams should decide whether each feed is read with inferred strings, inferred types or an explicit schema, and should fix that choice per table. Explicit schemas with a rescued data column make type problems visible instead of silently changing columns, and the rescued column itself can be monitored with a simple count of non-null values per load.

The third check concerns semantic layers. Once XML lands in Unity Catalog tables, it can feed metric views and BI models like any other table; the Unity Catalog metric view relationships Beta shows how Databricks is extending that layer for BI tools and AI assistants. AI agents that query lakehouse tables, such as the one described in OpenAI's Data agent for BigQuery, Snowflake and Databricks, only see what the ingestion step parsed, so rescued values never reach those answers unless they are reprocessed.

··········

DATABRICKS NATIVE XML GA: WHAT IS CONFIRMED AND WHAT TO TEST.

Confirmed by Databricks: general availability as of the October 29, 2026 release note; batch and streaming reads with Auto Loader, read_files and COPY INTO; the from_xml and schema_of_xml functions; a Databricks Runtime 14.3 or above requirement; no external jars; XSD row validation; rescued data handling; and XML writing through the DataFrame API.

Not stated in the announcement: any separate price, any file size limit specific to XML, or a migration timeline for the legacy spark-xml library. Behavior that depends on configuration includes whether columns are inferred as strings or typed, how schema evolution is handled in Auto Loader, and whether type mismatches surface as rescued values or as nulls. A short test on a representative file, with an explicit schema, rescued data enabled and a row count compared against the source, settles most of those questions before the first dashboard depends on the feed.

··········

FOLLOW US FOR MORE.

·····

DATA STUDIOS

·····

datastudios.org