BigQuery AI.PARSE_DOCUMENT is GA for parsing PDFs into SQL chunks

Google Cloud made the BigQuery AI.PARSE_DOCUMENT function generally available on November 2, 2026, according to the BigQuery release notes. The function sends documents stored in Cloud Storage to a Document AI layout parser and returns the extracted titles, headings, paragraphs, lists, tables and images as rows that can be queried, joined and stored with standard SQL. For a data team that keeps contracts, product sheets or reports as PDF files next to its warehouse, document parsing becomes one more table function in a BigQuery query, with no separate pipeline to schedule.

The output is chunked text. Each row carries a chunk ID, the first and last page the chunk covers, the extracted content and a status field that stays empty when the call succeeds and holds an RPC error message when it fails. Every column of the input table travels with the chunks, so the results can be filtered by file, folder or metadata and written straight into a table for search, embeddings or retrieval-augmented generation.

The GA status covers the BigQuery function. The parsing itself is done by a Document AI processor that the customer creates and pays for separately, and that processor carries its own page limits, versions and data-residency behavior, described below.

··········

WHAT GOOGLE ANNOUNCED FOR BIGQUERY AI.PARSE_DOCUMENT.

The November 2, 2026 release note states that AI.PARSE_DOCUMENT extracts layout elements such as titles, headings, paragraphs, lists, tables and images by using the Document AI layout parser to convert unstructured documents into structured data, and that the function is generally available. The same day's notes also list several other BigQuery releases, including general availability for data agent monitoring in Google Cloud Observability, for the AI and ML function support in Data Engineering Agent pipelines, and for remote catalog providers in the BigQuery lakehouse.

The function reference documents a short signature. The first argument is either a table, written as TABLE followed by the table name, or a query in parentheses. Three named arguments follow, and only the endpoint is required.

........

Argument	Type	Required	Documented behavior
TABLE or query	Table or query	Yes	Must include a column named ref of type OBJECTREF
endpoint	STRING	Yes	Layout parser resource path or full process URL
chunk_size	INT64	No	Parser chunk size, default 1000
connection_id	STRING	No	Cloud resource connection; end-user credentials if omitted

........

The reference lists five output columns added to the input columns: chunk_id (INT64, starting at 1), start_page and end_page (INT64), content (STRING) and status (STRING). A single chunk can span more than one page, which the start_page and end_page pair records. Each input document can have at most 130 pages.

··········

HOW AI.PARSE_DOCUMENT FITS INTO A BIGQUERY WORKFLOW.

Google's reference shows two ways to feed documents to the function. The first builds an object reference inline with OBJ.MAKE_REF, pointing at a single PDF in Cloud Storage. The second creates an external object table over a Cloud Storage path and passes the whole table. The example below follows the documented pattern; the bucket, dataset and processor ID are placeholders to replace with your own.

CREATE EXTERNAL TABLE mydataset.pdf_table
WITH CONNECTION DEFAULT
OPTIONS(
  object_metadata = 'SIMPLE',
  uris = ['gs://my-bucket/contracts/*']);

CREATE TABLE mydataset.parsed_chunks AS
SELECT * EXCEPT(ref)
FROM AI.PARSE_DOCUMENT(
  TABLE mydataset.pdf_table,
  endpoint => 'projects/123456789/locations/us/processors/123abc456def:process',
  chunk_size => 500
);

Before the first run, the Document AI API has to be enabled and a layout parser processor created in the project. If the query uses a connection, its service account needs the Document AI Viewer role (roles/documentai.viewer) on the project where the query runs. Without a connection, the user running the query needs that same role, since the call goes out with end-user credentials.

The persisted chunk table is where the analyst work starts. Because failures are reported per row in the status column, a quality check belongs right after every load. The following query counts chunks, failed chunks and successful chunks that came back with no text, and finds the highest page parsed.

SELECT
  COUNT(*) AS chunks,
  SUM(CASE WHEN status <> '' THEN 1 ELSE 0 END) AS failed_chunks,
  SUM(CASE WHEN status = '' AND content = '' THEN 1 ELSE 0 END) AS empty_chunks,
  MAX(end_page) AS last_page
FROM mydataset.parsed_chunks;

On a small illustrative table of five chunks from a four-page contract, where one chunk returned a DEADLINE_EXCEEDED error and one returned empty content, the query returns 5 chunks, 1 failed chunk, 1 empty chunk and a last page of 4. Those rows should be reprocessed or reviewed before the content reaches embeddings or a search index, because a retrieval system cannot answer from text that was never extracted.

For teams already building retrieval pipelines, Google's BigQuery tutorial on parsing PDFs for retrieval-augmented generation uses the related AI.CHUNK_DOCUMENT function, then AI.EMBED with the text-embedding-005 model, VECTOR_SEARCH and AI.GENERATE. The tutorial lists BigQuery, Agent Platform, Document AI and Cloud Storage as billable components. A similar in-warehouse approach on the Snowflake side is covered in our article on Snowflake AI_EXTRACT handling 1,000-page documents.

··········

DOCUMENT AI LAYOUT PARSER LIMITS, FILE TYPES AND VERSIONS.

Most practical limits come from the layout parser, and the Document AI documentation sets them separately from the BigQuery function. The layout parser documentation lists HTML, PDF, DOCX, PPTX, XLSX and XLSM as generally available file types subject to charges, with other types in preview. For PDFs it detects figures, paragraphs, tables, titles, headings, page headers and page footers, and for spreadsheets it reads tables with integer, float and string values in files of up to 5 million cells. Chunks are augmented with context such as ancestor headings and table headers, so a retrieved chunk still says which section and which table it came from.

........

Limit	Value documented by Google
Pages per document in AI.PARSE_DOCUMENT	130
Layout parser online request	15 pages (30 in imageless mode)
Layout parser batch request	500 pages
File size, online / batch	40 MB / 1 GB (general Document AI limits)
XLSX input	Up to 5 million cells

........

The 130-page cap in the BigQuery reference is the number to plan around. Long annual reports, technical manuals and bundled contract packs need to be split before they go into the object table, and the split points should fall on section boundaries so that chunks do not lose their headings.

Processor versions also deserve attention. The layout parser documentation lists a stable default version, pretrained-layout-parser-v1.0-2024-06-03, and several release-candidate versions built on Gemini 2.5, Gemini 3.0 and Gemini 3.1 Flash Lite models. Google notes that the v1.6 versions use the Vertex AI Gemini global endpoint, are not compliant with its data residency standards and may route requests anywhere globally. Teams with residency requirements should pin the processor to a version and location that meet them before pointing AI.PARSE_DOCUMENT at regulated documents.

··········

WHAT AI.PARSE_DOCUMENT COSTS ON A DOCUMENT ARCHIVE.

The BigQuery function reference does not state pricing. Google's Document AI pricing page, checked on November 3, 2026, lists the Layout Parser (including initial chunking) at $10.00 at list price, $9.00 with a one-year Gemini Enterprise Flexible Savings Plan and $8.00 with a three-year plan. The row does not print its unit; in the same table, Google's own pricing examples express the Form Parser price per 1,000 pages. The same page says failed requests with 4xx or 5xx response codes are not billed, and that page counting depends on format: one page per PDF page, one per PowerPoint slide, one per Excel tab and one per 3,000 characters for DOCX and HTML files.

The illustrative estimate below assumes the per-1,000-page unit that Google's examples apply to that table and excludes BigQuery processing and Cloud Storage charges.

........

Illustrative document set	Files	Pages each	Total pages
Vendor contracts	1,200	9	10,800
Product spec sheets	3,000	2	6,000
Annual reports	40	180	7,200
Total	4,240		24,000

........

At $10.00 per 1,000 pages, a one-time parse of the 24,000 pages costs $240.00; at the one-year and three-year savings plan rates it costs $216.00 and $192.00. The 40 annual reports exceed the 130-page limit, so each has to be split into two parts before parsing, which turns 40 files into 80 inputs without changing the page count. Re-parsing the whole archive after a change of chunk size or processor version repeats the full charge, so it pays to test settings on a sample folder first. Re-chunking documents that are already parsed is a separate Document AI SKU, listed at $0.02 per 1,000 count.

··········

WHAT CHANGES COMPARED WITH ML.PROCESS_DOCUMENT.

BigQuery already offered ML.PROCESS_DOCUMENT, which Google's comparison page describes as using Document AI processors for specialized processing of invoices, tax forms and financial statements, as well as chunking, with predefined structured output. For that function the page lists quotas of 120 requests per minute per processor type and 600 per project, and regional support in the EU and US multi-regions for all processors. AI.PARSE_DOCUMENT is narrower: it targets the layout parser, takes a table or query with an ObjectRef column, and returns a fixed chunk schema that drops directly into embedding and search steps.

Analysts who want a model to reason over a document's content still have the AI.GENERATE family of functions, and Google's guidance recommends Gemini-based functions for general language tasks and Document AI functions when a parsed, predefined structure is needed. Readers comparing in-warehouse parsing with chat assistants can find our comparison of which AI chatbots can extract tables from PDF files, and Google's other recent BigQuery release for spreadsheet users is covered in conversational analytics in Connected Sheets.

··········

BIGQUERY AI.PARSE_DOCUMENT: WHAT DEPENDS ON YOUR DOCUMENT AI SETUP.

Google confirms that AI.PARSE_DOCUMENT is generally available in BigQuery, that it accepts an object table or a query with an OBJECTREF column named ref, that it returns chunk ID, start and end page, content and status columns, and that each document is limited to 130 pages. The required Document AI Viewer role and the optional connection are documented in the function reference.

Cost, location and data handling depend on the Document AI processor you create. The layout parser is billed by Document AI at the rates on its pricing page, its versions differ in the Gemini model they use, and the v1.6 versions route through a global endpoint that Google says does not meet its data residency standards. The BigQuery reference does not list supported regions, quotas or BigQuery-side charges for the function, so those should be confirmed in the project's quotas page and billing reports during a pilot on a sample folder before a full archive is parsed.

··········

FOLLOW US FOR MORE.

·····

DATA STUDIOS

·····

datastudios.org