Extract data from PDFs
The OCR task: read PDFs with a vision model and land structured rows.
The OCR task reads PDFs with a vision model and writes structured rows into a lakehouse table. Point it at a folder of documents, declare the fields you want, and each run turns new PDFs into queryable records.
It's for the documents nobody wants to retype: course catalogs, invoices, statements, scanned reports — anything where the data is in a table on a page rather than in an API.
OCR carries a Beta badge in the task picker. It works and is supported; expect rougher edges than a Pipeline or SQL Script task.
How a run works
Every run:
- Picks up PDFs newer than the last successful run.
- Reads every page with a vision LLM.
- Appends one row per extracted record to the target table.
Layout doesn't matter. The model finds the data by meaning, so the same task handles documents where the table sits in a different place on each one. Already-processed documents are skipped automatically.
Setting one up
Add an OCR task to an automation stage and you get four sections to fill in.
1. Documents
- Document Source Connection — the file-storage connector holding the PDFs (S3, ADLS, SharePoint, Box, and so on).
- File Pattern — an exact filename or a glob within that connector's folder:
*.pdf,curriculum-*.pdf,Fall2026.pdf. - Archive PDFs for citations — optional; see below.
2. Fields to Extract
This is the part that decides whether extraction is any good. You declare the record shape: each field becomes a column, in the order you list them, and each carries a description that is sent to the model as extraction guidance.
The description isn't documentation — it's the instruction. "Course code" extracts less reliably than "The course code as printed in the left-most column, e.g. BIOL-201". A sentence of intent per field is what keeps extraction consistent across documents that are laid out differently.
3. LLM Connector
The model is pinned to the connector you choose here, and it must accept images — a current GPT-4o/GPT-5 class model or a current Claude. Calls go through the Databasin LLM proxy, so the same task runs on either provider.
Each extracted record carries the model's own extraction_confidence. Treat it as a triage signal for "which rows should a human look at" — it is not a calibrated probability.
4. Lakehouse Target
- Target Connection — the lakehouse the rows land in.
- Target Table — a fully-qualified
catalog.schema.table. It's created on the first run if it doesn't exist. - Write Mode — Append or Overwrite.
The created table holds your declared fields (as text) plus source_file, source_page, extraction_confidence, and run provenance.
Append vs. Overwrite
Append is the normal mode: each run adds the records from documents it hasn't seen before.
Overwrite clears the table first — and that has a consequence worth understanding before you pick it:
The run's memory of what it has already processed lives in the target table. Clearing the table clears that memory, so the next run re-reads and re-bills every matching document. Use Overwrite to deliberately re-extract; use Append for normal operation.
Archiving PDFs for citations
Turn on Archive PDFs for citations and each processed PDF is copied into your project's Databasin-managed document storage, with a link stamped into every extracted row as source_uri.
The payoff is in Databasin One: when the agent cites a number that came from an extracted row, it can open the exact source document at the exact page. Without the archive, you get the filename and page number but no link to follow.
The first time you enable it, Databasin provisions the project's document storage — you'll see a brief "Provisioning project document storage…" note.
What won't save
An OCR task refuses to save unless it has all three connections — document source, LLM connector, and lakehouse target — and at least one declared field. That gate exists because both failures used to surface at run time, which for an OCR task means after the model spend.
Cost, and keeping it down
Every page goes through a vision model, so cost scales with pages, not documents. Two things help:
- Narrow the file pattern.
invoices-2026-*.pdfbeats*.pdfif the folder holds history you've already processed. - Stay on Append. It's the mode that makes the "skip what we've seen" behavior work.