Automations 3 min read

The OCR task: turn a folder of PDFs into a lakehouse table

A new automation task reads PDFs with a vision model and lands structured rows in your lakehouse. You describe the fields you want in plain English; every page gets read, every row arrives with its source file, page, and a link back to the document. Now in beta.

Some data never arrives through an API. It arrives as a PDF: the course catalog, the invoice, the statement, the scanned report somebody emailed you. The numbers are right there, in a table, on page 14 — and the only integration anyone has ever offered you is a person retyping it.

OCR is a new task type in Automations. Point it at a folder of PDFs, describe the fields you want, and it writes rows to a lakehouse table on your schedule.

You describe fields, not layouts

There is no template to draw, no anchor text to configure, no zone to box off. Under Fields to Extract you list what you want and what it means:

  • course_codethe catalog identifier, e.g. BIOL 2010
  • credit_hoursnumber of credits; blank if the course is listed as variable

Those descriptions are the control surface. They go to the model as instructions, so the way you sharpen extraction is by saying what you mean more precisely — not by re-teaching a layout every time the vendor redesigns their PDF. Layout doesn't matter. The table can sit anywhere on the page, and it can sit somewhere else next quarter.

Every page, every run, with receipts

Each page is rendered and read by a vision model — there's no text-layer shortcut that quietly skips scanned documents. You pin which model does it, GPT or Claude, in the task's LLM connector.

Every row lands with provenance attached: source_file, source_page, source_last_modified, source_uri, extraction_confidence, plus the automation, task, and run that produced it. When a number looks wrong, you can get to the document it came from instead of arguing about it.

Turn on Archive PDFs for citations and the source document is stored alongside the data, so Databasin One can open the source document when it cites the row in an answer.

It only reads what's new

Runs are incremental by default, and the watermark is the output table itself — the task asks your own data what the newest document it has already read is, and starts there. No separate state to corrupt, nothing to reset. A document that fails doesn't advance the watermark, so it gets picked up on the next run instead of being silently skipped.

Long documents are handled in windows rather than one giant request, so a 200-page catalog is a normal input rather than a failure.

Fits where your other tasks fit

It's a task card on the same automation canvas as your SQL, dbt, notebook, and agent tasks. Chain it after a file drop, run it nightly, deliver what it produces however you deliver everything else. The output is an ordinary Iceberg table in your lakehouse — every engine and every dashboard can read it the moment it lands.

Beta, and honest about it

OCR ships as beta. Some specifics worth knowing before you point it at something important:

  • PDFs today. Not images, not Office documents.
  • Every extracted field is text. Casting to numbers and dates is downstream work, the same as any other raw landing table.
  • extraction_confidence is a triage signal, not a calibrated probability. Use it to sort what a human should look at, not to certify accuracy.
  • The target is your Databasin lakehouse in this version.

Documents are the last place where "we have the data" and "we can query the data" are still different sentences. This closes some of that gap — and it does it with the rest of your pipeline, on your schedule, in your warehouse.

Build one in Automations → add a task → Intelligence Tasks → OCR.

NewerPoint at a database, get a pipeline: ingestion that works itself out → News & insights ← Snowflake key-pair authentication is here — before you need itOlder

See it on your own data — five minutes, $50 in credit, no card.