Inside the document extraction pipeline behind CreditAI

Key takeaways

This blog covers the ingestion layer beneath CreditAI by Octus^®: How we turn scanned PDFs, image-heavy decks and low-quality tables into a clean, searchable index, and why we run our own OCR to do it across millions of pages.

• We run our own OCR on purpose: At our volume, managed OCR APIs cost 100 to 200 times more per page, and owning the stack lets us tune the part that decides credit answers, table and layout detection.
• Structure beats any single model: Per-document fan-out, idempotent re-runs and observable state are what let a small team operate millions of pages. Swap the model freely; the structure is the asset.
• Tables are first-class citizens: In credit, the gap between a right and a wrong answer is often one number in one cell, so we preserve tables as tables instead of letting OCR flatten them into scrambled prose.
• The ceiling is set at ingestion, not retrieval: No reranker surfaces a number that was never extracted, and no embedding represents a table that OCR shredded. Get the pipeline right and everything downstream gets easier.

“Our experience is that in credit, retrieval quality is capped by ingestion quality. ”

Introduction

At Octus, we build AI-powered intelligence for credit markets. Many of the answers our users need live in documents that have never touched a search index: Scanned PDFs, image-heavy presentations, files in many formats. AI agentic systems get most of the attention, but none of it works until the raw document has been turned into machine-readable text. At scale, that first step is a serious engineering problem.

That layer is the pipeline that ingests unstructured documents and turns them into the searchable index behind CreditAI. We describe how it works, why we run our own optical character recognition (OCR) models instead of calling a managed service, and what we learned from processing millions of pages.

Why credit documents are hard to extract

Credit documentation arrives in whatever form it was produced: Born-digital PDFs, scanned signature pages, Word documents, image-heavy presentations and 400-page PDFs with low-quality scanned tables. Three properties make this genuinely hard:

• No canonical format. Every document must be normalized before extraction can start.
• Tables carry the value. Pricing grids, amortization schedules, financial line items: The highest-value content in credit documents is tabular, and tables are exactly what naive OCR destroys.
• Layout decides extraction. A page is not just words. It contains paragraphs, tables, headers and figures, and each needs to be extracted differently. This is why object detection sits at the heart of our OCR: Getting every word right while getting the regions wrong still produces unusable output.

The architecture: Small jobs, large fan-out

Rather than one long-running process consuming a queue, every document travels through the pipeline as its own small, restartable job, with thousands in flight at once on cloud batch compute. A lightweight first pass determines which documents are new and worth processing (so a re-run never re-OCRs the archive). The heavy lifting then happens in three stages:

• Prepare: Download each document from its source, convert whatever arrived (Word, PowerPoint, HTML, plain text) into normalized PDF and stage it in object storage.
• Extract: The OCR stage, and the technical heart of the pipeline (more below). Every page produces plain text plus a JSON layout file with word-level bounding boxes.
• Index: Stitch per-page output back into documents, chunk, embed and upsert into our vector database.

Every stage emits status events to a message stream and metrics to our monitoring stack, so a single document’s journey, from pending OCR through fully indexed, is traceable end to end. When a conversion times out or a single scan is corrupt, that job fails alone and retries alone; the other jobs in flight are unaffected.

The main design lesson: The pipeline’s structure matters more than any single model in it. Fan-out with per-document isolation, idempotent re-runs, and observable per-document state is what lets a small team operate millions of pages. The OCR model can be swapped; the structure is the long-term asset.

Why we run our own OCR

The most common solution is to call a managed OCR API. We priced that path first and decided against it. We run our own models instead: an open-source two-stage OCR stack, with a detection model that finds the words on each page and a recognition model that transcribes them. The weights are baked directly into the container image, and pages are processed in GPU batches.

Four reasons:

• Cost at the scale we ingest. Cloud OCR pricing is reasonable for a small document flow and very expensive at our volume. When the unit of work is every page of every document across every source we ingest, plus everything new that arrives daily, per-page API pricing dominates the entire pipeline budget. Self-hosted models on batch GPU instances shift that cost to pure compute, and batch workloads are among the cheapest compute available.
• We can optimize where it matters to us. A managed API is a black box: you get its accuracy, on its document distribution, on its roadmap. Owning the stack means we can improve the components that matter most for credit documents, object detection above all. Table and layout detection is where deal documents are won or lost, and it is the piece we can retrain, tune, and swap independently of the rest of the pipeline.
• Data never leaves our network. Much of what we ingest is access-controlled. With OCR in our own containers, the bytes go from the source to our storage to our GPUs. No third-party document API touches them, which keeps the security review simple.
• Extraction, chunking, and embedding live in one pipeline. Because we own every stage, custom-configured chunking and embedding sit directly inside the pipeline rather than in a downstream service, and the output goes straight into our vector database through the same code path whether a single document arrives during live ingestion or a whole archive moves through bulk ingestion.

“If you are building document AI for this market, our advice is to invest in the pipeline before the models. ”

How we chose

We benchmarked the field before committing, on three axes: Quality, speed and cost. We tested a wide range of options: Managed services (AWS Textract, Bedrock document parsing, Mistral OCR, LandingAI, Unstructured.io), open-source stacks (DocTR, Docling, MarkItDown, YOLO + PaddleOCR) and multiple variants of our own stack, including different detection-recognition model pairings and reduced-precision versions of the front-runners.

We did not score quality with character-accuracy percentages. Every contender extracted the same set of real documents, and an LLM judge compared the outputs side by side, cataloging discrepancies in the categories that matter for credit work:

• numeric values, percentages and amounts
• identifiers and section references
• key financial and legal terms
• entity names, signatures and titles
• dates and periods
• table structure and completeness (missing rows, headers or whole tables)
• URLs, emails and contact details
• formatting and symbols

Each discrepancy was then graded by how severely it damages answer quality, trustworthiness, and completeness:

The high tier is the one that matters. A garbled word is obviously broken, so it does little harm. A misread digit still looks like valid data, gets indexed like valid data, and can be served back to a user as if it were correct. The failure modes we caught there were instructive: dropped rows and headers from tables embedded as images, entire tables flattened into a single line of text, and spurious symbols appended to dates and rates. Most discrepancies from every engine landed in the low tier; the high tier is where engines that looked comparable on raw text volume separated clearly. Every engine we tested, our own stack included, made high-severity mistakes somewhere. The real value of this catalog is knowing each engine’s failure patterns, which feeds our tuning priorities and tells the downstream pipeline what to double-check.

Here is how the options compared overall:

Cost was an even bigger separator. On our benchmark:

That is a 100-200x per-page difference that no volume discount closes. To be fair, in async mode the managed services are faster per page than our stack. Cost was the deciding factor.

Within the open-source stack we then ran a second sweep, testing every detection-recognition model pairing the ecosystem offers on the same benchmark. That is how we landed on our current pairing, and on half-precision inference to recover speed without giving up the accuracy that won the sweep.

The trade-off of going self-hosted is that performance engineering becomes our problem. We have invested in it: half-precision inference, page-level batching tuned to keep GPU utilization high and model weights baked into the container image so jobs start cold without pulling artifacts. During R&D we also explored compiling the models with ONNX and TensorRT for a further speedup. This is the ongoing engineering cost of owning the stack.

What this means for AI in credit

Most of the public conversation about RAG quality happens at the retrieval end: embeddings, rerankers, fusion strategies. Our experience is that in credit, retrieval quality is capped by ingestion quality. No reranker can surface a number that was never extracted, and no embedding model can represent a table that OCR shredded.

If you are building document AI for this market, our advice is to invest in the pipeline before the models. Make ingestion idempotent, observable per document and cheap enough to re-run, because you will re-run it every time your extraction improves. Treat tables as first-class objects. And do the cost math on managed OCR at your real archive size before you commit either way.

This same index is what customers reach through Octus Direct Data Services and the Octus MCP Connector. The model choices, orchestration and tuning are our own, but the lessons are free.

More in the AI & Data Science Series

Blog Post

Aug 13, 2026

Finding the comparable: How Octus built covenant similarity search

Read More