The framing of "OCR vs LLM" is wrong, but it's a useful wrong framing because it surfaces the real question: where does each technology actually fail when applied to insurance documents, and what does a system that handles both failure modes look like?
I've been building document extraction pipelines professionally for several years, first in financial document processing and now focused on insurance. The naive approaches to both OCR-only and LLM-only document extraction fail in predictable ways. Getting to production-grade accuracy on insurance documents — particularly loss runs and specialty submission packages — requires understanding both failure modes and building an architecture that handles them.
What OCR actually does — and where it stops
OCR (Optical Character Recognition) is a text recognition technology. Given an image of a page, OCR identifies the characters and returns text strings. Modern OCR engines, particularly those trained on large corpora of printed and typewritten documents, achieve high character-level accuracy on clean, high-resolution inputs — error rates well below 1% per character on ideal inputs.
The failure modes for OCR on insurance documents are well-characterized:
Table structure recovery: OCR returns text, not structure. A loss run table with eight columns and fifteen rows of data gets returned as a stream of text — characters in left-to-right, top-to-bottom reading order. Reconstructing which text belongs to which cell requires a separate layout analysis step that OCR itself does not perform. Standard OCR APIs provide bounding box coordinates for detected text elements, but translating bounding box coordinates into "this number is in column 4, row 7" requires additional spatial reasoning that the OCR layer doesn't include.
Low-quality input degradation: Fax-transmitted documents, photocopies of photocopies, documents with background shading or watermarks, documents with handwritten annotations — all of these reduce OCR accuracy significantly. For loss runs that originated as printouts from 1990s-era carrier systems and have been scanned and rescanned multiple times over the years, OCR error rates can reach levels where the extracted text is unreliable for financial data purposes.
Context-free extraction: OCR doesn't know what the text means. It can accurately recognize "142,500" but cannot determine whether that number represents paid losses, incurred losses, ALAE, or a policy limit. Assigning semantic meaning to recognized text requires a layer beyond character recognition.
What LLMs do — and where they hallucinate
Large language models are fundamentally different from OCR. They don't "read" images in the same sense — they process text input and generate text output based on learned statistical patterns over large training corpora. When applied to document understanding, the most common patterns are: feed the OCR-extracted text (or the raw document if the model has vision input capability) to the LLM, and ask it to identify and extract specific fields.
The genuine capability LLMs bring is semantic understanding: the ability to recognize that "Net Incurred to Date" and "Total Losses as Reported" and "Gross Case Incurred" likely represent variants of the same underlying field, and to map them to the correct output schema field without requiring an explicit mapping rule for each variation.
The failure mode that matters most for financial document extraction is hallucination. Language models generate outputs by predicting the next token given the preceding context. When asked to extract a specific number from a document, the model will sometimes generate a plausible-looking number that does not actually appear in the source — particularly when the document is dense, the values are numerically similar, or the model lacks sufficient confidence in what it sees.
For loss run extraction, a hallucinated number in an incurred loss column is not a recoverable error. If the system reports $142,500 in incurred losses for accident year 2021 and the actual figure from the source document is $248,500, the underwriter working from the extracted data will price the risk incorrectly. That's a material error for a specialty lines submission, and it's exactly the class of error that would make a CUO unwilling to let an automated system populate pricing model fields directly.
The hybrid architecture that actually works
The architecture that achieves production-grade accuracy on insurance documents combines both layers with explicit grounding rules that prevent hallucination from affecting numeric output:
- OCR with layout analysis: A modern OCR engine that returns both character text and spatial layout information — paragraph blocks, table boundaries, cell coordinates. The layout analysis step reconstructs table structure from the spatial data, producing a structured table representation with explicit row and column assignments. This takes the unstructured text stream and turns it into a structured data object.
- LLM semantic mapping: The structured table output is passed to a language model with the specific task of mapping column headers to a target schema. "Net Incurred" → paid_incurred_usd. "Clmt No" → open_claim_count. The LLM is doing header interpretation, not number generation.
- Value grounding: The critical constraint — the numeric values in the output must be sourced from the OCR-extracted text, not generated by the language model. The LLM identifies which OCR-extracted text token represents each target field; it does not produce the value itself. Any output value that cannot be directly traced to an OCR-extracted token is flagged as unverified rather than passed to the pricing model.
- Confidence scoring: Each extracted field gets a confidence score reflecting the quality of the underlying OCR output, the clarity of the column header mapping, and any structural anomalies (missing headers, inconsistent column counts across pages, merged cells). Low-confidence fields are surfaced for underwriter review rather than silently populated.
The document quality pipeline
Before the OCR-LLM pipeline runs, there's a document quality assessment layer that significantly affects downstream accuracy. A 1998-vintage loss run that arrived as a faxed photocopy has different processing requirements than a 2024 PDF generated directly from a carrier's PAS. The quality assessment step determines:
- Whether the document requires image enhancement (contrast adjustment, deskewing, despeckling) before OCR
- Which OCR engine configuration is appropriate for the document type (printed vs typewritten vs handwritten elements)
- Whether the document is text-native PDF (where text layer extraction bypasses OCR entirely and produces higher accuracy) or image-native PDF (requires full OCR)
- Whether confidence thresholds should be adjusted based on document quality
A text-native PDF — a PDF generated by a modern carrier's PAS that contains actual text elements rather than images — can often be extracted with near-perfect numeric accuracy because the text extraction is reading the embedded character data rather than inferring it from pixel patterns. The OCR step is bypassed entirely for text-native content, which eliminates the character recognition error class completely.
What this means for vendor evaluation
When evaluating extraction systems for specialty lines use, the key accuracy questions are about failure handling, not best-case performance. Any system will look good on clean, well-formatted PDFs. The questions that reveal production capability:
- What happens when a column header is absent or ambiguous? Does the system flag it or silently assign a default?
- What happens when a table spans multiple pages with headers only on the first page? Can the column alignment be maintained across the page break?
- What is the numeric accuracy on low-quality inputs — documents below a defined image quality threshold?
- How are hallucination-risk outputs handled? Are LLM-generated values that can't be traced to source text blocked from populating pricing fields?
The framing of "OCR vs LLM" implies a choice between two alternatives. In practice, the question is how the two are combined and how their respective failure modes are handled. Insurance document extraction that handles the production case — which includes the bad scans and the ambiguous headers and the multi-page tables — requires being honest about both failure modes and building accordingly.
