The Economics of Dirty Data: When to Deploy Vision AI and When It's a Trap

Part of a series on building cz-agents → under the hood

The unstructured-data trap

Most mid-sized companies sit on five to ten years of archived PDFs. Supplier invoices, contracts in shifting layouts, technical documentation, scanned forms from the migration off paper, exports from legacy systems that never received a structured-output story. The moment a request lands — "we need this searchable / fed into the warehouse / mined for reporting" — the same conversation begins with IT.

The estimate comes back at six to twelve months of work, plus two FTEs in permanent maintenance. The reason: no two documents are alike. OCR returns slightly different things from different scanners. One supplier's invoice layout shifts every year. A stamp across the text breaks regex. Each edge case demands a heuristic, each heuristic demands tests, each test demands monitoring in production.

This trap is widespread enough that an entire industry built up around it — from enterprise OCR vendors (ABBYY, Kofax) to RPA platforms promising "intelligent automation". The promised leverage is real. Realised value tends to be brutal. Long, expensive implementations, ROI that doesn't show up before year two, and the first edge-case scenario calls in consultants.

What Vision AI shifts in the last 18 months isn't necessarily output quality — that has been comparable to mature OCR vendors for a while. What shifts is unit economics and entry barrier. That's the reason it's worth writing a decision framework rather than another dev tutorial.

Two paths, different tradeoffs

To make the decision crisp, separate the two philosophies of approaching unstructured data:

• Classic ETL pipeline: An OCR engine (Tesseract, ABBYY) converts image to text; a parser (regex / heuristics / specific grammars) extracts structured fields; downstream validation handles edge cases. Most logic is deterministic, audit-friendly, infrastructure under your control.
• Vision AI API: A multimodal LLM receives the page as an image and a request for structured output. No explicit OCR step, no heuristics — the model handles stamps, two-column layouts, and fields that are stitched together "not in a table but in a sentence" in one go. The trade is vendor lock-in, opacity of decisions, per-call cost.

Comparison on the axes that actually drive the decision:

Neither path is universally better. The choice depends on which axes matter most to a given project — and that is not a technical question, it's a business one.

Case study: ISIR auction notices

For a concrete application of the framework, I'll use a domain where I have fresh data: ISIR, the Czech insolvency register. Inside a real-estate product, I needed to extract five fields from each new auction notice (cadastral district, parcel number, surface area, starting price, auction date), about 200 new PDFs per day.

ISIR doesn't standardise input. Each insolvency administrator typesets notices differently — one in Word, another in TeX, another scans a pre-printed form filled in by hand. Layout, fonts, terminology, section order — all of it shifts. Concrete situations where a classic parser breaks:

• An effective-date stamp slapped over the text — text extraction returns a blank line where the cadastral district should be.
• The notice is a scan, not a digital PDF — pdf-parse returns an empty buffer, OCR has to run separately.
• Parcel numbers in a two-column table — the text extractor smashes them into one line in random order.
• Surface area phrased differently by every office — "of an area of 348 m²", "area: 348", "land 348 m²".
• Occasionally a handwritten price correction in the bottom corner that OCR can't read.

You can tune five regexes for five notices. Not for 200 a day. This is exactly the type of task where "classic ETL" turns into a maintenance hell before MVP reaches production.

Applying the framework: when Vision AI pays off

Apply the table from the previous section to the ISIR case. The choice falls clearly on Vision AI, but it's worth understanding why, because the same decision would land elsewhere on a different dataset.

Upfront investment and time to MVP

For classic ETL, the minimum viable version means: build the OCR pipeline (Tesseract + post-processing), write a parser for at least the top 5 most-common layouts, test against historical data, push to production. A realistic estimate: 2–3 months of solo developer work. With Vision AI, the first working prototype was up in an hour — render page to image, send to API, validate schema. For a project with uncertain product validation, that acceleration is the ballgame.

Per-call cost vs. accuracy

A three-page notice at 200 DPI: roughly 4,500 input tokens and 800–1,200 output tokens on Claude Haiku 4.5. Batch-rate, that's pennies per document. At 200 notices per day, under USD 2 a month. For comparison: a GPU instance for an own OCR pipeline (even the smallest) would cost multiple thousands per month in infrastructure, plus maintenance time on every template change.

Per-call cost alone wouldn't save the decision if accuracy were poor. Real numbers from production: ~95% on clean digital PDFs, ~80% on dirty scans. For a product where every lead is reviewed by a human before any purchase decision, 80% is workable. For an automated invoice pipeline, 80% would be a disaster.

Auditability and vendor lock-in

Vision AI loses on this axis. ISIR data isn't used for compliance reporting where audit would be required. If it were, model output would have to be treated as "input that still must be validated independently", not as authoritative. Vendor lock-in on Anthropic API is acceptable because model output is portable — a Vision API from OpenAI or Gemini can be swapped for a few hours of refactoring.

Scalability

At 200 documents per day, linear per-call cost is a non-issue. At 100,000 per day, Vision AI would be more expensive than an own pipeline. The crossover keeps moving down with each generation of models — Haiku 4.5 is an order of magnitude cheaper than Opus 3 for the same task — but the crossover exists and decisions need to leave room for it.

Three principles for putting Vision AI in production

If the choice falls on Vision AI, three principles decide whether you ship a production system or a demo that breaks at the first real customer.

1. Model uncertainty in the UI and process, not in the prompt

The model can't reliably tell you "I wasn't sure about this field". The product wrapping it can. Per-field confidence (whether explicit from the model or via meta-validation against other sources), a pending_human_review queue, warning indicators in the UI, a link to the original document. End users don't have a problem with AI being occasionally wrong — they have a problem with errors that aren't visible. Trust isn't built by hiding limits, it's built by making them explicit.

2. Real cost of deployment isn't the cost of API

API costs are typically in the single to low double-digit percent range of TCO. Most of the cost is: human review of uncertain outputs, custom logic around retry/escalation, monitoring of model drift (behaviour can shift between model versions), schema validation, integration into business processes, end-user training. If the business case rests purely on "we save on the OCR licence", it usually isn't enough. If it rests on "we cut processing time from 5 days to 1 hour at comparable quality", it usually holds.

3. Default to the cheapest model, escalate only where it pays back

Haiku 4.5 handles 95% of routine documents. Sonnet and Opus only make sense where the error rate on Haiku is economically unacceptable — typically for high-value-per-document workloads (contracts, due-diligence reports). The strategy looks like: cheap model by default, escalate to a stronger one based on confidence threshold or domain flag. Without that layer, everyone "just to be safe" runs Opus on everything and the Vision AI economics stops working.

Closing: a decision, not a technology

The Vision AI vs. classic ETL choice isn't technical. It's a decision about which structure of tradeoffs fits the project — typically: how fast we need to be in production, how much data and what quality, how much error tolerance, what vendor lock-in is acceptable, what audit requirements look like. From the same technology menu, opposite decisions can be the right ones for different projects.

What rarely fits: defaulting to Vision AI because "it's new and modern", or defaulting to classic ETL because "AI isn't auditable". Both are heuristics standing in for thinking. The actual decision requires writing out the axes above, weighing them for the project at hand, and choosing on that basis. It takes an hour but saves months.