A practical AIoT maturity model for engineering managers and CTOs, from basic IoT devices to autonomous AI-powered systems.
Read: The AIoT Maturity Model →From Utility Bill to Trusted AI Insight
The architecture behind an OCR-to-LLM API that turns electricity, gas, water and heating bills into explanations people can verify, and why structured data must come before the language model.
A utility bill contains everything a customer needs to know, yet almost nobody can read one. Meter readings, tariff components, standing charges, taxes, consumption periods and contractual footnotes are spread across pages of tables that were designed for regulators, not people.
So the obvious idea appears in every product backlog: upload the bill, let AI explain it. The obvious implementation pastes raw utility-bill OCR text into a language model. It produces a demo in a week and a liability in production.
This article walks through the architecture we recommend instead: an API-first document-intelligence pipeline where the bill is first converted into validated, traceable structured data, and the LLM is only allowed to explain values that have already been checked. It applies to electricity, gas, water and district heating alike, and to any product that needs to explain documents, not just display them.
Why nobody reads their utility bill
Even when every number is technically on the page, ordinary questions stay unanswered:
Traditional PDF portals merely display the bill. A document-intelligence system converts it into something people can understand, question and act on. That conversion is an engineering problem, not a prompt.
Product experience: upload a bill, get a verified explanation
From the user's side it is deliberately simple. They open an app or portal and upload a supplier PDF, a photo of a paper bill, a scan, or several smartphone-captured pages. The system then works through nine steps:
The follow-ups are where trust is won or lost. "Why did my total cost increase even though my consumption went down?" "How much could I save by reducing my consumption by 10%?" "Does the meter reading look consistent with the billing period?" The AI must answer from the validated data of that specific document, not produce a generic response.
Architecture: OCR-to-LLM pipeline with specialised layers
The client, whether a mobile app, website, customer portal or third-party SaaS, stays lightweight: it selects documents, uploads them, displays results and sends questions. Everything else lives behind the API, as a sequence of specialised layers:
Authentication, file-type validation, size limits, malware scanning, encrypted transfer, duplicate detection, plus a processing job per upload. Multi-page documents run asynchronously; the client follows along via polling, server-sent events or a webhook.
Page separation, rotation and perspective correction, cropping, contrast enhancement, noise reduction, blank-page removal, region detection. This stage decides whether a folded, shadowed smartphone photo is usable at all.
Characters become text, but text alone is not enough. The system must understand headings, tables, columns, label-value pairs, totals and footnotes, and preserve coordinates and page references so every extracted value can be traced back to its location on the page.
Utility type, language, provider or general format, and whether this is an invoice, annual statement, correction or estimate. Electricity, gas and water share concepts, but their schemas and business rules differ, so classification selects the right extraction configuration.
OCR results become a canonical data model that stays stable across supplier layouts. Country-specific formats, tariffs and terminology are configurable adapters, not rebuilds of the platform.
Document ├── Provider ├── Customer and account references ├── Billing period ├── Utility type ├── Meter information ├── Previous and current readings ├── Consumption + unit of measurement ├── Tariff components ├── Fixed charges / variable charges ├── Taxes, fees, discounts, credits ├── Payments ├── Total amount └── Due date
Validation: the trust layer between OCR and the LLM
OCR and AI extraction make mistakes. A production system therefore treats no extracted value as correct until it has been checked. The validation layer asks:
Every extracted field carries its value, source page and location, extraction method, confidence score, validation results and any alternative candidates. Low-confidence values are flagged, and the user is asked to confirm rather than letting the system silently guess.
↓ OCR text
↓ LLM
↓ answer
↓ OCR + layout recognition
↓ structured schema
↓ validation + calculations
↓ grounded LLM
↓ traceable answer
Deterministic calculation first. Then the LLM explains.
Several functions belong in ordinary, testable software, never in the model: consumption differences, total verification, price per unit, period comparisons, fixed-vs-variable splits, anomaly thresholds, averages, chart datasets, and what-if scenarios. The LLM receives these verified calculations as context and puts them into words.
Here is that idea as a working example: the scenario below is computed by deterministic code from a typical 2-person household (2,550 kWh/year at €0.31/kWh plus a typical German fixed charge of €168/year → €958.50), exactly as the calculation engine would do it:
Deterministic code calculates. The LLM communicates. Assumptions are shown, and the saving is labeled an estimate.
Domain knowledge on demand with RAG
Retrieval-Augmented Generation supplies what the bill itself cannot: explanations of billing terminology, tariff rules, unit definitions, regional regulations, efficiency guidance and approved recommendation templates. The model receives only what is relevant to the current question. A grounded response context combines five things:
What the user gets back
The explanation layer turns the structured result into language anyone can follow: a concise summary (period, total, consumption, changes, deadline), a plain-language breakdown of each charge, notable findings such as unusual consumption, a higher unit price, estimated meter readings or duplicate charges, and suggested next questions like "Why did my unit price change?" or "Show me which costs I can influence."
Crucially, the API returns both text and machine-readable visualisation data. The LLM never draws a chart; the backend returns structured chart definitions such as cost breakdown, fixed vs. variable, period comparison, meter-reading progression and potential savings, which each client renders in its own design system:
{
"summary": {
"billing_period": "2026-01-01 to 2026-03-31",
"total_amount": 428.50,
"currency": "EUR",
"consumption": 1240,
"unit": "kWh"
},
"explanation": {
"headline": "Your total cost increased mainly
because of a higher unit price.",
"sections": []
},
"visualizations": [
{ "type": "cost_breakdown",
"title": "How your total is composed",
"data": [] }
],
"warnings": [],
"suggested_questions": []
}
Follow-up questions without re-processing
After the initial analysis, conversation continues against the existing structured document. The PDF is never parsed twice. A question retrieves the validated bill data, the relevant calculation results and the applicable domain knowledge, then generates a grounded answer with sources and confidence attached.
User uploads PDF or photo
Client sends POST /v1/documents, and the API returns a job_id immediately.
Pipeline runs asynchronously
OCR → extraction → validation → analysis, with status via webhook or polling.
Client fetches the analysis
GET /v1/documents/{id}/analysis returns the explanation, chart data, warnings, suggested questions.
User asks a question
POST /v1/documents/{id}/questions returns a grounded answer from the stored, validated data.
Saving advice that can defend itself
Recommendations are only trustworthy when they are tied to evidence. The system distinguishes three kinds:
Every recommendation states why it is being made, which bill information supports it, whether the saving is measured or estimated, and which assumptions were used. Decision support, never uncertain advice presented as a guaranteed result.
Document-intelligence API, not a one-off feature
The backend is a reusable service, not a feature welded to one app. A small, boring, versioned surface:
POST /v1/documents
GET /v1/jobs/{job_id}
GET /v1/documents/{document_id}
GET /v1/documents/{document_id}/analysis
POST /v1/documents/{document_id}/questions
DELETE /v1/documents/{document_id}
Around that surface: versioned schemas, idempotency keys, asynchronous jobs, webhooks, tenant separation, usage tracking, audit logs, configurable data retention, deletion requests, language selection and provider-specific adapters. That last item is what lets one core platform serve many utility types and markets.
Privacy, security and responsible AI
Utility bills carry names, addresses, account identifiers, meter numbers, payment details and consumption histories. Privacy is an input to the architecture, not a review step at the end.
Testing every stage independently
A pipeline is only as trustworthy as its weakest stage, so each one is evaluated on its own terms:
The product is the pipeline
The valuable product is not the OCR engine, and it is not the chatbot. It is the complete, controlled pipeline between them: a secure API that transforms an unstructured document into validated data, reliable calculations, understandable explanations, useful visualisations and evidence-based recommendations.
That architecture is what lets an organisation add document intelligence to an existing app or portal without rebuilding anything. The frontend keeps owning the customer experience; the backend provides the intelligence, trust, security and scalability. Designing and implementing that backend, with mobile products, cloud platforms, structured data and responsible AI connected into a production-ready service, is exactly the kind of system we build.
Planning a document-intelligence feature?
We design and implement the complete backend behind systems like this: secure APIs, OCR and extraction pipelines, validation layers, grounded LLM services and cloud architecture, so your app or portal only needs to upload a document and render the result.
Book an architecture callMore articles
Design reliable UX for connected devices across hardware, apps, BLE, Wi-Fi, cloud, onboarding, errors, and recovery.
Read: UX for Connected Devices: Designing Experiences Across Hardware, Apps, Connectivity, and Cloud →Hardware companion apps: hard parts, partner skills, and questions that separate specialists from agencies.
Read: How to Choose the Right Companion App Partner for Your Hardware Product →