Keep the math out of the LLM: building AI features you can trust

Large language models are remarkably good at language. They are remarkably unreliable at arithmetic. That sounds obvious written down, yet a lot of AI features in production today ask a model to do both at once, and nobody notices the problem until a customer does.

We learned this first-hand on a forecasting system for an ad-tech platform. This post walks through what went wrong and the pattern we now use on every AI project.

The symptom: plausible, wrong numbers

The first version of the tool gave the model a campaign brief and asked it to produce a 30-day forecast: impressions, CPM, reach and how the budget should split across strategies. The output looked great. It was well formatted, confidently explained and internally consistent at a glance.

But when we checked the numbers against the underlying inventory data, some of them were simply wrong. Not wildly wrong, which would have been easy to catch, but plausibly wrong. That is the worst kind of wrong.

The fix: a hard boundary

We redesigned the system around one rule: the model never computes a number.

1. Deterministic engines do the math. Inventory waterfalls, CPM tiering and strategy allocation are ordinary, testable code. Same input, same output, every time.
2. The model classifies and writes. It maps a free-text brief to known categories, normalizes messy location text and writes the narrative that explains the forecast.
3. Guardrails sit between them. Model output must match a JSON schema. If it doesn't, we retry with adjusted parameters. Post-processing overrides any number that slips into the narrative, and invariant checks (totals add up, splits sum to 100%, values fall within known ranges) reject anything suspicious.

Fail loudly when data is missing

The second lesson: when real data isn't available, don't let the model fill the gap. If a third-party API returns nothing for a category, the job fails with a clear error instead of approximating. A forecast that refuses to run is annoying. A forecast that is quietly invented is dangerous.

A checklist for your own AI feature

• Is the model producing any number a human will act on? Move that calculation into code.
• Is the output structured? Validate it against a schema and retry on failure.
• Are there invariants (totals, ranges, allowed categories)? Check them automatically.
• What happens when upstream data is missing? Make it fail visibly.
• Can you trace every figure on screen back to deterministic code?

If the answers make you uneasy, talk to us. Making AI features trustworthy is a big part of what we do.