How We Stop Our AI From Inventing Numbers

TL;DR. Our model is forbidden from writing a single figure. It returns prose plus references, and the server materialises every number from the scan data. Two audits check the figures and the causal claims, one unbacked number voids the whole summary, and too many failures relabel the output instead of dressing it up.

Every AI product team eventually has the same meeting. Someone demos a beautiful generated summary, somebody else checks one of the figures in it, and the room goes quiet. We had that meeting. The fix we landed on was more aggressive than prompt engineering, and it is the single design decision we would recommend to anyone else building on a language model.

How do you stop an AI feature from inventing numbers?

You take away its ability to write one. In our scan review, the model is forbidden from producing a figure. It returns prose plus references, and the server materialises every number from the scan data before anything reaches a screen. A model that cannot type a digit cannot invent one.

This sounds obvious written down and it was not the first thing we tried. The first thing we tried was what everybody tries: a firm instruction in the prompt saying only use numbers present in the provided data. That works most of the time, which is precisely the problem. Most of the time is a great result for a chat toy and an unacceptable one for a number a founder is going to put in a board update.

So we changed the contract instead of the wording. The model receives the scan data and is asked for two things: a summary written in natural language, and a set of references pointing at the specific records it is describing. Where a figure belongs, it emits a reference, not a value. The server then resolves each reference against the actual data and substitutes the real number. The model shapes the sentence. The database supplies the fact. Neither is allowed to do the other's job.

The side effect nobody predicted is that the writing got better. Freed from having to be arithmetically careful, the model spends its attention on explanation and structure, which is what it is actually good at. We had been asking one system to be both a writer and a ledger, and it was mediocre at both.

What does the number audit check?

It walks every figure in the output and confirms each one resolves to a real value in the source data. Anything without a backing reference fails. Because the model is not allowed to write numbers in the first place, a failure means a pipeline defect rather than a bad sentence.

Belt and braces is deliberate here. The generation contract should make unbacked figures impossible, and the audit exists on the assumption that it will not always hold. Models find creative routes around constraints. A digit can arrive spelled out as a word. It can arrive inside a quoted fragment of a page title. It can arrive as a rounded restatement of something in the reference set that is close to the truth but not the truth. The audit catches the escapes.

What we like about this check is that it produces an unambiguous verdict rather than a confidence rating. Either every figure traces to a record or it does not. There is no threshold to tune and no judgement call at request time. Checks that produce a binary answer are checks you can actually enforce, and checks you can enforce are the only ones that survive a busy release week.

Why does one unbacked number invalidate the whole summary?

Because trust in generated text is not divisible by sentence. A reader who spots one wrong figure cannot tell which remaining statements are sound. Quarantining the offending line and publishing the rest asks them to do verification they cannot do, so we void the entire summary.

This was contested internally and it is the rule we are most confident about in hindsight. The argument against it is efficiency: one bad sentence out of a dozen means eleven good sentences get thrown away, which feels wasteful. The argument for it is what the alternative implies. If the system publishes partially audited output, it is telling the user that some unknown subset of the text is verified and inviting them to sort it out. That is not a product, that is homework.

There is also a discipline effect. When a single failure costs the whole response, nobody is tempted to let small ones slide. A per-sentence policy creates an acceptable error rate, and an acceptable error rate always drifts upward. An all-or-nothing policy creates a bug that has to be fixed. We would rather occasionally show less than routinely show something a customer has to double-check.

Why do we audit causation separately?

Because "because" is a claim. The section explaining what changed is the one where a model will reach for a causal link the data cannot support, so it gets its own audit. Phrases like due to and as a result of assert something a scan comparison rarely proves.

The temptation is structural rather than accidental. A model asked to explain a change will explain it, because explanation is what the task asks for. It has your fix ledger on one side and your movement on the other, and the most fluent sentence available connects them. Your citations rose because you published the comparison page. That sentence is enormously satisfying and frequently unprovable. A competitor may have gone quiet. A crawler may have refreshed. The question sample may have shifted.

So the causation audit reads that section specifically and asks, for each causal assertion, whether the underlying evidence supports a cause or only a co-occurrence. Where it only supports co-occurrence, the language is downgraded to describe what happened alongside what else happened. "Your citations rose, and in the same window you published a comparison page" is a weaker sentence and a true one. Our AI Visibility Score feeds this section, and it is exactly the surface where an overconfident causal story would be most expensive, because customers plan the next month around it.

We accept that this makes some output read as hedged. That is the correct trade. A tool that tells you what caused your movement when it does not know is not being helpful, it is being agreeable, and the difference shows up two months later when you have repeated a fix that never worked.

What happens when too many sections fail?

The output changes its own label. Rather than presenting a narrative it could not stand behind, the system states plainly that the review was written from the scan data. You still get every figure and all the evidence. What you do not get is a generated story pretending to be analysis.

Graceful degradation is the part of this design we think is most transferable. Most teams build the happy path and then treat failure as an error state to be hidden. But an AI feature has an unusual middle ground: output that is well formed, readable, and only partly trustworthy. If your only two options are ship it or show an error, you will ship it. A third option, ship it with an honest label, is what stops the pressure from resolving the wrong way.

The same instinct shows up elsewhere in the product. Our Blog Creator, when it runs autonomously, is grounded in four things: your own audited pages, which are the only URLs a post may link to, your measured standing, live topic research, and one open opportunity the post exists to close. It is explicitly barred from inventing percentages, study results, survey findings, quotes, named customers, case-study outcomes or dates. And when it estimates traffic for a content idea, it always returns a range rather than a point estimate, because a range is what we actually know.

What does this cost, and is it worth it?

It costs more engineering than the naive version by a wide margin. Reference resolution, two audits and a degradation path are considerably more work than one prompt and a render. The five questions the scan review answers would each have taken an afternoon without them.

Those five questions are worth naming, because they are the reason the discipline pays for itself: why you were cited, why competitors were preferred, what actually changes your citations, what your last changes did, and what to do next. Every one is a question someone will act on. Every figure inside them is clickable through to the evidence behind it. There is no version of that feature where an occasional invented number is an acceptable cost of shipping faster.

The honest accounting is that the expensive part is not the guardrails, it is the data model underneath them. You cannot materialise a figure from a reference unless the figure exists as a retrievable record with a stable identity. Most of the work was making the scan data addressable enough that a reference could point at something. Teams that try to bolt this on to a system where the numbers only exist inside a rendered response will find the guardrail is the easy half.

What should you copy if you are building an AI feature?

Four things, in order of value. Separate the writing from the facts so the model never authors a figure. Audit the output rather than trusting the instruction. Make audit failure loud and total. And build an honest fallback label so the failure path has somewhere honest to go.

The unifying principle is that prompts are requests and architecture is a guarantee. Every instruction you put in a system prompt is a preference the model will usually honour. If a mistake in some category is genuinely unacceptable to your users, the constraint belongs in the shape of the system rather than in the text you send it. Ask yourself what your feature would do if the model ignored your instructions entirely, and if the answer is embarrassing, you have a design problem rather than a prompting problem.

The second principle is that the fallback deserves as much design attention as the happy path, because it is the one your reputation rests on. Anyone can ship an impressive summary when everything works. What customers remember is what your product did the day it was not sure. Saying "this was written from your scan data" is a small, slightly disappointing sentence, and it is the sentence that makes the other outputs believable.

Frequently asked questions

How do you stop an AI feature from inventing numbers?

Remove the model's ability to write one. In the DataEase AI scan review, the model returns prose plus references and never a figure. The server then materialises every number from the underlying scan data before the text reaches a screen. A model that cannot type a digit cannot hallucinate one, which is a stronger guarantee than asking it nicely in a prompt.

What is a number audit in an AI feature?

A number audit is an automated check that walks every figure in generated output and confirms each one traces back to a real value in the source data. In DataEase AI, any figure without a backing reference fails the audit. Because the model is not permitted to write numbers at all, a failure means something went wrong in the pipeline rather than in a sentence.

Why does one unbacked number invalidate an entire AI summary?

Because trust in generated output is not divisible by sentence. A reader who finds one wrong figure has no way to tell which of the remaining statements are sound, so quarantining a single line while publishing the rest asks them to do verification work they cannot do. DataEase AI invalidates the whole summary rather than the offending sentence.

What is a causation audit?

A causation audit is a separate check on the section that explains what changed. Words such as because, due to, and as a result of assert a causal link, and a scan comparing two points in time can rarely support one. DataEase AI audits that section specifically and downgrades causal language to described change when the evidence only shows correlation.

What happens when too many sections fail the audit?

The output is relabelled. Instead of presenting a generated narrative, DataEase AI states plainly that the review was written from the scan data. The user still gets the underlying figures and evidence, but nothing is dressed up as analysis the system could not stand behind. Degrading the label is always preferred to degrading the truth.

What questions does the DataEase AI scan review answer?

Five, each with clickable evidence: why you were cited, why competitors were preferred, what actually changes your citations, what your last changes did, and what to do next. Every figure in those five answers is materialised by the server from the scan data rather than written by the model.