Structured outputs from an AI system can be valid JSON, match a schema and still contain an incorrect date, invented source or wrong total. They improve the shape of a record; factual validation checks whether each value represents its source correctly.
This distinction matters when software automatically accepts extracted information. A tidy object can look more trustworthy than a paragraph, especially when every field has the expected name. The appearance is useful for processing, but it does not remove the need to verify meaning.
Separate three layers of checking
The first layer is syntax: can a JSON parser read the response? The second is schema: does the object contain the permitted fields with the required types and restrictions? The third is evidence: are those values correct for the source and task?
JSON Schema’s object reference explains properties, required fields and restrictions on additional properties. [1] Its numeric reference describes constraints on numerical values. [2] Such rules can reject malformed records and out-of-range values. They cannot independently establish that a number was copied from the correct document.
Consider a fictional invoice extraction workflow. A schema requires an invoice date, currency, subtotal, tax and total. The model returns every field in the correct type. It accidentally copies the delivery date into the invoice-date field. The schema may accept the response because both are legitimate date strings.
Build checks around the actual failure modes
| Layer | Example check | Failure it can catch |
|---|---|---|
| Parsing | The response is readable JSON | Missing brackets or invalid quoting |
| Schema | Currency is an allowed code | Unexpected free-text currency value |
| Arithmetic | Subtotal plus tax equals total | An inconsistent extracted total |
| Evidence | Date matches the labeled invoice date | Delivery date copied into the wrong field |
The example is illustrative. It shows why one green validation indicator should not stand for all four checks.
Arithmetic has its own limits. A fictional subtotal of 100 and tax of 10 produces a total of 110. If the actual invoice says 120, the fields can be internally consistent while still failing to represent the document. Reconciliation is a useful signal, not a substitute for reading the evidence.
Give unknown values an honest representation
A required field can encourage guessing if the workflow offers no way to represent uncertainty. Decide how missing, illegible and inapplicable values should appear. Those states have different meanings.
For example, a missing publication date should not automatically become today’s date. A missing amount should not automatically become zero. Use a documented null value or a separate status field when the application supports it. Then ensure downstream calculations handle that state explicitly.
Do not add a model-generated confidence number and treat it as a calibrated probability. It may help prioritize review only after its relationship to actual errors has been tested. A value such as 0.98 is still a claim about reliability, even inside a well-formed object.
Keep evidence attached to important fields
For research extraction, store the source URL or document identifier and a precise location such as a page, table or section. A record that says “source: government website” is difficult to audit. Our AI citation audit guide explains how a source ledger makes these checks repeatable.
Evidence should support the particular field. A real article URL does not validate a quotation that never appears in the article. A real financial report does not validate a number copied from a different reporting year. Check attribution, period, unit and scope together.
Version the extraction schema as well. If a field changes from annual revenue to quarterly revenue, existing records need a migration decision. Keeping the same field name while changing its meaning creates a silent comparability problem.
Route exceptions before automating action
Specify which failures block processing and which require human review. A harmless optional-field omission may need a different response from an unexplained payment-total mismatch. Log the decision and avoid repeatedly asking the model to produce a valid-looking replacement without preserving the original evidence.
A practical release test includes clean documents, difficult layouts, missing values, contradictory labels and examples outside the supported task. Measure factual errors as well as schema compliance. Report the number and selection of tested documents so a perfect score on an easy sample is not mistaken for production proof.
Structured records become useful when the system knows both how to read them and when to question them. The goal is a correct, traceable value in a predictable format, with an explicit route for cases the system cannot resolve.
Sources and method
- JSON Schema object reference. Living specification guide; publication date not established. Checked October 20, 2026.
- JSON Schema numeric reference. Living specification guide; publication date not established. Checked October 20, 2026.
Numerical teaching examples are original and illustrative unless explicitly identified otherwise. No hands-on product testing or individual medical assessment is claimed.
To test field-level evidence rather than syntax alone, see AI Answer Review Worksheet: Check Answers Against Evidence.