This AI answer review worksheet separates format, schema, source relevance, factual support, completeness, uncertainty, and action risk so a pass on one check is not mistaken for proof that an answer is correct.
Scope: This is a review aid, not a benchmark, certification, or claim that a model has been tested. The examples below are synthetic teaching cases.
1. AI answer review worksheet: Record the review
Use the AI answer review worksheet to record each layer separately; do not merge format, factual support and action risk into one overall pass rate. Keep unknown checks visible.
- Review ID and date: ______________________________
- Reviewer or team: ______________________________
- Application and model version: ______________________________
- Prompt or task version: ______________________________
- Source documents and versions: ______________________________
- Output schema and version: ______________________________
- Intended user or decision: ______________________________
- Review type: routine sample / release check / incident / other
2. Check each layer separately
- Parsing: Can the output be read in the expected format? Result: pass / fail / not applicable. Notes: __________
- Schema: Are required fields, types, allowed values, and constraints satisfied? Result: pass / fail / not applicable. Notes: __________
- Source selection: Are the retrieved documents relevant and current for this task? Result: pass / fail / not applicable. Notes: __________
- Claim support: Does each material claim follow from the cited passage? Result: pass / fail / not applicable. Notes: __________
- Factual consistency: Do values agree with the source, units, and surrounding context? Result: pass / fail / not applicable. Notes: __________
- Completeness: Are required facts present and omissions visible? Result: pass / fail / not applicable. Notes: __________
- Uncertainty: Are ambiguous, conflicting, or missing details represented honestly? Result: pass / fail / not applicable. Notes: __________
- Action risk: Could an unsupported answer trigger a harmful or irreversible action? Result: pass / fail / not applicable. Notes: __________
A pass in one row does not imply a pass in another. Valid JSON can contain a false value; a correct value can still violate a required schema.
3. Log claims against evidence
For each important field or conclusion, record the output, the exact source and version, and the passage that supports it. Mark the claim supported, partly supported, unsupported, or contradicted. A citation or retrieval ID is not proof by itself: check that the referenced material supports that specific claim.
- Claim or field: __________ Output: __________ Source ID/version: __________ Passage/location: __________ Status: supported / partial / unsupported / contradicted
- Claim or field: __________ Output: __________ Source ID/version: __________ Passage/location: __________ Status: supported / partial / unsupported / contradicted
- Claim or field: __________ Output: __________ Source ID/version: __________ Passage/location: __________ Status: supported / partial / unsupported / contradicted
4. Synthetic examples: three different failure types
A. Valid shape, wrong fact
Synthetic source: “The sample package weighs 2 kilograms.” Synthetic response: {"weight_kg":20}. Parsing and a numeric schema may pass, but the factual consistency check fails because the value conflicts with the source.
B. Correct value, missing required field
Synthetic schema: both weight_kg and source_id are required. Synthetic response: {"weight_kg":2}. The value matches the example source, but schema validation fails because source_id is absent.
C. Citation present, support absent
Synthetic source A: “The package weighs 2 kilograms.” Synthetic source B: “The package is blue.” Synthetic response: {"weight_kg":2,"source_id":"B"}. The number matches source A, but source B does not support the weight claim. Mark the citation-support check as failed.
5. Decide what happens next
Save the completed AI answer review worksheet with the output, source versions, failed checks and release decision so the review can be repeated after a model or prompt change.
- Failed or unknown checks: ______________________________
- Human review required before use? Why? ______________________________
- Block release or escalate? ______________________________
- Proposed change and owner: ______________________________
- Retest date and version: ______________________________
- Decision: acceptable for stated use / revise and retest / reject or escalate / insufficient evidence
A small set of examples does not establish performance across users, topics, sources, or future model versions. Record how cases were selected and expand coverage when the decision requires it.
6. Summarize the review
Use a short, bounded statement in a release note or audit record: “We reviewed [number] outputs from [version] for [task] against [source set and date range]. We checked parsing, schema, source relevance, claim support, factual consistency, completeness, uncertainty, and action risk separately. The review found [results]. It covers [scope] and does not establish performance beyond that scope.” Keep the sample-selection method and unresolved issues with the record.
Further reading
Read WorldLensia’s guide to checking AI structured outputs against evidence. For JSON Schema object validation concepts, see the JSON Schema documentation.
Use the Build an AI Citation Audit: A Source Ledger That Catches Gaps to retain claim-level evidence, and return to the research directory for related guides.