Problem
An LLM will answer a question whether or not it actually knows the answer. Before a chat response reaches a visitor, something needs to check that every factual claim is backed by approved evidence, that the tone and content are appropriate for a public page, and that a claim's verb doesn't overstate what the evidence supports.
Data boundary
The evidence manifest is this portfolio's own public, generated evidence (backend/evidence/manifest.json). No resume text, contact PII, or internal notes are ever included in it or sent to a provider.
Pipeline
Question → two providers answer in parallel with structured JSON output → adjudicator parses and checks each candidate against the evidence manifest, safety policy, PII screen, and claim-boundary policy → highest-ranked admissible answer wins → validated response.
Validation rules
| Gate | Checks | Rejects |
|---|---|---|
| Evidence | Every cited evidence_id exists in the manifest | Fabricated or unsupported claims |
| Safety | Banned-phrase screen (tone, injection markers, internal notes) | Prompt injection, desperation language, layoff narrative |
| PII | Phone/email/street-address/rate pattern screen on the answer text | Hallucinated contact details |
| Claim boundary | Claim verb rank vs. evidence's own verb rank | "validated" upgraded to "owned" |
| Confidence | Self-reported confidence meets policy minimum | Low-confidence guesses |
Output
A response containing only answer, coverage, and citations -- no provider name, model id, or raw evidence id ever reaches a client.
Failure modes
- Both providers fail or produce unsupported claims → one static safe-fallback answer, never a blend of the two.
- A provider times out → the request still returns within its timeout budget with a clear, non-alarming message.
- A claim escalates beyond its evidence (e.g. "validated" → "owned") → rejected, not silently allowed.
What this proves
A working policy-plus-evidence gate in front of an LLM, with deterministic, testable rejection behavior -- not just a prompt asking the model to "be careful."
What it does not claim
This does not claim the underlying LLMs are incapable of producing unsupported content -- it claims the adjudicator, not provider refusal, is what blocks it from reaching you.
Suggested questions
- Hiring manager: "How does the evidence system prevent unsupported claims?"
- Technical recruiter: "What proof can I send to a client?"