How to build a RAG flow with validation

1. 1How It Works

   A production RAG pipeline has (at minimum) three logically distinct jobs: retrieve relevant material, generate an answer using that material, and confirm the answer actually reflects the material. It's tempting to combine generation and validation into a single "answer well" prompt, but splitting them into a generation step you validate afterward is what makes the grounding checks meaningful. A validator can only check what's structurally separable, so an answer step producing a bare paragraph gives a validator nothing concrete to check against.

   Retrieval happens outside ValGuard, before the request. Your application (or vector database, or search index) does the actual similarity search and returns top-matching chunks. ValGuard does not replace this. It validates what happens after retrieval, once those chunks are in a request.

   Step 1: Normalize context (optional but recommended). A dedicated step that takes raw retrieved chunks and metadata and formats them into a stable, validator-friendly JSON envelope (consistent field names, length limits per chunk, chunk IDs the generation step will cite). This step is cheap and pays for itself the moment your retrieval source's output shape changes. The generation step's prompt stays stable even if the underlying retriever's raw format doesn't.

   Step 2: Generate the answer. A tightly-scoped prompt: answer using only the provided, normalized context, cite sources by the IDs from Step 1, and explicitly say when the context doesn't support an answer. Lower temperature settings generally help for factual use cases. Creative variance is exactly what you don't want here.

   Step 3: Validate citations and claims. Either a dedicated validation step, or (more commonly) a rich validator set attached directly to Step 2's output: rag_source_presence, rag_source_match, citation_format_valid, plus structural checks (required_fields, json_schema_validate), the exact set covered in depth in the prevent-hallucinations tutorial.

2. 2Prerequisites

   • A working retrieval pipeline (vector DB, search index, or any system that returns relevant chunks for a query). This lives in your own application, not in ValGuard
   • Familiarity with the validators covered in the prevent-hallucinations tutorial
   • An agent (or willingness to create one) per step: normalization (optional), generation, and, if you split it out, a dedicated validation step
   • A small set of test questions where you know the correct, source-backed answer, to verify grounding actually works

3. 3Step-by-Step Setup

   Step 1: Confirm your retrieval pipeline's output shape

   Before building anything in ValGuard, know exactly what your retriever returns per query: chunk text, chunk ID, source document name, relevance score. This is the raw input Step 1 (or Step 2 directly, if you skip normalization) will receive.

   Step 2: Create the normalization agent (optional)

   If your retriever's raw output is inconsistent or verbose, create an agent whose only job is reformatting it into a clean envelope: {"sources": [{"id": "doc_2", "text": "..."}]}. Apply field-presence and length-limit validators to this step.

   

   Step 3: Create the generation agent with a tightly scoped prompt

   Create the answer-generation agent with a system prompt along these lines: "Answer using only the provided sources. Cite each source by ID in the format [source: id]. If the sources don't contain the answer, say so explicitly rather than guessing." Set temperature low.

   

   Step 4: Attach grounding validators to the generation step

   Add required_fields (for answer + sources), rag_source_presence, rag_source_match, and citation_format_valid, the same set from the prevent-hallucinations tutorial, applied here inside an actual playbook rather than a standalone agent.

   

   Step 5: Wire the playbook: normalize → generate

   In Dashboard → Orchestration, create the graph with normalize_context → generate_answer, using append_previous on the generation step so it receives both the original question and the normalized sources.

   

   Step 6: Add a route for the validation-fail case

   Add a route from generate_answer on on_validation_block to either a reask (agent-level, configured on the rule itself) or a fallback step. A policy-answer-guarded-style playbook typically routes ungrounded answers to a safe default response ("I don't have enough information to answer that confidently") rather than surfacing a raw error.

   

   Step 7: Handle the empty-retrieval case explicitly

   Test what happens when your retrieval pipeline finds nothing relevant. The normalization step should produce an empty sources array rather than erroring, and your generation prompt should be explicit that an empty sources array means "say you don't know," not "answer from general knowledge."

   

   Step 8: Test with a genuinely wrong answer

   This is the step people skip and shouldn't: manually craft a completion (via the Playground, not a real model call) that answers incorrectly relative to the supplied sources, and confirm your validator set actually catches it. If it doesn't, your rule configuration has a gap, not your retrieval.

   

   Step 9: Simulate the full playbook end to end

   Run a real question with real retrieved context through Try it → Simulate and confirm both steps execute correctly and the final answer is properly grounded and cited.

   Step 10: Publish and call with X-VG-Flow

   curl -si "$VG_PROXY/v1/chat/completions" \
     -H "Authorization: Bearer $VG_API_KEY" \
     -H "X-VG-Flow: rag-policy-answer" \
     -H "Content-Type: application/json" \
     -d '{"model":"openai/gpt-4o-mini","messages":[{"role":"user","content":"What is our parental leave policy?\n\nRetrieved: doc_2: Parental leave is 12 weeks paid."}]}'

4. 4Diagram / Flow

5. 5Configuration Examples

   RAG playbook shape:

   id: rag-policy-answer
   graph:
     entry_step_key: normalize_context
     steps:
       - step_key: normalize_context
         node_type: agent
         agent_slug: rag-normalize
         input_mode: pass_through
       - step_key: generate_answer
         node_type: agent
         agent_slug: rag-answer
         input_mode: append_previous
       - step_key: safe_default_response
         node_type: terminal
     routes:
       - from_step_key: normalize_context
         to_step_key: generate_answer
         condition_type: always
       - from_step_key: generate_answer
         to_step_key: safe_default_response
         condition_type: on_validation_block
         is_fallback: true
   ```
   
   **Generation agent validators:**
   
   ```json
   {
     "validators": [
       { "name": "required_fields", "on_fail": "block", "params": { "fields": ["answer", "sources"] } },
       { "name": "rag_source_presence", "on_fail": "block", "params": {} },
       { "name": "rag_source_match", "on_fail": "block", "params": {} },
       { "name": "citation_format_valid", "on_fail": "warn", "params": { "pattern": "\\[source:\\s*\\w+\\]" } }
     ]
   }

6. 6Testing and Verification

   1. Grounded happy path. Real question, real retrieved context, confirm the answer cites a genuinely supplied source and passes every rule.
   2. Deliberately wrong answer (Playground). Prove the validator set catches an ungrounded claim before trusting it against real traffic.
   3. Empty retrieval. Confirm the flow produces a genuine "I don't know" rather than a hallucinated answer when no relevant context exists.

   

7. 7Troubleshooting

   "The generation step ignores the supplied sources and answers from general knowledge." Tighten the system prompt to be explicit and repeated: only use provided sources, and say so explicitly when they're insufficient. Lower temperature also helps reduce this drift.

   "Normalization step adds latency without obvious benefit." If your retriever's output is already stable and clean, you can skip Step 1 entirely and pass raw chunks directly into the generation step. Normalization is a recommendation for messy or evolving retrieval sources, not a mandatory step.

   "Empty retrieval produces a hallucinated answer instead of a refusal." Check your generation prompt explicitly handles the empty-sources case. A prompt that only says "use the sources" without addressing what to do when there are none will often default to answering from general training data instead.

   "citation_format_valid keeps failing on otherwise-correct answers." Verify the regex actually matches your prompt's requested citation format exactly. Small formatting drift (a colon vs. no colon, brackets vs. parentheses) is a common mismatch between what you asked for and what the pattern checks for.

8. 8Best Practices

   • Keep retrieval and generation cleanly separated, with ValGuard responsible only for the generation-and-validation half. Trying to make ValGuard "do retrieval too" is a scope mismatch.
   • Always test the empty-retrieval case explicitly. It's one of the most common real-world triggers for hallucination, and one of the easiest to test deliberately.
   • Route validation failures to a safe default, not a raw error, for user-facing RAG flows. "I don't have enough information to answer that confidently" is a better experience than an opaque validation error.
   • Lower temperature on the generation step for factual RAG use cases. Creative variance actively works against grounding.
   • Prove the validators catch a bad answer before trusting them against real questions. A rule set that's only ever been tested against correct answers hasn't been tested at all.

9. 9Advanced Options

   Multi-source cross-referencing. For questions requiring synthesis across several sources, the rag_answer_completeness_agent template (see the prevent-hallucinations tutorial) combined with a custom validator checking that claims are distributed across the relevant subset of supplied sources (not just any one of them) can catch answers that cherry-pick a single convenient source while ignoring contradicting ones.

   Reask on grounding failure. Rather than an immediate block or fallback, consider on_fail: reask on rag_source_match specifically. A model reminded that its citation was invalid sometimes self-corrects on retry, which may be a better user experience than falling straight to a safe-default response. Compare both against a real traffic sample before deciding.

   Multi-provider research draft pattern. For research-heavy use cases, some teams match sources with one model and synthesize the final answer with a different one. A more advanced orchestration pattern worth exploring once the basic three-step RAG flow here is solid.

10. 10Summary and Next Steps

    A production RAG flow is retrieval (yours) feeding into generation and validation (ValGuard's job): normalize context if your retriever's output is messy, generate with a tightly scoped low-temperature prompt, and validate citations and grounding before the answer ever reaches a user, with a deliberate, tested path for both the "wrong answer" case and the "no relevant context" case.

    From here, preventing hallucinations goes deeper on the specific grounding validators used here, and fallback logic is worth revisiting for designing the safe-default and escalation paths for a RAG flow with more nuance than a single fallback terminal.