ValGuard.ai logoValGuard.ai
Operations~10 min read

How to close coverage gaps without blocking prod

Validation coverage shows where production traffic skipped part of the validator chain or never attached rules at all. This tutorial walks the Growth+ Validation coverage dashboard, reads gap rows without blocking customers, and closes the top issues using shadow mode and Simulate before enforce.

By the end, you will know:

  • How full-chain % is calculated and what partial vs no-validator rows mean
  • How to export CSV for a compliance worksheet
  • A ordered remediation path: routing first, then simulate, then shadow-to-enforce
  • Which playbooks match a 30-day gap program
UI note: In current UI you may see both labels: Playbooks and Orchestration (same section, depending on plan/version). Agent creation is in Dashboard -> Agents -> Create agent. If a guide mentions Validation logs, open Dashboard -> Logs and apply filters.

  1. 1# Close coverage gaps without blocking prod

    Blocking production to “fix governance” creates the incident you were trying to prevent. Coverage work should start read-only: measure full-chain execution, rank gaps by volume, fix bypass paths, then tighten rules under shadow and Simulate.

  2. 2Prerequisites

    • Growth plan or higher (analytics entitlement)
    • At least one agent receiving production traffic through ValGuard
    • Migration through 000105 applied (coverage columns on requests)

  3. 3Step 1 — Open Validation coverage

    1. Sign in to the dashboard.
    2. Go to Monitor → Validation coverage.
    3. Choose a window (7, 14, 30 days, or custom UTC range).

    Read the summary cards:

    CardMeaning
    Full chain% of requests with configured rules where every rule executed
    Partial executionCount of requests that stopped before the full chain
    No validatorsTraffic that hit the org with zero configured rules on the agent
    Shadow would-blockShadow-mode dispositions that would have blocked if enforcing

    Full-chain % is not “validators ÷ all requests.” High-volume unconfigured agents appear under No validators on purpose.

  4. 4Step 2 — Export and share the worksheet

    Click Export CSV. The file lists each agent with counts for total requests, with validators, full chain, partial chain, no validators, shadow would-block, and healed rows.

    Use this export for:

    • Monthly compliance attestation (counts only — no prompts)
    • Sprint planning (“top three agents by partial chain”)
    • Before/after snapshots when fixing routing

    The same data is available from GET /api/validation-coverage?format=csv for automation.

  5. 5Step 3 — Triage the gap list

    The Coverage gaps table highlights agents with:

    • Partial validator execution
    • No validators on meaningful volume
    • Elevated shadow would-block signal (rollout candidate, not a routing fix by itself)

    Click an agent name to open Validators for that agent.

    Remediation order:

    1. No validators — Confirm the route uses the proxy and X-VG-Agent (or playbook agent steps). Install coverage for new agent for greenfield agents.
    2. Partial chain — Look for early validator exit, mock/emergency bypass, or orchestration hops that skip ValGuard. Fix wiring before editing rule semantics.
    3. Shadow-heavy + full chain — Rules are running; use shadow counts to plan enforce. Run Simulate (test policy before production), then shadow mode rollout.

    Pillar for metric definitions: how to measure AI guardrail coverage. Why shadow ≠ coverage: shadow mode is not coverage.

  6. 6Step 4 — Run the 30-day playbook (optional)

    For a structured program, install coverage gap remediation:

    • Week 1: export + assign owners for top three agents
    • Week 2–3: routing fixes and validator catalog gaps
    • Week 4: shadow → enforce on one rule with Simulate proof (shadow-to-enforce rollout)

    Re-export CSV at day 30; full-chain % should rise on owned agents without a big-bang enforce flip.

  7. 7Step 5 — Pair with proof when auditors ask “did rule X run?”

    Coverage answers chain completeness at scale. When someone needs integrity on one request, use request detail Proof and verify guardrail proof.

  8. 8Related

What's next?