ValGuard vs structured outputs: schema validity is not policy

OpenAI Structured Outputs, Pydantic, and Instructor fix JSON shape. ValGuard adds business rules, policy packs, shadow rollouts, and an audit trail with rule IDs. Schema validity is necessary. It is not a full production policy.

ValGuard is a runtime validation and policy-enforcement layer for LLM calls and agent steps. Deterministic rules run on every completion before the next model, tool, or customer-facing action.

Quick answer

Use structured outputs and Pydantic when you need typed JSON in one language and one deploy unit. Add ValGuard when the same payload must obey cross-field math, PII packs, or shared policy across services. Keep both: parse first, then enforce rules on the request path.

Verdict

Structured outputs win on shape inside one client; ValGuard wins when policy must outlive a single schema and a single language.

What each is built for

Structured outputs / Pydantic / Instructor constrain generation or parse text into models. They answer whether the JSON is the right shape.

ValGuard answers whether the completion may proceed under named business and compliance rules. It runs as an OpenAI-compatible layer with shadow mode, block/re-ask/warn/log, and optional playbooks.

Comparison table

Where ValGuard is stronger

• Cross-document consistency, refund caps, and policy phrases that JSON Schema does not express alone.
• Same packs on Node, Python, and HTTP canvas nodes.
• Shadow → enforce with per-rule telemetry.
• Playbook export (valguard-playbook-template/1) with optional validators.

Where structured outputs / Pydantic are stronger

• No network hop and no vendor on the critical path.
• Excellent DX for typed models in one codebase.
• Free and local for unit tests and notebooks.
• Enough when shape alone is the contract and one service owns every call.

Cost and latency

Pydantic cost is CPU in your process. ValGuard adds about 0.36 ms p50 on the HTTP path (mocked upstream) plus plan fees: Developer $69, Growth $149, Production $399, Enterprise from $1499. Engine checks stay in microseconds. Model time dominates. Block/re-ask buffer streams. See methodology.

Failure example

Structured output returns valid JSON: refund_amount is a number and order_id is a string. The amount exceeds the captured charge. Pydantic is happy. Finance is not.

Polish invoicing flows hit the same gap: a vendor nip field is ten digits and passes str typing, but the checksum is wrong. Schema validators do not know NIP or PESEL rules unless you code them. ValGuard ships identifier packs with named rule IDs.

With ValGuard, a range, cross-field, or checksum rule fails. Re-ask once or block with a rule ID. The tool never runs. The structured output pillar covers related failure modes.

Code

What twenty lines of Pydantic cover, and where they stop:

from pydantic import BaseModel, conint, confloat
from typing import Literal

class Triage(BaseModel):
    team: Literal["billing", "fraud", "general"]
    urgency: conint(ge=1, le=5)
    confidence: confloat(ge=0, le=1)

Triage.model_validate_json(llm_text)  # ok for shape
# Not covered here: cross-document consistency, PII in free text,
# per-rule audit trail, shadow→enforce, same policy across services.

ValGuard gate after (or instead of) local parse:

curl -s https://api.valguard.ai/v1/chat/completions \
  -H "Authorization: Bearer $VG_API_KEY" \
  -H "X-VG-Agent: triage" \
  -H "Content-Type: application/json" \
  -d '{"model":"openai/gpt-4o-mini","messages":[{"role":"user","content":"Route this billing dispute"}]}'

Choose structured outputs / Pydantic if

• One language and one deploy unit own the contract
• Shape alone is enough for the next step
• You cannot accept a proxy hop on that path
• You only need CI and local tests for now

Choose ValGuard if

• Policy must be identical across services
• You need shadow mode before enforce
• Auditors want rule IDs, not prompt archaeology
• Cross-field and compliance packs matter as much as schema

Using both

Parse with Pydantic or Structured Outputs. Enforce with ValGuard on the hot path. See when JSON must be exact and provider updates that break structured output.

Objections

1. Why not only Pydantic? Because production breaks on rules outside the schema.
2. Latency. Quote three numbers; do not compare µs packs to model ms.
3. ValGuard down. Fail the call; build client fallback. Trust.
4. Data residency. SaaS by default; Enterprise self-host for VPC. No free local runtime.
5. False positives. Shadow first.
6. Misses. No rule signature means no catch; side effects off-path are out of scope.
7. Exit. Per-playbook / per-agent export today; org-wide policy file later.

FAQ

Is Instructor enough? For parsing into models, often yes. For shared runtime policy, no.

Do OpenAI Structured Outputs replace validators? They reduce malformed JSON. They do not encode your refund policy.

Where should I read more? LLM output validation and Pydantic / structured output post.

Related

• vs Guardrails AI
• vs LLM-as-judge
• Validation
• Trust
• Pricing

Next step

Quickstart plus validate LLM output.