ValGuard.ai logoValGuard.ai

ValGuard + LiteLLM: validation closest to the model

Set ValGuard as api_base for a LiteLLM model alias so routing and fallbacks sit upstream while rules run on every completion.

Last verified:

LiteLLM routes, budgets, and fallbacks across vendors. Put ValGuard closest to the model by setting api_base on the alias you want guarded so every completion passes your rule packs.

When you need this

LiteLLM retries on any error. A ValGuard block looks like a failed completion unless your fallback rule keys off status codes. Separate provider failures from policy blocks in router config.

Setup

  1. Create ValGuard agents and packs.
  2. Add a LiteLLM model_list entry with api_base: https://api.valguard.ai/v1 and extra_headers for X-VG-Agent.
  3. Point application code at the guarded alias only for flows that need enforcement.
  4. Shadow mode first on traffic mirrors.

Code

model_list:
  - model_name: gpt-4o-mini-validated
    litellm_params:
      model: openai/gpt-4o-mini
      api_base: https://api.valguard.ai/v1
      api_key: vg_live_...
      extra_headers:
        X-VG-Agent: support-triage

import litellm

resp = litellm.completion(
    model="gpt-4o-mini-validated",
    messages=[{"role": "user", "content": "Hi"}],
)

Honest limits

  • LiteLLM fallback logic runs around ValGuard responses. Document which HTTP codes mean policy block vs provider outage.
  • LiteLLM guardrail plugins are separate from ValGuard. Do not assume both run unless you wire both.
  • Model time still dominates latency.

Related

Next step

Quickstart and methodology.