A validation layer that adds 200 ms to every call is a product you will turn off. A layer you cannot explain to an auditor is a product you will not ship. Both failure modes show up in vendor slides. This piece is the method behind the ValGuard benchmarks: how we measure validation latency, what the numbers include, and what they leave out.
The latest published snapshot is dated 16 September 2026 for the HTTP path, and 31 August 2026 for the playbook graphs. Those dates matter. Numbers drift when packs change. We republish rather than round.
Three benches, not one headline
We run three separate programs. Mixing them into a single "ValGuard is X ms" claim would be easier to market and less true.
The rule engine. go test -benchmem on every production template. This is CPU time inside the validator, with p50, allocations per operation, and bytes per operation. No sockets. No Postgres. No JSON HTTP envelope. A four-rule RAG source-matching pack sits at 0.52 µs p50 (1 alloc, 16 B). A twelve-rule invoice extraction pack sits at 21 µs p50. KYC cross-document consistency (kyc_cross_document_consistency_agent, 10 rules) is 14.2 µs. The clinical patient-facing guard used in prior authorization intake is 19.1 µs. These are the numbers that answer "does the check itself stall the request?"
The HTTP layer. End-to-end through the live OpenAI-compatible stack: auth, input prompt guard, a production output pack, persist queue. Concurrency 1. Median of five runs. Upstream LLM replaced by a zero-latency httptest server. The guarded path (input guard plus customer_support_reply, 7 rules) is 0.37 ms. That is ValGuard infrastructure. It is not GPT-4.
The playbook graph. orchestration-bench walks published 3 to 5 step graphs over X-VG-Flow with per-agent mocks. Linear, branch, merge, and fan-out. 100 iterations after 20 warmup. The 3-step support handoff (triage, compliance, reply, 3 validations) is 2.14 ms p50, 6.17 ms p95, 10.57 ms p99, about 367 graphs per second on that box. True fan-out with four validations converging at merge is 2.37 ms p50.
If a number in this article does not say which bench produced it, treat it as incomplete.
Environment and method
| Item | What we do |
|---|---|
| Engine | Go testing.B, -benchmem, p50 of the op |
| HTTP | Live web + engine path, concurrency 1, 5-run median |
| Playbooks | 100 timed iterations, 20 warmup, mocked agents |
| Upstream | Local zero-latency mock, not a paid model |
| Caps | No monthly budget cap, no per-agent token cap |
| Hardware | The production-class host used for published snapshots |
| Rounding | We keep unrounded p50. 0.369 ms stays 0.37 ms, not "sub-ms magic" |
We do not measure "user-visible chat latency." That number is dominated by the model. Adding our layer on a mocked upstream is how we isolate ourselves. If we included a real GPT-4-class call, we would be publishing the model's weather, not ours.
Industry writeups often quote ~820 ms as a median chat completion for that class of model. That figure is a baseline for scale, not a lab result from this host. We stopped putting it on the About page for that reason. It is useful only as a reminder: even if the model is hundreds of milliseconds, a 0.37 ms control plane is not the queue.
Load is a separate artifact. Orchestration concurrency sweeps at 1, 10, 50, 100, 500, and 1000 live on the scenarios page. This article stays on the isolated p50 path, because that is what a product manager asks first: "will customers feel this?"
What the microseconds mean on a real flow
Take a prior-auth intake. Extract clinical fields. Guard the patient message. Summarize for the reviewer or escalate. The patient-facing guard is 19.1 µs. The HTTP envelope around a similar pack is a fraction of a millisecond. The reviewer still takes seconds to open the next case. The patient still waits on the payer, not on our rule.
Take KYC document review. A nickname mismatch across passport and utility bill is a policy question, not a latency question. The consistency pack is 14.2 µs. If the bundle is ambiguous, the expensive part is the human who reads the handoff, not the check that built it.
Take a 3-step support playbook: classify, draft, send. The full graph with validation on every step is 2.14 ms p50 on the mocked path. The model still does the language. The layer decides whether the JSON is complete, whether PII leaked, whether the branch should be a human handoff instead of a send. Customers do not wait on that decision in any sense that shows up in a UX session. They wait on the model and on the business.
That is what "invisible" means here. Not that the layer does nothing. That it is faster than the noise already in the stack: TLS, JSON parse, the next await in the client, the time to first token from the provider.
Playbook shapes, same order of magnitude
The dedicated orchestration snapshot is the one to use when someone asks "what about multi-agent?"
| Graph | Validations | p50 | p95 | p99 | Notes |
|---|---|---|---|---|---|
| Linear support (triage → compliance → reply) | 3 | 2.14 ms | 6.17 ms | 10.57 ms | Happy path through end_success |
| Branch, validation pass | 2 | 1.23 ms | 2.85 ms | 3.87 ms | Classify then reply |
| Branch, validation block | 3 | 2.39 ms | 4.79 ms | 9.35 ms | Classify, draft, human handoff |
| JSON field route + merge | 3 | 2.22 ms | 4.06 ms | 5.47 ms | Route A; route B is 2.23 ms |
| Fan-out + merge | 4 | 2.37 ms | 4.45 ms | 5.49 ms | Wall clock ≈ slowest branch |
Fan-out does not triple the 2 ms. Parallel branches overlap. Merge waits on the slowest mocked agent plus the cross-check. That is the number we wanted from this bench: adding a parallel KYC or invoice branch does not turn the control plane into another model-length wait.
A block path is slower than a pass path in this table (2.39 ms vs 1.23 ms) because the graph has an extra step: the handoff agent. That is workflow cost, still milliseconds, still cheaper than a second LLM judge on every turn.
What we refuse to claim
We do not claim these numbers on a cold laptop against a live gpt-4o account. We do not claim p50 under a lock-convoy on a noisy shared VM. We do not claim that 0 allocs on a four-rule pack means the HTTP path allocates nothing. The HTTP path talks to Postgres and a queue. That is why it is 0.37 ms instead of 6 µs.
We do not claim that faster rules make a bad policy correct. A KYC name match that is too strict will still false-positive. It will just false-positive quickly. Shadow mode exists so you measure that before you block.
We do not claim a universal speedup versus "doing nothing." Doing nothing is 0 ms of validation and 100% of the model. The comparison that matters is versus the usual alternative: a second model that grades the first, or a canvas that treats an OpenAI node as a policy engine. A second completion is not in this band. Neither is n8n's IF node. ValGuard vs n8n, Portkey, and Guardrails AI is the product map. This page is only the stopwatch.
How a team should read this
If you own latency SLOs: budget the model. Treat ValGuard as rounding error next to TTFT, then confirm with a canary that has your real pack attached. If p95 of the playbook graph on your hardware is not in the same band as 6 ms, you have an environment issue, not a "rules are slow" issue. If the question is which step in a six-step agent blew the budget, that is a different article: latency budgets per step.
If you own risk: look at the pack times for the workflow you actually run. Invoice math at 21 µs. Coverage language at 6.3 µs. Wire-instruction checks at 10.4 µs. Those are cheap enough to run on every request, which is the only way an audit trail is honest.
If you own cost: a deterministic check is not an extra token bill. An LLM judge is. At 10k requests a day, a judge that even occasionally calls a mid-size model is the line item. The 2.14 ms graph is not.
The public tables live on benchmarks. The method text on that page matches this article. When the snapshot file changes, the numbers in posts that use bench tokens move with it. That is deliberate. A methodology page that fossilizes a lucky run is how you get marketing numbers.
The practical test is still the one we use internally. Put the pack on a flow you already run. Watch p50 of the request with the layer on and the model mocked. If that delta is a millisecond and the block reasons are ones a reviewer understands, the layer is doing the job. If you need a second model to tell you whether JSON had a required field, you are paying for drama the engine already settled in microseconds.