Validator reference

160 deterministic validators for LLM output

Find a rule by the failure it catches, then compare its default severity, failure action, and measured runtime. Validators check explicit contracts. They do not determine whether a model response is true or complete unless you encode that requirement as a rule.

Need setup details? See the validator configuration docs.

source: content/benchmarks/validator-rules-latest.json

Per-rule benchmarks are refreshed from the validation benchmark snapshot. Results depend on hardware and rule configuration.

Filter catalog

Structure

Allowed values (enum)

enum_validator

Field values must be one of the allowed options. Enable fuzzy repair per field to auto-fix near-miss typos when Response Repair is on.

pack: core · default severity: error · default on_fail: block

benchmark: p50 2.4 µs | 18 allocs/op | from 40 templates

Conditional fields

field_presence_rules

If field A exists, field B must also exist.

pack: core · default severity: error · default on_fail: block

benchmark: p50 14 µs | 37 allocs/op | from 1 template

Field types

type_checking

Each field must match the expected type (string, number, etc.).

pack: core · default severity: error · default on_fail: block

benchmark: p50 2.3 µs | 17 allocs/op | from 78 templates

JSON array max length

json_array_max_length

Cap the number of items in selected array fields.

pack: core · default severity: error · default on_fail: block

benchmark: p50 3.0 µs | 31 allocs/op | from 2 templates

JSON max depth

json_max_depth

Reject JSON nested deeper than max_depth levels.

pack: core · default severity: error · default on_fail: block

benchmark: not sampled yet

JSON Schema

json_schema_validate

Validate output against a JSON Schema object (type, required, properties).

pack: core · default severity: error · default on_fail: block

benchmark: p50 1.2 µs | 18 allocs/op | from 8 templates

List length

list_length

Array field must have between min and max items.

pack: core · default severity: error · default on_fail: block

benchmark: p50 1.7 µs | 11 allocs/op | from 18 templates

No duplicate JSON keys

no_duplicate_json_keys

Top-level JSON object must not repeat keys.

pack: core · default severity: error · default on_fail: block

benchmark: p50 2.2 µs | 14 allocs/op | from 66 templates

No extra JSON fields

json_no_extra_fields

Reject JSON objects with fields outside an allowed list.

pack: core · default severity: error · default on_fail: block

benchmark: p50 1.8 µs | 22 allocs/op | from 3 templates

Output shape

output_shape

Output must match json, list, table, or markdown shape.

pack: core · default severity: error · default on_fail: block

benchmark: p50 0.8 µs | 6 allocs/op | from 4 templates

Required fields

required_fields

All listed JSON fields must be present.

pack: core · default severity: error · default on_fail: block

benchmark: p50 2.4 µs | 17 allocs/op | from 80 templates

Valid format

valid_format

Output must be parseable in the selected format (JSON, YAML, XML, TOML, or CSV).

pack: core · default severity: error · default on_fail: block

benchmark: p50 2.2 µs | 17 allocs/op | from 94 templates

Format

Brackets balanced

bracket_balanced

Parentheses, brackets, and braces must match.

pack: core · default severity: error · default on_fail: block

benchmark: p50 4.1 µs | 8 allocs/op | from 2 templates

Code block languages

code_block_language_whitelist

Only allowed language tags in fenced code blocks.

pack: core · default severity: warn · default on_fail: warn

benchmark: not sampled yet

Code fences balanced

code_block_balanced

Triple-backtick code blocks must open and close.

pack: core · default severity: error · default on_fail: block

benchmark: p50 4.9 µs | 3 allocs/op | from 6 templates

Forbidden text

forbidden_patterns

Block output containing any of these phrases or patterns.

pack: core · default severity: error · default on_fail: block

benchmark: p50 2.4 µs | 11 allocs/op | from 98 templates

Keep keywords

keyword_preservation

Output must include all required keywords.

pack: core · default severity: warn · default on_fail: warn

benchmark: p50 6.2 µs | 1 allocs/op | from 1 template

Markdown images

markdown_image_well_formed

Images must be well-formed ![alt](url).

pack: core · default severity: error · default on_fail: block

benchmark: not sampled yet

Markdown links

markdown_link_well_formed

Links must be well-formed [text](url).

pack: core · default severity: error · default on_fail: block

benchmark: p50 0.8 µs | 3 allocs/op | from 4 templates

Markdown structure

markdown_format

Basic markdown formatting checks.

pack: core · default severity: warn · default on_fail: warn

benchmark: p50 0.8 µs | 2 allocs/op | from 6 templates

Markdown table columns

markdown_table_columns_consistent

Table rows must have the same number of columns.

pack: core · default severity: warn · default on_fail: warn

benchmark: p50 0.6 µs | 2 allocs/op | from 1 template

Maximum length

max_length

Limit total characters in the output.

pack: core · default severity: warn · default on_fail: warn

benchmark: p50 3.0 µs | 5 allocs/op | from 20 templates

Minimum length

min_length

Output must be at least this many characters.

pack: core · default severity: warn · default on_fail: warn

benchmark: p50 2.9 µs | 11 allocs/op | from 41 templates

Must match pattern

regex_pattern

Entire output must match a regular expression.

pack: core · default severity: error · default on_fail: block

benchmark: p50 13 µs | 15 allocs/op | from 2 templates

No markdown wrapper

no_markdown_fenced_wrapper

Block code fences or markdown headings wrapping raw output.

pack: core · default severity: error · default on_fail: block

benchmark: p50 2.0 µs | 17 allocs/op | from 73 templates

No repeated phrases

no_repeated_phrases

Same phrase cannot repeat too many times.

pack: core · default severity: warn · default on_fail: warn

benchmark: not sampled yet

Repetition limit

repetition_ratio_max

Cap overall text repetition ratio.

pack: core · default severity: warn · default on_fail: warn

benchmark: p50 3.3 µs | 5 allocs/op | from 2 templates

Required phrases

instruction_compliance

Output must include all required phrases.

pack: core · default severity: error · default on_fail: block

benchmark: p50 3.4 µs | 11 allocs/op | from 6 templates

Required sections

required_sections

Markdown headings that must appear in the output.

pack: core · default severity: error · default on_fail: block

benchmark: p50 3.2 µs | 11 allocs/op | from 9 templates

Sentence ending

complete_sentence_ending

Prose should end with proper punctuation.

pack: core · default severity: warn · default on_fail: warn

benchmark: p50 4.0 µs | 7 allocs/op | from 14 templates

Tone & style

tone_style_rules

Required opening/closing text and bullet count.

pack: core · default severity: warn · default on_fail: warn

benchmark: not sampled yet

Valid Base64

valid_base64

JSON string fields must be valid Base64.

pack: core · default severity: error · default on_fail: block

benchmark: not sampled yet

Valid email

valid_email_rfc5322

Fields must be valid email addresses.

pack: core · default severity: error · default on_fail: block

benchmark: p50 1.6 µs | 7 allocs/op | from 3 templates

Valid hex color

valid_color_hex

Fields must be #RGB or #RRGGBB colors.

pack: core · default severity: error · default on_fail: block

benchmark: not sampled yet

Valid IP address

valid_ipv4_or_ipv6

JSON string fields must be IPv4 or IPv6.

pack: core · default severity: error · default on_fail: block

benchmark: not sampled yet

Valid ISO date

valid_iso_date

Date/datetime fields in ISO 8601 format.

pack: core · default severity: error · default on_fail: block

benchmark: p50 2.9 µs | 17 allocs/op | from 15 templates

Valid JWT structure

valid_jwt_structure

Fields must look like header.payload.signature (no signature verify).

pack: core · default severity: error · default on_fail: block

benchmark: not sampled yet

Valid phone (E.164)

valid_phone_e164

Phone fields must look like +48123456789.

pack: core · default severity: error · default on_fail: block

benchmark: p50 1.7 µs | 8 allocs/op | from 2 templates

Valid semver

valid_semver

Fields must be semantic version strings.

pack: core · default severity: error · default on_fail: block

benchmark: p50 2.6 µs | 22 allocs/op | from 1 template

Valid URL

valid_url_rfc3986

Fields must be valid URLs.

pack: core · default severity: error · default on_fail: block

benchmark: not sampled yet

Valid UUID

valid_uuid

Fields must contain valid UUIDs.

pack: core · default severity: error · default on_fail: block

benchmark: p50 1.4 µs | 12 allocs/op | from 2 templates

Word count

word_count_range

Output word count must stay within min/max.

pack: core · default severity: warn · default on_fail: warn

benchmark: p50 2.8 µs | 10 allocs/op | from 7 templates

Logic

Amount requires currency

amount_requires_currency

When amount fields are present, currency must be a valid ISO 4217 code.

pack: core · default severity: error · default on_fail: block

benchmark: p50 2.7 µs | 23 allocs/op | from 1 template

Cross-field formula

cross_field_logic

Same as numeric formula — checks relationships between fields.

pack: core · default severity: error · default on_fail: block

benchmark: p50 0.9 µs | 5 allocs/op | from 1 template

Currency format

currency_format

Fields must look like valid money amounts.

pack: core · default severity: error · default on_fail: block

benchmark: not sampled yet

Date order

date_order_valid

Earlier date fields must not be after later date fields.

pack: core · default severity: error · default on_fail: block

benchmark: p50 3.5 µs | 39 allocs/op | from 1 template

Decimal places

decimal_precision_max

Limit decimal precision per numeric field.

pack: core · default severity: error · default on_fail: block

benchmark: p50 11 µs | 4 allocs/op | from 2 templates

Field dependencies

dependency_rules

Conditional field presence rules.

pack: core · default severity: error · default on_fail: block

benchmark: not sampled yet

Integers only

integer_only

Numeric fields must be whole numbers.

pack: core · default severity: error · default on_fail: block

benchmark: p50 1.2 µs | 6 allocs/op | from 2 templates

Line item math

line_item_math_valid

quantity × unit_price must equal line_total within tolerance.

pack: core · default severity: error · default on_fail: block

benchmark: p50 3.1 µs | 31 allocs/op | from 2 templates

Non-negative numbers

non_negative

Numeric fields must be ≥ 0.

pack: core · default severity: error · default on_fail: block

benchmark: p50 2.5 µs | 8 allocs/op | from 17 templates

Number range

range_validator

Numeric field must stay within min and max.

pack: core · default severity: error · default on_fail: block

benchmark: p50 1.8 µs | 17 allocs/op | from 13 templates

Numbers match input

numeric_match

Numbers in output must appear in the user message.

pack: core · default severity: warn · default on_fail: warn

benchmark: not sampled yet

Numeric formula

numeric_relations

Expression must hold (e.g. net + tax == gross).

pack: core · default severity: error · default on_fail: block

benchmark: p50 1.6 µs | 8 allocs/op | from 11 templates

Percentage format

percentage_format

Percentage fields within allowed scale.

pack: core · default severity: error · default on_fail: block

benchmark: not sampled yet

Sum consistency

numeric_consistency

Sum of fields should equal a total field.

pack: core · default severity: error · default on_fail: block

benchmark: p50 0.8 µs | 1 allocs/op | from 1 template

Security

Email domain denylist

email_domain_denylist

Block output containing emails from denied domains.

pack: core · default severity: error · default on_fail: block

benchmark: not sampled yet

Forbidden content

forbidden_content

Block output matching forbidden phrases or patterns.

pack: core · default severity: error · default on_fail: block

benchmark: p50 4.8 µs | 1 allocs/op | from 3 templates

Homoglyph detection

homoglyph_detection

Detect mixed-script homoglyphs in URLs or JSON string fields.

pack: core · default severity: error · default on_fail: block

benchmark: p50 1.8 µs | 7 allocs/op | from 2 templates

HTML no script

html_no_script

Block script tags and inline event handlers in HTML output.

pack: core · default severity: error · default on_fail: block

benchmark: not sampled yet

No stack trace leak

no_stack_trace_leak

Block stack traces, panics, and internal paths in user-facing output.

pack: core · default severity: error · default on_fail: block

benchmark: p50 12 µs | 18 allocs/op | from 2 templates

Path traversal

path_traversal

Block ../ and similar in paths.

pack: core · default severity: error · default on_fail: block

benchmark: p50 5.5 µs | 18 allocs/op | from 3 templates

PII detection

pii_detection

Detect personal data patterns in output.

pack: core · default severity: error · default on_fail: block

benchmark: p50 7.0 µs | 11 allocs/op | from 15 templates

Prompt injection

prompt_injection

Block prompt override attempts in output.

pack: core · default severity: error · default on_fail: block

benchmark: p50 6.2 µs | 18 allocs/op | from 8 templates

Secrets detection

secrets_detection

Block API keys and secrets in output.

pack: core · default severity: error · default on_fail: block

benchmark: p50 6.3 µs | 18 allocs/op | from 15 templates

Sensitive topics

sensitive_content

Block output mentioning forbidden topics.

pack: core · default severity: error · default on_fail: block

benchmark: p50 5.5 µs | 2 allocs/op | from 11 templates

SQL injection

sql_injection

Block SQL-like injection patterns.

pack: core · default severity: error · default on_fail: block

benchmark: p50 6.3 µs | 2 allocs/op | from 3 templates

URL safety

url_safety

Only allow links to approved domains.

pack: core · default severity: error · default on_fail: block

benchmark: p50 2.3 µs | 2 allocs/op | from 2 templates

XSS patterns

xss_pattern

Block script injection patterns.

pack: core · default severity: error · default on_fail: block

benchmark: p50 5.3 µs | 8 allocs/op | from 4 templates

Anti-AI

Limit hedging

no_hedging

Limit words like might, perhaps, maybe.

pack: core · default severity: warn · default on_fail: warn

benchmark: p50 2.0 µs | 6 allocs/op | from 3 templates

No apologies

no_apologies

Block unnecessary sorry/apology phrases.

pack: core · default severity: warn · default on_fail: warn

benchmark: p50 2.4 µs | 1 allocs/op | from 3 templates

No filler text

no_lorem_ipsum

Block lorem ipsum and dummy filler.

pack: core · default severity: error · default on_fail: block

benchmark: p50 3.8 µs | 3 allocs/op | from 1 template

No meta commentary

no_meta_commentary

Block intros like "Sure!" or "Here is your answer".

pack: core · default severity: error · default on_fail: reask

benchmark: p50 1.4 µs | 6 allocs/op | from 1 template

No placeholders

no_placeholder_text

Block TBD, N/A, unknown placeholders.

pack: core · default severity: error · default on_fail: block

benchmark: p50 3.2 µs | 8 allocs/op | from 14 templates

No prompt leakage

no_system_prompt_leakage

Block system prompt fragments in output.

pack: core · default severity: error · default on_fail: block

benchmark: p50 4.0 µs | 8 allocs/op | from 6 templates

No refusals

no_refusal_phrases

Block typical AI refusal phrases.

pack: core · default severity: error · default on_fail: reask

benchmark: p50 3.0 µs | 2 allocs/op | from 2 templates

No role markers

no_role_marker_leakage

Block User:/Assistant: labels in content.

pack: core · default severity: error · default on_fail: block

benchmark: p50 2.8 µs | 6 allocs/op | from 2 templates

No self-reference

no_self_reference

Block "as an AI" and similar phrases.

pack: core · default severity: error · default on_fail: reask

benchmark: p50 1.4 µs | 6 allocs/op | from 1 template

No thinking tags

no_thinking_tag_leakage

Block leaked chain-of-thought tags.

pack: core · default severity: error · default on_fail: block

benchmark: p50 6.1 µs | 3 allocs/op | from 3 templates

No truncation

no_truncation_markers

Block signs of cut-off output.

pack: core · default severity: error · default on_fail: reask

benchmark: p50 3.8 µs | 5 allocs/op | from 5 templates

Compliance

Age from birthdate

age_from_birthdate_in_range

Derived age must be within min/max.

pack: compliance · default severity: error · default on_fail: block

benchmark: not sampled yet

AI disclosure

ai_disclosure_required

Require disclosure phrases on customer-facing outputs.

pack: core · default severity: warn · default on_fail: warn

benchmark: not sampled yet

Business days only

business_day_only

Dates must not fall on weekends.

pack: compliance · default severity: error · default on_fail: block

benchmark: not sampled yet

Command injection

command_injection_pattern

Blocks shell injection patterns in output.

pack: compliance · default severity: error · default on_fail: block

benchmark: p50 5.5 µs | 35 allocs/op | from 3 templates

Date in range

date_in_range

Date fields must fall between min and max (ISO dates).

pack: compliance · default severity: error · default on_fail: block

benchmark: p50 2.6 µs | 26 allocs/op | from 1 template

Detect Medicare MBI

pii_medicare_mbi

Blocks Medicare Beneficiary Identifier patterns.

pack: compliance · default severity: error · default on_fail: block

benchmark: not sampled yet

Detect NIP (PII)

pii_nip

Blocks output containing valid Polish tax IDs (NIP).

pack: compliance · default severity: error · default on_fail: block

benchmark: not sampled yet

Detect PESEL (PII)

pii_pesel

Blocks output containing valid Polish PESEL numbers.

pack: compliance · default severity: error · default on_fail: block

benchmark: p50 6.0 µs | 1 allocs/op | from 2 templates

Detect Swiss AHV

pii_ahv

Blocks Swiss social insurance number patterns.

pack: compliance · default severity: error · default on_fail: block

benchmark: not sampled yet

Detect UK NHS number

pii_nhs_number

Blocks UK NHS number patterns.

pack: compliance · default severity: error · default on_fail: block

benchmark: not sampled yet

Detect US EIN

pii_ein_us

Blocks US Employer Identification Number patterns (##-#######).

pack: compliance · default severity: error · default on_fail: block

benchmark: not sampled yet

Detect US ITIN

pii_itin_us

Blocks US Individual Taxpayer Identification Number patterns.

pack: compliance · default severity: error · default on_fail: block

benchmark: not sampled yet

Detect US SSN

pii_ssn_us

Blocks US Social Security Number patterns.

pack: compliance · default severity: error · default on_fail: block

benchmark: p50 8.6 µs | 10 allocs/op | from 6 templates

EU VAT number

valid_vat_eu

Validate EU VAT number format on JSON fields.

pack: core · default severity: error · default on_fail: block

benchmark: not sampled yet

Financial disclaimer

financial_disclaimer_required

Requires disclaimer when investment keywords appear.

pack: compliance · default severity: error · default on_fail: block

benchmark: p50 2.8 µs | 2 allocs/op | from 11 templates

Future dates only

date_future_only

Date fields must be in the future.

pack: compliance · default severity: error · default on_fail: block

benchmark: not sampled yet

Geo coordinates in bounds

geo_coordinates_in_bounds

Latitude/longitude in JSON must stay inside a geographic bounding box.

pack: compliance · default severity: error · default on_fail: block

benchmark: not sampled yet

Geo country whitelist

geo_country_whitelist

Country fields must use allowed ISO 3166-1 alpha-2 codes (e.g. ship only to EU markets).

pack: compliance · default severity: error · default on_fail: block

benchmark: p50 14 µs | 37 allocs/op | from 1 template

Geo forbidden countries

geo_forbidden_countries

Blocks forbidden ISO country codes in output (sanctions / market restrictions).

pack: compliance · default severity: error · default on_fail: block

benchmark: not sampled yet

HIPAA PHI patterns

phi_us_hipaa_patterns

Detects common US protected health information markers.

pack: compliance · default severity: error · default on_fail: block

benchmark: p50 13 µs | 7 allocs/op | from 6 templates

ISO country code

valid_iso_country

Fields must be ISO 3166-1 alpha-2 (PL, US, DE…).

pack: compliance · default severity: error · default on_fail: block

benchmark: p50 3.3 µs | 15 allocs/op | from 2 templates

ISO currency code

valid_iso_currency

Fields must be valid ISO 4217 codes (USD, EUR, PLN…).

pack: compliance · default severity: error · default on_fail: block

benchmark: p50 2.2 µs | 8 allocs/op | from 15 templates

ISO language code

valid_iso_language

Fields must be ISO 639-1 (en, pl, de…).

pack: compliance · default severity: error · default on_fail: block

benchmark: p50 2.9 µs | 12 allocs/op | from 1 template

Masked card only

masked_card_only

Allow only masked card displays; block full PAN sequences.

pack: compliance · default severity: error · default on_fail: block

benchmark: not sampled yet

Medical disclaimer

medical_disclaimer_required

Requires disclaimer when health-related keywords appear.

pack: compliance · default severity: error · default on_fail: block

benchmark: p50 7.7 µs | 7 allocs/op | from 6 templates

No internal URLs

internal_network_url

Blocks links to localhost and private networks.

pack: compliance · default severity: error · default on_fail: block

benchmark: p50 5.0 µs | 9 allocs/op | from 2 templates

Past dates only

date_past_only

Date fields must be in the past.

pack: compliance · default severity: error · default on_fail: block

benchmark: not sampled yet

PCI-DSS patterns

pci_dss_patterns

Detects payment card sensitive data patterns.

pack: compliance · default severity: error · default on_fail: block

benchmark: p50 8.7 µs | 9 allocs/op | from 6 templates

Template injection

template_injection

Blocks server-side template markers.

pack: compliance · default severity: error · default on_fail: block

benchmark: p50 4.2 µs | 5 allocs/op | from 1 template

Timezone required

timezone_present

Datetime fields must include offset or IANA timezone.

pack: compliance · default severity: warn · default on_fail: warn

benchmark: p50 2.6 µs | 26 allocs/op | from 1 template

Unsubscribe link

unsubscribe_link_required

Marketing content must include opt-out wording.

pack: compliance · default severity: error · default on_fail: block

benchmark: not sampled yet

Valid card (Luhn)

valid_credit_card_luhn

Card number fields must pass Luhn check.

pack: compliance · default severity: error · default on_fail: block

benchmark: not sampled yet

Valid DE VAT ID

valid_de_vat_id

German USt-IdNr (DE + 9 digits) checksum validation.

pack: compliance · default severity: error · default on_fail: block

benchmark: p50 3.9 µs | 8 allocs/op | from 1 template

Valid IBAN

valid_iban

JSON fields must contain valid IBAN checksums.

pack: compliance · default severity: error · default on_fail: block

benchmark: p50 3.9 µs | 8 allocs/op | from 3 templates

Valid ISBN

valid_isbn

ISBN-10 or ISBN-13 checksum validation.

pack: compliance · default severity: error · default on_fail: block

benchmark: not sampled yet

Valid NIP

valid_nip

Structured NIP fields must have valid checksum.

pack: compliance · default severity: error · default on_fail: block

benchmark: not sampled yet

Valid PESEL

valid_pesel

Structured PESEL fields must have valid checksum.

pack: compliance · default severity: error · default on_fail: block

benchmark: not sampled yet

Valid postal code

valid_postal_code

Postal codes must match country format.

pack: compliance · default severity: error · default on_fail: block

benchmark: p50 1.7 µs | 7 allocs/op | from 1 template

Valid REGON

valid_regon

Polish REGON (9 or 14 digit) checksum.

pack: compliance · default severity: error · default on_fail: block

benchmark: not sampled yet

Valid US routing number

valid_us_routing_number

ABA routing transit numbers must pass the mod-10 checksum.

pack: compliance · default severity: error · default on_fail: block

benchmark: p50 3.9 µs | 8 allocs/op | from 1 template

XXE patterns

xxe_pattern

Blocks XML external entity markers in XML output.

pack: compliance · default severity: error · default on_fail: block

benchmark: not sampled yet

Code

CSV validity

csv_validity

Consistent CSV rows and columns.

pack: code · default severity: error · default on_fail: block

benchmark: not sampled yet

Function signature match

function_signature_match

Regex patterns for declared function signatures.

pack: code · default severity: error · default on_fail: block

benchmark: not sampled yet

Step count max

step_count_max

Limits numbered workflow steps in output.

pack: code · default severity: error · default on_fail: block

benchmark: not sampled yet

TOML validity

toml_validity

Parseable TOML document.

pack: code · default severity: error · default on_fail: block

benchmark: not sampled yet

Tool call args schema

tool_call_args_schema

Required arguments per tool (schemas map in JSON editor).

pack: code · default severity: error · default on_fail: block

benchmark: not sampled yet

Tool call whitelist

tool_call_whitelist

Only allowed tool/function names in tool_calls JSON.

pack: code · default severity: error · default on_fail: block

benchmark: p50 5.0 µs | 36 allocs/op | from 4 templates

Valid Dockerfile

valid_dockerfile

Requires FROM and known instructions.

pack: code · default severity: error · default on_fail: block

benchmark: not sampled yet

Valid GraphQL query

valid_graphql_query

Validates GraphQL operation structure.

pack: code · default severity: error · default on_fail: block

benchmark: not sampled yet

Valid JS/TS syntax

valid_js_syntax

JSON or bracket-balanced JavaScript in output.

pack: code · default severity: error · default on_fail: block

benchmark: not sampled yet

Valid Kubernetes manifest

valid_kubernetes_manifest

YAML manifest with apiVersion and kind.

pack: code · default severity: error · default on_fail: block

benchmark: not sampled yet

Valid OpenAPI spec

valid_openapi_spec

OpenAPI 3.x / Swagger document structure.

pack: code · default severity: error · default on_fail: block

benchmark: p50 12 µs | 94 allocs/op | from 2 templates

Valid package.json

valid_package_json

JSON with name and version fields.

pack: code · default severity: error · default on_fail: block

benchmark: not sampled yet

Valid Python syntax

valid_python_syntax

Structural Python checks (optionally from markdown code fences).

pack: code · default severity: error · default on_fail: block

benchmark: not sampled yet

Valid requirements.txt

valid_requirements_txt

Python dependency line format.

pack: code · default severity: error · default on_fail: block

benchmark: not sampled yet

Valid SQL syntax

valid_sql_syntax

Checks generated SQL for balanced syntax and recognizable statements.

pack: code · default severity: error · default on_fail: block

benchmark: p50 6.3 µs | 2 allocs/op | from 1 template

Valid Terraform HCL

valid_terraform_hcl

HCL block structure and delimiters.

pack: code · default severity: error · default on_fail: block

benchmark: not sampled yet

XML required paths

xml_xpath_required

Required element paths must appear in XML output.

pack: code · default severity: error · default on_fail: block

benchmark: not sampled yet

XML validity

xml_validity

Well-formed XML.

pack: code · default severity: error · default on_fail: block

benchmark: p50 4.4 µs | 13 allocs/op | from 1 template

XML XSD schema

xml_xsd_schema

Well-formed XML; full XSD when schema_path is configured.

pack: code · default severity: error · default on_fail: block

benchmark: not sampled yet

YAML validity

yaml_validity

Parseable YAML document.

pack: code · default severity: error · default on_fail: block

benchmark: p50 23 µs | 170 allocs/op | from 1 template

RAG

Citation format

citation_format_valid

Validate citation marker syntax and unknown source IDs.

pack: core · default severity: warn · default on_fail: warn

benchmark: p50 1.4 µs | 11 allocs/op | from 2 templates

RAG source coverage

rag_source_coverage

Output must cite all required source IDs.

pack: core · default severity: warn · default on_fail: warn

benchmark: p50 1.0 µs | 6 allocs/op | from 2 templates

RAG source match

rag_source_match

Terms in output must appear in input or RAG source text.

pack: core · default severity: error · default on_fail: block

benchmark: p50 0.6 µs | 1 allocs/op | from 2 templates

RAG source presence

rag_source_presence

Output must cite each source from X-VG-Sources (e.g. [1]).

pack: core · default severity: error · default on_fail: block

benchmark: p50 1.4 µs | 11 allocs/op | from 2 templates

Fidelity

Fact match

fact_match

JSON field values must match between input text and output.

pack: core · default severity: error · default on_fail: block

benchmark: p50 0.8 µs | 1 allocs/op | from 1 template

Input coverage

input_coverage

Output must include all required input fragments (defaults to input lines).

pack: core · default severity: warn · default on_fail: warn

benchmark: p50 0.8 µs | 1 allocs/op | from 1 template

Input/output diff

input_output_diff

Block numbers and terms in output that are not in input or RAG sources.

pack: core · default severity: error · default on_fail: block

benchmark: p50 0.5 µs | 1 allocs/op | from 1 template

Workflow

Branch outputs present

branch_outputs_present

Merged fan-out payload must include each expected branch step output.

pack: core · default severity: error · default on_fail: block

benchmark: p50 60 µs | 727 allocs/op | from 2 templates

Branch pass count

branch_pass_count

Require a minimum number of parallel branches to have passed validation.

pack: core · default severity: error · default on_fail: block

benchmark: not sampled yet

Cross-branch field match

cross_branch_field_match

Compare JSON fields across parallel branch outputs (e.g. invoice vs PO totals).

pack: core · default severity: error · default on_fail: block

benchmark: p50 60 µs | 727 allocs/op | from 2 templates

Multi-step numeric logic

multi_step_numeric_logic

Numeric rules across steps (e.g. sum(step.total) == step.grand_total).

pack: core · default severity: error · default on_fail: block

benchmark: not sampled yet

Required step order

required_step_order

Steps array must follow the configured order of step names.

pack: core · default severity: error · default on_fail: block

benchmark: not sampled yet

State transitions

state_transition_rules

Allow only configured from→to state transitions in JSON output.

pack: core · default severity: error · default on_fail: block

benchmark: not sampled yet

Step consistency

step_consistency

All objects in a steps JSON array must share the same keys.

pack: core · default severity: error · default on_fail: block

benchmark: not sampled yet