Your JSON Is Legitimate however Your Knowledge Is Mistaken: 5 Failure Modes LLM Structured Outputs Will not Catch

Constrained decoding solved an actual drawback. Earlier than grammar-based strategies like Outlines and SGLang, getting legitimate JSON from a language mannequin was a retry loop. You prompted, parsed, caught the trailing comma, reprompted. Constrained decoding ended that: power token choice via a finite-state machine, and each output parses.

Groups adopted it quick. Schema compliance hit close to 100%. After which a quiet assumption crept into manufacturing codebases: if the JSON validates towards the schema, the information is right.

BAML’s benchmarks say in any other case. On function-calling duties, unconstrained era with post-hoc parsing reached 93.63% accuracy; constrained decoding on the identical mannequin scored 91.37%. The always-valid JSON was much less correct than the sometimes-broken JSON.

I began monitoring this after a classification pipeline I constructed started returning believable however fabricated values on roughly one in twelve runs. The JSON all the time parsed. Pydantic by no means complained. It took weeks to note, as a result of each downstream test was structural.

5 failure modes hold displaying up. All of them produce schema-valid output that breaks your pipeline silently:

1. Enum hallucination: legitimate enum, unsuitable which means

2. Assured fabrication: believable values in free-text fields

3. Cross-field contradiction: fields legitimate individually, not possible collectively

4. Distributional collapse: convergence on secure defaults

5. Array hallucination: fabricated entries as a substitute of empty arrays

Constrained Decoding: The Drawback It Truly Solved

Structured output used to imply hoping the mannequin behaved. Constrained decoding was constructed to repair that, and it did, simply not the entire drawback.

The development was actual. Immediate-and-pray JSON, the place you appended “reply in JSON format” and crossed your fingers, gave approach to regex-guided era (LMQL), then to grammar-based constrained decoding.

XGrammar, now the default backend for vLLM and TensorRT-LLM, provides near-zero overhead per token. The syntax drawback is solved.

However fixing syntax created a blind spot. Schema validation checks whether or not a subject is typed accurately: a string is a string, a quantity is a quantity. It says nothing about whether or not that string or quantity is right.

A lock on a submitting cupboard retains the drawers organized. It says nothing about whether or not the papers inside are correct. Schema validation works the identical approach.

The explanation this issues: forcing a mannequin right into a strict output format prices it one thing. It has to spend a part of its consideration on staying contained in the format, as a substitute of spending all of it on getting the precise reply proper.

Lee et al. measured that price straight. Throughout open-weight fashions, forcing structured output codecs produced a 3-to-9 share level accuracy drop. On math reasoning duties particularly, the place getting the reasoning proper issues greater than the format, the hole exceeded 15 share factors.

Tam et al. discovered the identical sample from a distinct angle: the stricter the formatting guidelines, the more severe the reasoning bought. The format isn’t free, and most groups aren’t accounting for that price.

5 Failure Modes: What Survives Your Schema

Schema validation catches sort errors. It doesn’t catch these 5 failure modes, as a result of each produces output that’s structurally legitimate and substantively unsuitable.

Enum hallucination. The mannequin picks a legitimate enum worth that’s semantically unsuitable for the enter. Think about a precedence enum of ["low", "normal", "high", "urgent"]: the grammar ensures a type of 4 values, but it surely doesn’t weight them by enter context. The mannequin can return “pressing” on a routine request or “low” on a important one, and the schema will settle for each.

Assured fabrication. Free-text fields return believable however invented information. BAML demonstrated this by submitting a photograph of an elephant as a receipt: constrained decoding returned an entire, schema-valid expense report as a substitute of refusing. Constrained decoding eliminates the mannequin’s skill to refuse or categorical uncertainty. The schema requires a price; the mannequin gives one, whether or not or not the enter helps it.

Cross-field contradiction. Schema validation checks every subject in isolation. It by no means checks whether or not the fields agree with one another. A sentiment extractor can return {"sentiment": "constructive", "rating": 0.1}, a constructive label with a rating near zero, which ought to imply unfavorable. A date parser can return {"begin": "2026-03-15", "finish": "2026-03-10"}, an finish date earlier than the beginning date. 

Each outputs go each particular person subject’s validation. Neither is sensible when you have a look at the file as an entire, and no single-field validator is constructed to catch that, as a result of the constraint lives between fields, not inside any one among them.

Distributional collapse. The mannequin converges on secure, generic values throughout completely different inputs. Constrained decoding biases towards high-probability tokens throughout the legitimate set, and “secure” defaults (0.95, “medium”, “normal”) carry larger base likelihood than context-specific values.

I caught this when confidence scores in a classification pipeline flatlined at 0.98 for 3 weeks. Collin Wilkins paperwork an analogous case the place confidence was 0.99 on each output, together with gibberish. Each file had legitimate sorts, right enums, cheap numbers. The distribution had stopped transferring, and nothing alarmed as a result of every particular person output was structurally right.

Array hallucination. Fashions resist returning empty arrays. Beneath constrained decoding, [] is a low-probability token sequence as a result of the grammar weights object-producing paths extra closely than the empty-array path. When a schema requires an objects subject of sort array, the mannequin fabricates entries fairly than returning nothing.

In extraction duties, this produces phantom outcomes: your pipeline stories “discovered 3 matches” when the proper reply is zero.

5 failure modes mapped to their observable alerts, root causes, and detection methods.

The Validation Entice: Why Extra Guidelines Will not Repair This

The primary intuition is to jot down extra validation guidelines. For identified patterns, that works. A Pydantic model_validator catches start_date > end_date. A customized test flags sentiment-score mismatches. You may construct cross-field constraints for each failure you have already seen.

The issue is the failures you have not seen. Structural correctness is closed: you may enumerate each legitimate JSON form for a given schema. Semantic correctness is open-ended. You may’t write a rule for a unsuitable reply you have not encountered but. In manufacturing, the mannequin finds new methods to be unsuitable quicker than you write validators, and every new validator solely covers the final bug.

There is a contrarian argument for resampling over constraining that deserves weight right here. As an alternative of forcing the mannequin’s output right into a grammar because it generates, let it write freely, then test the outcome: a parser validates the output towards the schema, and if it fails, the mannequin merely generates once more. That is resampling. Free-form era lets the mannequin purpose with out format stress; the format test occurs after, not throughout. BAML’s benchmarks present parse-and-retry outperforming constrained decoding by over 2 share factors on the identical mannequin.

You commerce assured first-pass parse success for larger accuracy when the output does parse. Whether or not that trade-off holds at excessive quantity, the place retries compound latency, continues to be an open query.

However the deeper drawback cuts throughout each approaches. Structured output hides uncertainty. When the schema requires a price, the mannequin fills it in. A risk_score subject all the time will get a quantity, even when the mannequin has no foundation for the evaluation. A abstract subject all the time will get textual content, even when the enter comprises nothing to summarize.

There isn’t a commonplace mechanism for the mannequin to precise “I have no idea” or “this subject doesn’t apply to this enter.” The schema is a forcing operate, and unsuitable solutions emerge with the identical confidence as proper ones.

After the third time I caught schema-valid-but-wrong output in manufacturing, I finished treating schema validation as a high quality gate and began layering semantic checks on high. Schema compliance is the ground, not the ceiling.

Three-Layer Protection: Schema, Semantics, Uncertainty

Layer 1: Schema and structural validation. That is what you have already got: Pydantic, JSON Schema, Zod. It catches sort errors, lacking fields, and syntactically invalid enum values. Preserve it. It solves the syntax drawback effectively.

Layer 2: Semantic validators. Cross-field constraint features that encode enterprise logic: “if sentiment is constructive, rating should exceed 0.5.” Distribution screens that monitor field-value entropy over time. When entropy drops under a threshold, you catch distributional collapse earlier than downstream metrics drift.

Periodic pattern audits on outputs from ambiguous or edge-case inputs catch assured fabrication. This layer requires area data and ongoing upkeep, but it surely covers a lot of the 5 failure modes, as a result of it checks which means, not form.

Layer 3: Uncertainty surfacing. Add an optionally available confidence subject alongside each extracted worth, so the mannequin can categorical what it does not know. Cleanlab’s CONSTRUCT benchmark reveals that per-field trustworthiness scoring detects errors in structured outputs from GPT-5 and Gemini with larger precision than prompt-level confidence estimates.

For prime-stakes fields, add LLM-as-judge verification: a second mannequin name that evaluates whether or not the extracted worth is supported by the enter. The price is latency. The payoff is catching failures earlier than they attain your pipeline.

Most groups have Layer 1 solely. Including Layer 2 catches the vast majority of silent failures. Layer 3 is for fields the place a unsuitable reply prices greater than the additional latency.

Three Indicators: When Your Pipeline Is Silently Failing

You’ll not catch these failures by inspecting particular person outputs. The alerts are statistical.

• Output entropy is dropping. If a subject that ought to differ throughout inputs begins clustering round one or two values, the mannequin is collapsing to secure defaults. Plot worth distributions weekly.

• No output is ever empty. If an array subject that ought to generally be empty by no means returns [], the mannequin is fabricating entries. Examine the empty-array charge towards your anticipated base charge.

• Downstream metrics drift with out upstream modifications. If your online business metrics shift however the mannequin model, immediate, and schema have not modified, the mannequin’s semantic accuracy could have degraded whereas structural compliance stayed excellent. That is the toughest sign to attribute, and it is typically the primary one.

Conclusion: Schema Is the Ground, Not the Ceiling

Default to constrained decoding for parse reliability. Construct the semantic checks it was by no means designed to supply.

Schema validation won’t ever inform you the information is true. It tells you the information is formed accurately. The hole between these two claims is the place these 5 failure modes stay.

Additional Studying

• Structured Outputs Create False Confidence (BAML’s benchmark on accuracy degradation below constrained decoding)

• Let Me Communicate Freely? (EMNLP 2024 research on reasoning decline below format restrictions)

• The Format Tax (measuring the 3-9pp accuracy price of structured output codecs)

• JSONSchemaBench (10K real-world schemas throughout six constrained-decoding frameworks)

• Cleanlab CONSTRUCT (per-field trustworthiness scoring for structured LLM outputs)

• From Hallucination to Construction Snowballing (how constrained decoding triggers formatting traps throughout self-correction)

···

Thanks for studying. I am Mostafa Ibrahim, founding father of Codecontent, a developer-first technical content material company. I write about agentic techniques, RAG, and manufacturing AI. If you would like to remain in contact or talk about the concepts on this article, you could find me on LinkedIn right here.