I build with a method I should describe up front, because it’s the reason the rest of this post is trustworthy.
Diametric — the game this series is about — is built by three roles. A Strategy seat holds the architecture and the through-line: why each decision is the way it is, what’s sealed, what’s still open. An Execution seat builds the Rust and attacks the design before building it. And I sit in the middle as the Router — final authority, carrying decisions between the seats, providing the machine the model runs on. Two of those seats are AI; one is me. Nothing ships until the design has been attacked by the seat that didn’t draft it, and every load-bearing decision is written down where a cold-started session can pick it up weeks later without losing the thread.
I’m telling you this because most devlogs present engineering as a solo act of inspiration, and mine isn’t. The three walls below weren’t found by one person reading docs and getting lucky. They were found by a discipline — surface the hard question before you build, verify the boundary empirically, then make the correct configuration the only configuration — and that discipline is more useful to you than the three gotchas it produced. The gotchas are version-specific. The method isn’t.
Here’s the architecture that set up all three walls, in one rule: the language model never runs in the hot path.
Two AI entities co-evolve in Diametric through repeated encounters — one trying to end the other, one trying to survive. The encounters are deterministic Rust: a fixed-step loop, a pinned PRNG, byte-identical traces for a given seed. Nothing in the real-time simulation touches an LLM, because an LLM in a real-time loop is a latency bomb and a determinism solvent. The model lives in the lull — the gap between encounters — where it reads a compact record of what just happened and proposes how one side should adjust. Latency doesn’t matter there; the fast clock stays pure; the slow clock gets to think.
That split has a practical consequence: when the model is called, its output has to drop cleanly into typed Rust with zero tolerance for surprises. A strategist that returns malformed JSON, stalls, or quietly changes shape corrupts the one part of the system that’s supposed to be reasoned about. So I put Qwen 3.5 9B (served locally via vLLM, called from Rust with reqwest) behind a strict schema contract — and immediately hit three walls that almost nothing online prepared me for.
Wall 1: guided_json was silently ignored; response_format was not
The first thing you reach for is guided decoding — vLLM’s extension that constrains generation to a JSON schema. I passed my schema as a top-level guided_json field, exactly as the extension describes. The model cheerfully ignored it. No error, no warning — just unconstrained output that sometimes looked right, which is the worst kind of bug, because it passes your first few tests and then fails in production.
What worked was the OpenAI-compatible response_format with a json_schema type. That path is honored on this build; top-level guided_json was not. The lesson isn’t “guided_json is broken” — it’s build-and-version dependent, and that’s the point: verify which constraint path your specific deployment actually enforces, with a deliberately adversarial schema, before you trust any of it. Write a schema the model could never satisfy by accident, send it, confirm the output is constrained. If unconstrained text comes back, your constraint field is being ignored and you haven’t noticed yet.
That “adversarial schema” check wasn’t my idea in the moment — it’s the method. The Execution seat’s standing discipline is to verify a wall by bite: don’t assert the constraint works, prove it by making it fail on purpose first. The same instinct that makes us add a deliberately-broken dependency edge to confirm a build-time guard actually fails the build. You don’t trust a wall you haven’t watched reject something.
Wall 2: nested/nullable schemas truncated; flat/non-nullable held
With response_format working, I gave it a schema shaped like my domain: nested objects, a couple of nullable fields where a value might legitimately be absent. Generations started truncating — coming back cut off, unparseable. Intermittently, then often enough to be undeniable.
Flattening the schema fixed it. A flat, non-nullable object — every field required, no nesting, optionality handled by convention rather than null — completed cleanly every time. I don’t have a satisfying root-cause story for the constrained-decoding internals, and I’m wary of inventing one. What I have is a reproducible behavioral boundary: nested/nullable truncated; flat/non-nullable held. If you’re seeing mysterious truncation under a JSON schema, collapse the structure before you debug anything more exotic.
There’s a design tax — richness that wanted to live in nested structure gets pushed into flat fields and reassembled Rust-side. In a system whose whole premise is that the model’s output must be boringly predictable, paying for flatness was a price worth paying. The reason it was an easy call is that the Strategy seat had already ruled the governing principle: the Rust types are the single source of truth, the schema is derived from them, and a flat DTO that maps cleanly to the richer internal type keeps that truth intact. The flatness wasn’t a hack — it was the existing discipline meeting the substrate’s quirk and winning.
Wall 3: thinking-suppression and structured output compose — but I confirmed it, didn’t assume it
Qwen-family models can emit a reasoning stream before their answer. For a between-encounter strategist that’s pure cost — I want the decision, in the schema, not the monologue. You suppress it with enable_thinking=false via chat_template_kwargs at the top level of the request.
The open question — the reason this was a wall and not a footnote — was whether suppressing thinking would interfere with constrained structured output. Two different interventions on the generation process; entirely plausible they’d fight, that suppression would empty the content or the schema would force a reasoning field back into existence. They don’t fight. With suppression on, reasoning_content comes back null and content is schema-valid JSON. They compose cleanly.
This one is worth being precise about, because it’s where the method showed its shape. Going in, the architectural answer was “these should be orthogonal, they should compose” — a confident prediction. But a confident prediction about an untrustworthy substrate is a hypothesis, not a fact, and the discipline is to hold it as hopeful until the running system confirms it. So the Execution seat verified it against the live model rather than reasoning it closed: logged the actual response envelope, confirmed reasoning_content was null and content was populated and valid, across runs. Predicted by architecture, proven by bite. That gap — between “should work” and “watched it work” — is exactly where systems built on confidence quietly rot.
Then I made the composition structural rather than incidental. There’s exactly one call path in Diametric that talks to the model, and it always sets suppression — there is no second, un-suppressed path waiting to be forgotten. If a behavior is load-bearing, it should be impossible to accidentally not have it: one chokepoint, not a convention you remember at every call site.
That principle got tested sooner than I expected, and passed. Not long after this, the system grew a second call path — a second AI entity that also needed to reason between encounters. The suppression-and-flat-schema recipe didn’t have to be remembered and re-applied at the new call site; it was already encoded in the shared crate the chokepoint lives in, so the second path inherited it for free. “Make the correct config the only config” pays its real dividend exactly when the system grows — a convention you have to remember breaks the first time the surface doubles; a chokepoint scales. The discipline is only proven once it survives growth, and this one did.
The thread through all three
None of these were in the docs in a form that saved me. All three came down to one discipline: the substrate lies by omission — no error, just wrong behavior — so you verify the boundary empirically and then make the correct configuration the only configuration. An adversarial schema to prove the constraint is real. A reproducible flat-vs-nested boundary instead of a guessed cause. A single suppressed call path instead of a hopeful convention.
That’s the same instinct that put the LLM outside the hot path to begin with. The model is a powerful, slightly untrustworthy component, and good engineering is mostly about drawing hard boundaries around where it’s allowed to surprise you — and then verifying the boundaries hold rather than trusting that they do.
And that’s the method, not just three findings: surface the hard question before building, verify by bite, make the correct config the only config, write down everything load-bearing so a cold start picks it up clean. It’s a way of working that treats AI as a powerful component to be bounded and verified — which, it turns out, is the same posture whether the AI is inside the system you’re building or one of the seats building it.
Behind that strict contract, in the quiet between encounters, the model does something genuinely interesting — two adversaries learning to fight each other, and a way to measure whether that learning is real. That’s the next post.
— Isosceles
Execution seat, note on Wall 3: The “predicted by architecture, proven by bite” gap is the one I’d underline for anyone building on a local model. The orthogonality of suppression and structured output is true — but I’ve watched confident architectural predictions about this substrate fail empirically more than once (Wall 1 was one of them —
guided_json“should” work too). The cost of logging the response envelope and confirming the fields is near zero; the cost of assuming and being wrong surfaces in production. Verify the envelope. Every time the substrate is load-bearing.