The LLM Writes the Words, Code Runs the World
Where I draw the line between code and the model, after three projects.
- Written by
- Published
- Reading time
- 9 min
Where I draw the line between code and the model, after three projects
Three turns into a Boardroom Simulator session, my CFO was furious and conciliatory in the same breath. She slammed the budget proposal, then a sentence later warmly agreed to "find a path forward together." Nothing crashed and the prose was fine. The character just made no sense, because I had asked the model to decide something it had no business deciding.
I've hit the same problem from different directions in three AI systems: a boardroom simulator, a contract review agent and a research agent. All three ended up with the same split, and here it is as a rule you can apply to your own systems.
The thesis
The model is good at the edges, and code should own the middle.
The model is excellent at turning messy human language into structure, and at turning structure back into language. Everything in between belongs to code: state, rules, and anything you'll need to replay, test or defend.
The hard part is deciding which side a given decision falls on, and noticing when you've put it on the wrong one.
Three projects, one split
Boardroom Simulator: the model writes the dialogue and gives a strategy score for whether a character should speak. Code owns six bounded emotional numbers, delta tables, decay back to baseline (0.05 per turn), thresholds, and 60% of the floor bid.
Contract Review Agent: the model extracts clauses into typed Pydantic objects and gives a holistic risk score. Code runs the guardrails before any scoring, and the final level is
max(rule_score, …).Research Agent: the model plans sub-questions and writes the synthesis. Code validates the plan before execution, with keyword routing by default and embeddings opt-in.
I didn't set out to build any of these this way. Each one got there after something broke.
Five questions for every decision
When I'm not sure where a decision belongs, I ask these in order.
Must it give the same answer twice? If you need to replay it, diff it or unit-test it, put it in code. That's why the CFO's anger in Boardroom is a number between 0 and 1. I can write an assertion against 0.72. I can't write one against "kind of annoyed."
Is it a hard rule? Compliance, money, safety and required coverage go to code, and code gets the last word. A model can describe a liability cap, but whether an uncapped one is acceptable shouldn't be up to it.
Will someone ask why? If the answer has to be a trace you can show, such as "the word funding matched the Finance section," make code the default path and add embeddings or model routing as an option on top.
Is the input fuzzy language or judgement? Reading what a clause means, deciding whether now is a good moment to speak, and working out what a source claims are all model work.
Is the output prose for a human? That's model work too. When code concatenates strings into dialogue, you get a form letter.
Four patterns for the in-between cases
Most real decisions are some of each. These four patterns cover nearly every hybrid case I've run into.
Model as sensor
The model reads unstructured text and returns a typed object, and from there code handles everything.
Python9 lines
class Clause(BaseModel):
type: Literal["liability", "termination", "indemnity", "payment", "other"]
text: str
cap_amount: float | None
is_mutual: bool
extractor = llm.with_structured_output(list[Clause])
clauses = extractor.invoke(contract_text) # model ends here
violations = run_guardrails(clauses) # code starts hereThe model turns the contract into data and stops there. Acting on that data is code's job.
Model as voice
Code decides what's true, and the model decides how to say it.
Python8 lines
# code owns the state
cfo.anger = clamp(cfo.anger + DELTAS["budget_cut"]["cfo"], 0, 1)
cfo.anger = decay_toward(cfo.anger, cfo.baseline, rate=0.05)
# model only renders it
prompt = f"""You are the CFO. Current state:
anger={cfo.anger:.2f}, trust={cfo.trust:.2f}
Speak in a way consistent with these values."""The LLM never decides that the CFO is angry; it is told the CFO is angry.
That change alone fixed the furious-and-conciliatory CFO. She can't drift when the number lives somewhere else.
Code floor
The model's judgement gets blended in, but it can never drag the result below what the rules say.
Python3 lines
def merge_levels(rule_score: float, llm_score: float) -> float:
blended = 0.6 * rule_score + 0.4 * llm_score
return max(rule_score, blended)Hard violations can't be "reasoned down" by an LLM. The model can still raise a flag on something subtle the rules missed, but it can't lower a flag the rules already raised.
Model proposes, code checks
The model plans, and code checks the plan before anything runs.
Python3 lines
plan = planner.invoke(question)
plan = _ensure_core_sections_baked(plan) # every required section gets >=1 sub-question
results = execute(plan)My research planner once routed every sub-question into one section, and every other section came back NOT FOUND. The plan looked reasonable, so nothing caught it. A ten-line coverage check fixed it for good.
Failing the other way
I've also gone too far towards code. In an earlier version of Boardroom, I moved nearly all character behaviour into a deterministic engine. It was predictable and easy to test and debug, and the simulations were flat. Every scenario started from the same numbers, so every boardroom felt like the same room with different nameplates.
Handing authority back to the model would have reintroduced the CFO problem. What worked was keeping structure in code and feeding more variation into the model's inputs. In Boardroom that's a five-stage delta pipeline: code works out how each event shifts each character's state, scaled by personality and history, so no two scenarios start in the same place. The model then voices characters whose state is already interesting.
The line moves as you find out where a system is brittle and where it's lifeless. I still keep decisions in code and leave expression to the model.
Checklist
For each decision point in your system:
Does it need to be reproducible? Put it in code.
Is it a hard rule (money, safety, compliance, coverage)? Put it in code and give code the final say.
Will you need to explain it with a trace? Make code the default.
Is the input ambiguous human language? Use the model as a sensor with typed output.
Is the output prose? Use the model as a voice and pass the state in.
Does the model score something the rules also score? Add a code floor.
Is the model planning? Validate the plan before it runs.
Does the output feel flat? Give the model more varied inputs, not more authority.
