A runtime AI agent harness keeps a production agent purposefully weak. A coding harness gives a builder agent power, a verifier, and a human deploy gate.
Part 4 in our series When LLMs meet enterprise data explores why the most dangerous agent isn't necessarily the one with the most tools. It's the one that can change the tools it has.
That distinction matters when an agent moves from development into production. The agent helping a developer build a system needs room to explore. It should be able to inspect the data model, write code, run it, see what failed, and try again. A production agent answering a customer's question needs almost the opposite. It should see only the data it needs, use only the tools it has been given, stay within defined cost and latency limits, and have no way to expand its own capabilities.
Those are two very different jobs. Trying to handle both with a single agent harness – and simply switching between modes – creates an unnecessary security and reliability boundary.
We found it cleaner to treat them as two separate harnesses around the same model: a runtime harness designed to keep a production agent deliberately constrained, and a coding harness designed to give a development agent the power to build, test, and iterate.
The model can be the same. The harness shouldn't be.
An agent harness is the runtime layer around an LLM that controls what it can see, what it can do, how it loops, and what happens before its output becomes an action. It defines the:
The model is only one component of the system. Increasingly, the engineering differentiation is in the harness around it.
Once we started thinking in those terms, we realized that we weren’t running one agent with two modes. We were running two separate harnesses around the same model, built to be:
Safe in front of a customer
Productive in front of a developer
The problem starts when a model in one harness borrows powers from the other.
Let’s start with the harness that runs in production. Look at what it gives the model and what it deliberately holds back.
|
What a runtime agent gets |
What a runtime agent doesn't get |
|
One entity's data (one Micro-Database™) |
Other entities’ data |
|
Fixed set of tools |
Tool creation |
|
Read-only identity |
Write access |
|
Bounded context |
Schema changes |
|
Token budget |
Unlimited execution |
|
Latency budget |
Open-ended loops |
The runtime agent can't make itself more powerful. Whatever it can do at 9 am is exactly what it can do at 3 am.
Enterprise-ready means something specific here. The runtime harness must satisfy people who will never talk to the model:
None of these requirements depends on how capable the model is. Each one is a property of the harness.
The limits that meet these requirements also make the agent faster and cheaper to run. One entity's data keeps the context short. A fixed set of tools keeps the loop short. A shorter context and fewer steps mean lower latency and a lower cost per answer.
Security and performance don't have to be opposing goals here. The same constraints can improve both.
Before building any of this, we spent a long time on the obvious shortcut: take a coding harness such as Claude Code or Codex, wrap it in a service, and ship it.
Coding harnesses are powerful because they are designed for a different purpose: letting an agent explore, inspect files, choose tools, test ideas, recover from errors, and iterate until it solves the problem. That is exactly what you want when a developer is supervising, and the goal is to build something.
But those same properties become liabilities in a customer-facing runtime.
A production agent shouldn't be free to explore indefinitely, reach for unexpected tools, or change its own capabilities. Its behavior needs to be bounded: the same tools, the same data boundaries, the same authorization scope, and predictable cost and latency.
We could have started with a coding harness and tried to constrain it into becoming a runtime harness. Instead, we built the runtime harness around a different set of requirements.
Three reasons ultimately drove that decision:
Some coding harnesses are closely coupled to their provider's models and terms, and our customers don't all get that choice. Some run on Bedrock because that's what their cloud agreement allows. Some run on Azure. Some run a self-hosted open model because nothing leaves the building. And some want to switch next quarter.
A harness that assumes one provider turns that provider into a structural dependency instead of a configuration line. In our harness, the model is a plug. The same agent definition can run against Anthropic, OpenAI, Gemini, Bedrock, or a local Ollama model. Switching is a setting, not a porting exercise.
The second reason is the interesting one. Coding harnesses leave room for ingenuity. Production harnesses need consistency.
A good coding agent should be versatile and creative. It might try an approach, notice it's wrong, and back out. It may reach for a tool nobody expected or run for twenty minutes if the problem deserves it.
That's the right design when a developer is watching, and it's why those harnesses work so well.
It's the wrong design for answering a customer's question at 3 am. The same question should produce the same kind of answer, through the same tools, at the same cost and latency, today and next month.
Everything that makes a coding harness powerful – open-ended exploration, a long leash, and an eagerness to find a clever path – also introduces variance.
Coding agents optimize for exploration.
Runtime agents optimize for bounded behavior.
A coding agent needs room to explore. It can try an approach, discover it's wrong, and try another. With a developer watching, that variance is useful.
A production agent needs the opposite. Its tools, data access, cost, latency, and execution path should be bounded and reproducible. That makes the agent easier to secure, debug, monitor, and operate at scale.
Coding harnesses are still excellent tools. We use them every day for development, where versatility is exactly what you want. But for production, we wanted control at every point where the model's intent becomes an action.
The third reason is what settled it: we wanted our own hooks. If you own the loop, you can put code at every point where the model's intent turns into an action. Those hooks are where all the runtime guardrails live.
In our runtime, an agent is a declarative YAML definition. The loop around it exposes these hooks:
– step_start can rewrite the user's message before the model sees it.
– pre_tool_call fires after the model decides to call a tool and before the call happens.
– It gets the tool name and arguments, and returns ALLOW or BLOCK.
– post_tool_call can rewrite or redact the result before it re-enters the context window, or stop
the run outright.
– step_end can grade the answer and inject a continuation, so the agent loops again instead of shipping
something weak.
– on_error decides what the customer sees when something breaks.
A set of built-in hooks is always registered alongside these: an iteration cap (20 by default), a retry policy, a completion guard, logging, and the observability hook that records a trace of every session so it can be replayed.
We didn't bolt any of this on as a plugin. The hooks are part of the loop itself.
Owning the loop also lets us enforce data entitlement in code. The agent's trusted parameters, like the customerId that scopes everything, reach hooks and tool flows as an immutable init-params map. That map is separate from anything the model can produce.
A sub-agent's inputs are different, because the LLM may supply them, and our own documentation says it plainly: treat them as untrusted. When a flow needs the authorization scope, it reads it from the trusted map, never from an argument the model passed in.
Never let model-generated parameters define the security boundary.
For example:
hooks:
pre_tool_call:
flow: authorize_tool
post_tool_call:
flow: redact_result
step_end:
flow: evaluate_response
The model requests get_customer_orders(customerId=123). pre_tool_call checks the requested tool and trusted customer scope. The call is allowed. post_tool_call masks restricted fields before the result enters the context.
The important point isn't the specific hook names. It's the enforcement point: the model can request an action, but trusted code decides whether that action is allowed and what data can return to the model.
A coding agent needs capabilities that would be dangerous in production: visibility into the broader data model, a place to execute changes, and the ability to iterate against test results.
We built ours around four things:
A coding agent that sees files but not data will write plausible flows against the wrong join. It needs an augmented data catalog that describes what each table means, how entities relate across source systems, and which fields are PII.
The runtime agent gets a small schema: the data it needs for one customer. The coding agent gets the opposite – the broader data model, because its job is to change it.
The coding agent needs a dev instance with data that looks like production but isn't production. That's where masked snapshots come in.
In that environment, the agent can run its new flow, see what fails, and fix it without touching the live system.
The evaluation harness that grades the runtime agent also becomes the coding agent's test suite.Pin a snapshot, run the fixed question set through the runtime agent with the proposed change, score the answers, and compare them with the baseline. The coding agent can iterate before a human reviews the change.
A production bug therefore becomes a concrete development loop:
1. Import the masked snapshot.
2. Reproduce the wrong answer.
3. Change the schema or flow.
4. Re-run the evaluation.
5. Stop when the answer is fixed and nothing else has regressed.
The evaluation is only as reliable as the judge. We therefore validate the judge itself against a set of known “golden” cases, use deterministic assertions where the expected result is unambiguous, and periodically recalibrate the scoring criteria as the agent and test set evolve.
The result is a versioned change set: schema edits, flows, functions, and an evaluation delta showing what changed. A developer reviews the diff and the evaluation results, then deploys.
The coding agent never gets production access or deploy rights. The human gate is the boundary between the two worlds.
That separation matters even when a proposed change contains a mistake. If the coding agent creates a tool that writes data, the runtime role remains read-only, so the tool fails at runtime rather than silently succeeding.
That's separation of duties for AI agents in practice: each harness is designed on the assumption that the other might make a mistake.
Exactly two things cross between the two worlds, and both move in one direction only.
When something misbehaves in production, you don't debug it in production. You export the exact data state the customer's agent ran against and import it into the dev environment.
The important point is that this isn't generic test data. It's a reproduction of the production case, with the customer's PII removed. The data keeps its shape and relationships, so the coding agent can reproduce the behavior without accessing the customer's real records.
The export runs through masking on the way out. PII is anonymized, but relationships remain intact. The result is production behavior reproduced safely enough to debug.
You get the fidelity of a production bug without dragging production PII into the development environment – or into the coding agent's context window.
Code moves from dev to production through the deploy gate, with a human in it.
The coding agent's output is a change to the runtime harness itself. A new flow tagged as a tool is a new capability the runtime agent will have tomorrow. A schema edit changes what it can see.
So, the deploy gate is more than a code review. It's where you decide whether the weak agent in production should become a bit stronger – and you make that call with the eval delta in front of you, before the change goes live.
Nothing else crosses.
The runtime agent never sees the schema tools and the coding agent never sees a live token. The two harnesses share a model, a data platform, and a masking engine, but nothing else.
K2view was split along this line long before agents came along, so the pattern fits without bending anything.
K2Studio is the design-time environment, where the data product schema, functions, and flows are authored in versioned projects.
K2catalog gives the coding agent the broader data context it needs: data-aware maps, source profiling, relationship tracing, and PII discovery and classification.
The coding agent edits the data product schema and writes flows against it. Flows tagged as MCP integration tools become runtime agent capabilities only after deployment.
The dev environment provides the sandbox. Authoring in K2Studio changes nothing live; changes become real only through the deploy step, with versioning and soft-deploy providing the human gate.
The AI evaluation and observability harness becomes the verifier. It runs the same question set and LLM-as-judge criteria against a pinned snapshot, so the change under review can be compared with the baseline.
The Micro-Database provides the production-to-dev bridge. An entity's state is a self-contained file, so the production session and state can be exported and imported into dev.
K2view data masking runs on that path with cross-instance consistency. The same value masks the same way, so joins continue to work while real customer data stays protected.
The result is a clean separation: the coding agent can see broadly and build freely in a sandbox, while the runtime agent sees narrowly and operates within fixed boundaries.
Giving an agent access to your system is really two different requests. They deserve two different agent harnesses, rather than one harness with a mode switch.
At runtime, lend the least power that still answers the question, and fix that power at deploy time so the agent can't grow it.
At design time, be generous. Give the agent full visibility, a sandbox, a verifier to iterate against, and a human gate at the end.
And keep the boundary between them simple:
Masked data flows down. Versioned code flows up. Nothing else crosses.
If you're ever tempted to merge the two, remember that a mode switch implemented as a prompt is not a security boundary.
Coming soon: Did we accidentally build an agent framework?