How to Make an AI Harness: Tools, State, and Tests

An AI harness is the code that wraps a model call: it prepares the context, sends the request, parses the response, and connects the result to the rest of your application. The model itself is one piece; the harness is everything around it that makes the output usable and testable. Getting the harness right matters more than picking the best model, because a good model inside a broken harness produces bad results.

A harness that connects to an external project record over MCP can read shared context at the start of a run and save progress at the end. Tools like fenbs supply that record, so the harness operates on shared project state rather than state local to one machine or one session. The AI agent harness guide covers the concept in more depth.

What an AI harness does

At its simplest, a harness does three things: it assembles the prompt, calls the model API, and handles the response. In practice it also manages errors and retries, enforces token budgets, formats the output for the next stage of the pipeline, and provides enough observability to debug a run that produced wrong results.

More capable harnesses handle tool calls (when the model requests an external action), run evaluation checks on the output, and maintain state across multi-step workflows. Each addition increases complexity. Add pieces when you have a clear need, not speculatively.

The parts of a harness

A working harness has several distinct pieces:

• Context assembly: gathering the system prompt, conversation history, retrieved documents, and any structured data the model needs. This is where most token waste happens.
• The model call: the API request with model parameters, retry logic, and error handling.
• Output parsing: extracting structured data from model text, validating the format, and handling cases where the model did not follow instructions.
• Tool dispatch: handling requests the model makes to call external functions, then feeding the results back.
• State management: saving what the run produced so the next call or the next session has the right starting point.

Keep each piece separately testable. A context assembly function that can be tested without calling the model is easier to debug than one that is inlined with the API call.

Managing state in the harness

State is what persists between model calls. In a simple conversational harness, state is the conversation history. In a multi-step workflow harness, state is the accumulated results of previous steps, the decisions made along the way, and the tasks still outstanding.

Decide what state belongs in the harness itself (in-memory, short-lived) versus what belongs in an external store (durable, shared). Conversation history for a single run belongs in memory. Architecture decisions and project rules that should apply across runs belong in a durable external record.

When the harness connects to an MCP server at the start of a run, it can pull durable state from that server into the context window. This is more reliable than embedding rules in the system prompt and easier to update than hard-coded constants. The what is an AI brain article explains the pattern.

Writing tests for harness behavior

Testing a harness is different from testing ordinary code because the model output is probabilistic. You cannot write a test that asserts exact text. Instead, test the properties of the output: does it contain the required fields, does it meet the format, does it stay within the required limits, does it trigger the right tool calls.

Write a small set of representative inputs with known expected behaviors. Run the harness against them after every change. A run that passes on familiar inputs gives you confidence the change did not break the common case, even if it cannot guarantee correctness on every possible input.

Regression tests matter most. When a run produces an unexpected result, add it to the test set with the correct handling so the same failure cannot silently recur. Record the failure, the root cause, and the fix as a lesson in your project record.

Common traps in harness design

Context bloat is the most common trap. Every retrieved document, every previous turn, and every tool result added to the context increases cost and can push the most important instructions out of the effective attention range. Be deliberate about what you include. Start with less and add only when the output is clearly worse without it.

Error handling is the second trap. Model APIs return errors. Rate limits, timeouts, and malformed outputs all happen in production. A harness with no error handling fails opaquely. Decide in advance what each error means: retry, fall back, or surface to the caller.

The third trap is assuming determinism. Running the same prompt twice will often produce different outputs. Tests that pass once may fail later. Build in enough logging to reproduce a failing run: log the full prompt, the model parameters, and the output for any run that produced an unexpected result.

How to do this with fenbs

1. Create a free account. Add an AI harness project with the model, API, and intended use as context.
2. Record the design decisions in Decisions and rules: context strategy, token budget, error handling approach, test requirements.
3. Add tasks for each piece of the harness to build and each test scenario to implement.
4. Connect your development assistant over MCP from Integrations. Ask it to read the project context at the start of each session.
5. At the end of each session, save progress and update tasks. When a test reveals a new failure mode, add it as a lesson for future harness work.