Hermes Agent System Prompt: Customize and Test Instructions

Hermes models from NousResearch are fine-tuned to follow instructions precisely, and they use the ChatML message format for their system prompt. Setting a system prompt for a Hermes agent means writing the instructions in the system role of the ChatML structure before any user or assistant turns.

Tools like fenbs let you keep reusable agent prompts and shared project rules together so any session or team member starts from the same approved instructions rather than rewriting them from scratch.

The ChatML format for Hermes

Hermes models are trained to expect messages wrapped in ChatML tokens. The system prompt sits in the im_start system block before the first user turn. In most inference frameworks, such as Ollama or llama.cpp, this is handled automatically when you pass a system field; you write the instruction in plain text and the framework adds the tokens.

<|im_start|>system
Your agent instructions here.
<|im_end|>
<|im_start|>user
User message here.
<|im_end|>
<|im_start|>assistant

If you are using the Ollama Modelfile to set a default system prompt, add a SYSTEM field. If you are calling the model via an OpenAI-compatible API, pass the instructions as the system role in the messages array. The effect is the same: the instructions appear before any user input.

Writing the system prompt for an agent

An agent system prompt is more than a role description. It needs to tell the model what tools are available, how to decide when to use them, and what to do when a tool call returns unexpected output.

• Role: what the agent is and what it is responsible for.
• Tools: which tools are available and what each one does.
• Decision rules: when to call a tool versus when to answer directly.
• Output format: how the final answer should be structured.
• Fallback: what to say when the agent cannot complete the task.

Hermes 3 and later versions include function-calling support. If you are using tool calling, the tool definitions are typically passed separately in the API call. The system prompt should describe the purpose of the tools rather than repeating their schemas.

For an overview of agent memory systems that affect what goes in the system prompt versus what is retrieved at runtime, choosing an agent memory system is a useful reference. For multi-agent setups, context engineering multi agent systems covers how context is managed across agents.

Testing instruction precedence

When a Hermes agent receives both a system prompt and a user message that give different instructions, the system prompt generally takes precedence. But this is not absolute: a well-crafted user message can override a vague system prompt.

Test precedence explicitly before deploying an agent:

1. Write a system prompt that specifies one clear constraint (such as "only respond in English").
2. Send a user message that asks the agent to violate that constraint.
3. Check whether the system instruction held.
4. If not, rewrite the constraint more specifically or move it earlier in the system prompt.

For nuanced tasks, also test what happens when the user message is ambiguous relative to the system instructions. An agent that guesses in ambiguous cases is harder to rely on than one that asks for clarification. Add an explicit instruction for the ambiguous case: "If the task is unclear, ask one clarifying question before proceeding."

Handling conflicts between instructions

Instruction conflicts happen when the system prompt, a retrieved context snippet, and the user message all say something about how to behave. Hermes models trained for instruction following will attempt to reconcile them, but the reconciliation is not always what you intended.

The safest approach is to keep the system prompt focused on stable rules (role, tools, output format) and let retrieved context and user messages handle task-specific details. Mixing stable rules and task-specific instructions in the same system prompt makes conflicts more likely.

Shared project decisions, like which data source to trust or which format to use for responses, belong in a stable place the agent always reads. fenbs Decisions and rules are shown first in the AI context, making them the right place for rules that must not be overridden by a task-specific message.

How to do this with fenbs

1. Create a free account and add the project the agent works on.
2. Add the tested system prompt to Skills and prompts in fenbs with a name that identifies the agent and its version.
3. Record the project decisions that belong in the agent's stable rules as Decisions and rules in fenbs.
4. Connect your AI app over MCP from Integrations. The agent reads project rules before each session, so stable decisions do not need to be repeated in the system prompt.

Where to find the Hermes model documentation

NousResearch publishes Hermes model cards and usage notes on Hugging Face. The Ollama model library lists available Hermes versions and the configuration options for each. Check those sources for the current supported format before writing a system prompt, since the recommended structure can change between model versions.

For general system prompt patterns that also apply to Hermes, system prompt examples and prompt engineering cover the principles behind instruction structure and testing.