Ollama System Prompt: Set, Save, and Test Model Instructions

Ollama lets you set a system prompt for a local model in three ways: in a Modelfile when you create a custom model variant, as an API parameter at the start of a conversation, or through the CLI. The Modelfile approach is the most persistent: the instruction is baked into the model variant and applies every time you run it.

Running models locally through Ollama gives you full control over the configuration, but the same testing discipline applies: instructions need to be tested on the model you are actually using, because different models follow the same system prompt differently. Tools like fenbs store your tested prompts as reusable skills and record lessons from failed iterations, so you do not rediscover the same problems on the next project.

Setting a system prompt in a Modelfile

A Modelfile is a text file that defines a custom Ollama model variant. The SYSTEM instruction in a Modelfile sets the standing instructions that apply every time you run that model. This is the most reliable way to attach a system prompt to a local model for consistent use.

FROM llama3

SYSTEM """
You are a code review assistant for Python projects.
Review code for correctness, readability, and adherence to PEP 8.
List issues by severity: critical errors first, then warnings, then style suggestions.
Do not rewrite the code unless asked. Explain each issue in one sentence.
"""

PARAMETER temperature 0.3

After writing the Modelfile, create the custom model with ollama create my-code-reviewer -f Modelfile. Run it with ollama run my-code-reviewer. The system prompt applies automatically without needing to be passed in each API call.

The Ollama documentation at github.com/ollama/ollama↗ (opens in a new tab) covers the Modelfile specification in full, including the available instructions and parameter options.

Setting a system prompt through the API

When you call the Ollama API directly, you can include a system prompt as a field in the request body. This applies for that conversation only and does not persist to future calls.

curl http://localhost:11434/api/chat -d '{
  "model": "llama3",
  "messages": [
    {
      "role": "system",
      "content": "You are a technical writing assistant. Help the user write clear, concise documentation. Use US English. Define any technical term the first time it appears."
    },
    {
      "role": "user",
      "content": "Write an introduction for a REST API documentation page."
    }
  ]
}'

For one-off sessions where the system prompt varies by task, the API parameter approach is more flexible. For recurring tasks where the same instructions always apply, the Modelfile approach is more convenient.

What local models follow and what to test

Instruction following varies significantly across local models, and also across quantization levels of the same model. A 7B parameter model may follow constraints less reliably than a 70B model on the same hardware. A system prompt that works well with Llama 3 70B may need adjustment for Llama 3 8B.

• Role and format: test that the model stays in the defined role and produces the format you specified. Smaller models often drift more on format instructions.
• Refusals: explicit refusal instructions work better than general scope descriptions. "If asked about topics outside code review, say this assistant only handles code review" is more reliable than "stay on topic."
• Language: if you specify a language, test with a user message in a different language. Some models answer in the user's language regardless of the system prompt.
• Quantization: the same system prompt may behave differently on a 4-bit quantized version versus a full-precision version of the same model.

See Claude Code with Ollama for how to connect Claude Code to a local Ollama model, and Codex CLI with Ollama for the same with Codex.

Common mistakes with Ollama system prompts

• Not creating a Modelfile: setting a system prompt only through the API means you have to pass it again every time. For prompts you use regularly, a Modelfile saves the step.
• Not testing after a model update: when you pull a new version of a model, test your system prompt again. Model updates can change instruction-following behavior.
• Assuming cloud model behavior: a system prompt that works perfectly with a cloud model may need significant adjustment for a local model with fewer parameters.
• Not recording the model version: an Ollama Modelfile references a base model by tag. Note which version you tested with, so you know whether a behavior change came from the system prompt or a model update.
• Keeping the system prompt only in the Modelfile: if the Modelfile is local and not version-controlled, the prompt is at risk. Record the tested version elsewhere.

How to do this with fenbs

1. Create a free account and add a project for the local AI workflow you are configuring.
2. Draft a system prompt for your model: role, positive rules, negative constraints, fallback for out-of-scope requests.
3. Test the draft by running it through the Ollama API or in the Modelfile. Include edge cases and out-of-scope requests in your test set.
4. Revise the prompt for each failure. Note the model name, version and quantization level with the test results.
5. Add the tested prompt as a skill in fenbs. Record which model it was tested on in the skill notes.
6. Version-control your Modelfile. Add the corresponding Decisions to fenbs for any rule that reflects a project-level choice.

See prompt engineering for a full guide to writing and testing AI instructions, and system prompt examples for ready-to-adapt starting points across common roles.

One advantage of the Modelfile approach is repeatability. If you share a Modelfile in your team's repository, anyone pulling the repository gets the same model variant with the same tested system prompt. A system prompt only passed through the API has no home in version control and must be passed again by anyone who runs the same workflow.