Gemma System Prompt: Format and Practical Examples

Gemma is a family of open models from Google, available on Hugging Face and through Google AI Studio. Gemma 4 is the current generation. Like other open models, Gemma uses a specific turn format that must be applied correctly for system instructions to work. Sending a system message without the right tokens can cause the model to ignore it or treat it as user text. Tools like fenbs keep the approved prompts and evaluation lessons outside any single environment, ready for the next session.

Check the Gemma model cards on Hugging Face↗ (opens in a new tab) and Google's documentation↗ (opens in a new tab) for the current format specification. Gemma's chat template is included in the tokenizer config and may change between model releases.

How Gemma handles system instructions

Gemma models use special tokens to mark the boundaries of each turn. The format uses start_of_turn and end_of_turn markers with a role identifier: "model" for the assistant and "user" for the user. Gemma 4 supports a system role in the chat template. Earlier Gemma versions handled system content differently -- for those, it was common practice to prepend the system instructions to the first user turn. Always check the model card for the specific Gemma version you are running.

Always verify the current format against the specific model version you are using. The tokenizer config in the model's Hugging Face repository contains the authoritative chat template.

The Gemma 4 turn format

With the Hugging Face apply_chat_template() method, the correct format is applied automatically. The resulting string for a conversation with a system turn looks like this:

<start_of_turn>system
You are a helpful assistant. Answer in plain English. Be concise.<end_of_turn>
<start_of_turn>user
What is a context window?<end_of_turn>
<start_of_turn>model

Note the role name for the assistant turn is model, not assistant. This is specific to Gemma. If you are constructing the prompt manually, using the wrong role name is a common mistake. Use apply_chat_template() whenever possible to avoid these errors.

from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("google/gemma-4-9b-it")

messages = [
    {"role": "system", "content": "You are a helpful assistant. Answer in plain English. Be concise."},
    {"role": "user", "content": "What is a context window?"}
]

formatted = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)
print(formatted)

Practical examples

Three starting points for common Gemma use cases, written as the content you would pass in the system role message:

You are a writing assistant for non-technical readers.
Use short sentences. Avoid jargon.
When technical terms are unavoidable, explain them in one phrase.
Do not use bullet points unless the user asks for a list.

You are a data analysis assistant for a US financial firm.
Follow SEC and FINRA plain-language guidelines.
Present numbers with commas for thousands and two decimal places.
Note data limitations or assumptions clearly.
Do not state investment conclusions.

Review the provided code for correctness, readability, and security.
Organize your response: 1) Bugs or errors, 2) Style issues, 3) Security concerns.
For each item, cite the line number and explain the problem.
Do not rewrite the code unless asked.

Accessing Gemma through Google AI Studio

Google AI Studio provides web and API access to Gemma models. In the AI Studio web interface, you set system instructions in a dedicated field and the formatting is handled automatically. For API access, the Gemini API format is used -- check the current Google AI documentation for the endpoint and parameters specific to Gemma models.

Vertex AI on Google Cloud also hosts Gemma models for enterprise deployments. The instruction format is the same, but the authentication and endpoint configuration differ. Consult the Vertex AI documentation for Gemma-specific setup. If you are evaluating Gemma before committing to a deployment, AI Studio is the fastest way to test a system prompt without any infrastructure setup.

Testing your instructions

After setting up a system prompt, print the formatted string using apply_chat_template() and read it carefully. Verify that the system turn appears with the correct markers, that the content is complete, and that no unintended characters were introduced.

Then run the model with five to ten varied inputs. Check that each response respects the rules in your system prompt. For format rules, vary the complexity of the input: a simple question may trigger the format, but a complex multi-part question might not. Test edge cases: empty input, off-topic questions, and inputs in a language you did not specify.

Testing across model sizes matters too. Gemma 4 comes in instruction-tuned variants at different parameter sizes. A rule that holds consistently for a larger model may need simplification for a smaller one. Note which size you tested on and keep separate prompt versions if the behavior differs significantly.

Common format errors

• Using "assistant" instead of "model" for the assistant role: Gemma uses "model" as the assistant turn's role name.
• Skipping the generation prompt: forgetting add_generation_prompt=True means the model does not know to start generating a response.
• Using an older Gemma template for Gemma 4: earlier Gemma versions did not support the system role. If you are adapting code written for Gemma 2 or Gemma 3, check the template.
• Applying no template: constructing the prompt string manually without the special tokens, which the model cannot parse correctly.

How to do this with fenbs

1. Create a free account. Start a personal kanban board for your Gemma prompt work.
2. Add a task for each system prompt you are developing. Note the Gemma model version, parameter size, and the inference environment.
3. Test the prompt and record the results in the task. Note what worked, what failed, and what you changed.
4. Save the approved prompt version as a Skill in fenbs. Include the model version and the date you tested it.
5. When you discover a format issue or behavior difference, add a Lesson so the next session can skip that problem.

Related: prompt engineering covers the general principles for writing system instructions. System prompt examples has patterns you can adapt for Gemma. For the broader picture of working with open models locally, system prompt for local LLM covers chat templates in detail.