System Prompt in LangChain: Templates and Message Roles

LangChain passes system instructions to a language model through a SystemMessage object or a ChatPromptTemplate that includes a system component. The model receives that message before the user's input and applies it as standing instructions for the conversation.

The flexibility of LangChain templates makes it easy to write prompts with variable slots, but that flexibility also creates new failure modes: variables that produce unexpected output when filled at runtime, or system messages that conflict with the model's behavior in ways that only show up in production. Tools like fenbs store approved prompt content and the reasons behind each rule, so the tested version is available as the project evolves.

The SystemMessage class

In LangChain, a SystemMessage is the direct equivalent of passing {"role": "system", "content": "..."} to a chat API. You create one with the standing instructions as a string, then pass it as the first message in a list to the model.

from langchain_core.messages import SystemMessage, HumanMessage

messages = [
    SystemMessage(content="You are a technical writing assistant. Help the user write clear documentation for software engineers. Use US English. Define any technical term the first time it appears."),
    HumanMessage(content="Write an introduction for our REST API documentation.")
]

response = model.invoke(messages)

For a one-off system message, SystemMessage is the simplest approach. When the same system message is reused across multiple chains or needs variables filled at runtime, a ChatPromptTemplate is more practical.

Using ChatPromptTemplate for system instructions

ChatPromptTemplate lets you define the structure of a prompt, including the system component, with named variables you fill at runtime. The from_messages method accepts a list of role-content tuples.

from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a {role} for {company}. Help users with {scope}. Respond in US English. Keep responses under 200 words. If a request is outside {scope}, explain that and offer to escalate."),
    ("human", "{user_input}")
])

chain = prompt | model
response = chain.invoke({
    "role": "customer support assistant",
    "company": "Acme Software",
    "scope": "Acme Software's desktop application",
    "user_input": "How do I reset my password?"
})

Variable substitution happens at runtime when you call invoke. Each {variable} in the template is replaced with the value you pass. The model never sees the template syntax; it receives the filled string.

Testing variable substitution

Variables in system prompts create a new category of failure: the template may look correct, but a particular combination of values at runtime produces instructions that are unclear or contradictory.

• Test with the real values you will pass in production, not with placeholder text. Short variable values may not produce the same behavior as long ones.
• Check that sentence structure is correct after substitution. A variable containing a company name that includes a comma or a conjunction can produce a malformed sentence.
• Test the edge case where a variable value is empty or unexpected. What does the model do if a variable slot receives an empty string?
• Test that the filled system message still produces correct behavior on out-of-scope user messages.

See prompt engineering for broader guidance on testing AI instructions, and context engineering vs prompt engineering for when a system prompt alone is not enough.

Common mistakes in LangChain system prompts

• Putting request-specific context in the system message: the system message should hold standing instructions, not information that changes per request. Move request-specific context to the human message or inject it as a variable.
• Overly long templates: a system message that runs to several paragraphs competes with the actual user message for the model's attention. Keep standing instructions focused on the rules that matter across all requests.
• Not testing after model changes: LangChain supports many model providers. When you switch from one to another, the same system message may produce different behavior. Test on the model you are actually deploying with.
• Ignoring template injection risk: if a user-controlled value is substituted into a system message template variable, a malicious input can alter the instructions. Never put unsanitized user input into a system message variable. See indirect prompt injection for more on this risk.

Versioning and storing prompt content

LangChain prompt templates are Python code. Version-controlling the template in your repository is straightforward, but the reasoning behind a rule in the template is usually in someone's head or in a comment. When the template needs changing months later, the comment is gone and the reason is unclear.

A separate record that pairs each system prompt rule with the decision behind it makes future revisions safer. When a constraint was added for a specific reason, that reason shows whether the constraint should change when circumstances change.

LangChain also offers a LangSmith hub for managing and versioning prompts. Whether you use that or a simpler approach, the goal is the same: the current tested version is findable, and the reasoning behind each rule is accessible.

How to do this with fenbs

1. Create a free account and add a project for the LangChain application you are building.
2. Write the system prompt content: role, positive rules, negative constraints, fallback for out-of-scope requests.
3. Build the ChatPromptTemplate or SystemMessage from the content. Test with your real production variables.
4. Run the precedence tests: out-of-scope requests, requests that try to override the rules, edge cases for each variable.
5. Add the tested prompt content as a skill in fenbs. Record the reasoning behind any rule as a Decision.
6. When the template needs updating, check the Decision record first to understand what was deliberate before changing it.

See what is LangChain for an overview of where the system prompt fits in a LangChain application, and ChatGPT prompts for work for user-turn prompt examples that pair with your system instructions.

When your LangChain application handles multiple types of requests with different standing instructions, consider a separate chain per instruction type rather than one chain with a very long combined system message. Shorter, focused instructions tend to be followed more consistently than long ones that try to cover every case in a single block.