What is a context window in a chatbot?

The context window is the total amount of text the model can consider at once: the instructions, the conversation history, any attached documents, and tool results. Everything in the window shapes the response. Older messages can drop off when the window fills, which is why chatbots sometimes seem to forget earlier parts of a long conversation.

How does a chatbot decide what to say?

It generates a response one token at a time, where each token is chosen based on probabilities learned during training and the current input. The response reflects both what the model learned in training and what is in the current session's context, which is why instructions and context both affect the output.

Can a chatbot remember past conversations?

Not by default. A new session starts with no memory of previous ones. Some AI apps offer a memory feature that carries selected facts forward, but this is separate from the conversation itself. For ongoing work, it is more reliable to store context in an external tool the model reads at the start of each session.

What is RAG in chatbots?

RAG, short for retrieval-augmented generation, means the model retrieves relevant information from an external source and includes it in the context before generating an answer. Instead of relying only on training data, the model can use current or specific information retrieved at query time.

What is the difference between a chatbot and an AI agent?

A chatbot responds to messages in a conversation. An AI agent can also plan a sequence of steps, use tools, and take actions to complete a task with less step-by-step direction from a person. Many modern AI apps blur the line, supporting both conversational responses and multi-step tool use in the same interface.

How AI Chatbots Work: Context, Answers, Tools, and Limits

An AI chatbot turns instructions and conversation history into responses using a language model. Understanding how it works helps you use it better and know where its limits are.

October 19, 2026Updated October 21, 20266 min read

An AI chatbot works by feeding a conversation, including instructions and history, into a language model that generates a response. The model does not retrieve a stored answer. It produces an output based on patterns learned during training and the specific input it receives in that session.

That matters because it explains both what chatbots do well and where they fail. The model does not remember previous conversations. It only knows what is in the current session's input. Tools like fenbs let you store project notes, decisions, and progress outside the chat and give a connected assistant that context at the start of each session.

The building blocks: instructions, context, and a language model

Every AI chatbot session starts with instructions. These might be called a system prompt, a custom configuration, or a project description depending on the app. The instructions tell the model its role, what it should and should not do, and any facts it needs upfront.

On top of the instructions, the model receives context. Context is everything else it knows for this session: the messages exchanged so far, any files or documents provided, and any information retrieved from external tools. Together, instructions plus context form the input the model uses to generate its response.

How a chatbot turns input into a response

A language model reads the full input, instruction by instruction and word by word, and generates a response one token at a time. A token is roughly a word or part of a word. The model assigns probabilities to each possible next token and selects one, producing text that is statistically likely given the input.

This is why AI chatbots can be wrong with great confidence. If the training data contained a plausible-sounding wrong answer, or if the input context leads in a wrong direction, the model may generate that answer with no obvious signal that it is wrong. Checking important outputs against primary sources is always a good practice.

Tool use: when a chatbot acts, not just talks

Modern AI chatbots can use tools. A tool is an external function the model can call to get information or take an action. Examples: a web search tool that retrieves current information, a code execution tool that runs a snippet and returns the result, or a task management tool that reads and updates a project board.

MCP (Model Context Protocol) is a standard way for an AI app to use an outside tool, adopted by several major AI platforms. When a chatbot calls a tool, it receives the result back into its context and uses it to generate a better-informed response. Tool use extends what a chatbot can do, but also extends the range of actions it can take, which means access and permissions matter.

The context window and why memory is limited

The context window is the total amount of text the model can process at once: instructions, conversation history, retrieved documents, and tool results together. Everything in the context window influences the response. Everything outside it does not.

Context windows have grown significantly over time, but they are still finite. A long conversation eventually exceeds the window and older messages drop off. When that happens, the model no longer knows what was said early in the conversation. Starting a new session always resets the context completely.

This is why a chatbot "forgets" things between sessions. It does not have a persistent memory that carries forward. What it knows is only what is in the current session's input. Anything you want it to know in the next session has to be put back in.

How fenbs helps

Give each session a useful starting point

Connect a supported AI app to fenbs over MCP and it reads your current projects, rules, and Where we left off at the start of each session. When the session ends, save progress so the next session starts from the same known state. The context lives outside the chat window, so starting a new conversation does not mean starting from zero.

Create a free account

Free for one person · No credit card · Works with Claude, ChatGPT, Cursor and more

Where projects and decisions need to live outside the chat

If you are using a chatbot for ongoing work, information that matters across sessions should live outside the chat. That might be a document, a knowledge base, or a connected tool the model can read at the start of each session. The chatbot can help you maintain that record, but the record itself should not depend on any single chat history.

Decisions made in a chat are particularly easy to lose. If you and an assistant agree on an approach in one session, that agreement disappears when the session ends unless it is written somewhere. Recording decisions outside the chat is how you keep them usable across future sessions and future assistants.

How to do this with fenbs

Create a free account. Add a project with the context you currently repeat at the start of each chat.
Write the rules and decisions that should always apply as Decisions and rules. These are the first thing a connected assistant reads.
Connect your AI app from Integrations. Supported apps include Claude, Claude Code, ChatGPT, Cursor, Codex CLI, Gemini CLI, GitHub Copilot, and Devin Desktop.
At the start of each session, ask the assistant to read the project context. At the end, ask it to save Where we left off.
When you switch apps or models, the context is already in fenbs. The next assistant reads the same record without you rebuilding it.

For more on how chatbots are used in business applications, AI chatbot for business covers setup and use cases. The how to use AI at work page covers the broader workflow.