What is the minimum viable chatbot I can build quickly?

The simplest chatbot sends user input to a language model API with a system prompt and returns the response. Add a FAQ document as context if you want it to answer specific questions. Start there, test it thoroughly, and add retrieval, memory, or tool connections only when a real user need requires them.

Do I need RAG for my chatbot?

RAG, or retrieval-augmented generation, adds a search step that retrieves relevant documents before generating a response. It is useful when your knowledge base is too large to fit in the context window, or when you need the chatbot to cite specific documents. If your knowledge base is small and changes infrequently, a well-written system prompt with the content pasted in may be simpler and good enough.

How do I handle questions the chatbot cannot answer?

Write an explicit fallback instruction in the system prompt that tells the model what to say when it does not know the answer. Test the fallback with out-of-scope questions before deploying. A clear refusal, such as "I can only answer questions about X; for anything else, contact support at this link," is far better than a confident but wrong answer.

How often should I re-evaluate the chatbot after deployment?

Run a full evaluation whenever you change the system prompt, data sources, or the underlying model. For production chatbots, review a sample of logs weekly and run a targeted evaluation if you see a pattern of poor responses. Model providers update their models without notice, so evaluation should be a regular practice, not a one-time event.

What should I document about the chatbot for the next developer?

Document the scope, what questions it is designed to answer and what it refuses, the approved data sources and why each was chosen, any sources that were excluded and why, the system prompt version history with evaluation results, and any known edge cases or failure modes. This documentation is as important as the code.

Chatbot App Development: Sources, Boundaries, and Testing

Building a chatbot app requires decisions about data sources, access limits, and how you will evaluate responses before deploying to users. Here is a practical framework for each stage.

October 2, 2026Updated October 21, 20266 min read

A chatbot app is a software product that uses a language model to answer questions or take actions on behalf of users. The development process is similar to any software project: define the scope, choose your data sources and infrastructure, build access boundaries, evaluate the output, and maintain the approved behavior over time. Tools like fenbs help development teams keep the implementation decisions and current progress in one record, so any AI assistant connected to the project reads the agreed context before proposing a change.

Define scope and data sources first

The most important decision in chatbot development is what the chatbot is allowed to answer and what it should not try to answer. Start by listing the specific questions or tasks you are building for. A chatbot that handles a narrow set of questions well is far more useful than one that handles everything poorly.

Your data sources are the documents, databases, or APIs that the chatbot will draw on to answer questions. Common sources include a product FAQ, a knowledge base, a database of records, or a set of approved documents. List each source, confirm you have the right to use it, and decide whether it will be read-only or whether the chatbot will be allowed to update records through it.

Choose an API or framework

Most chatbot apps call a large language model through an API rather than running the model themselves. The main providers are OpenAI, Anthropic, Google, and Mistral. Each has an API with its own pricing, context window size, and model options. Framework choices include LangChain, LlamaIndex, and similar tools that add retrieval, memory, and tool-calling on top of a basic API call. A simple FAQ chatbot may not need a framework at all; start with the minimum and add complexity only when a real requirement demands it.

For spec-driven approaches to building the chatbot, see spec-driven development tools. Writing a clear specification before you choose a framework reduces the risk of building the wrong thing. The spec should describe the user input, the expected output, the sources the chatbot is allowed to use, and what it should say when it cannot answer.

Build access boundaries

Access boundaries are the rules that prevent the chatbot from doing things it should not do. At a minimum, define which data sources it may read, which it may write to, what it should refuse to answer, and how it handles requests it cannot fulfill. Write these rules down before you build them in.

Prompt injection is a real risk: a user or an external document can include text that tries to override the chatbot's instructions. Test this explicitly by including adversarial inputs in your evaluation set. For chatbots with write access to any system, add a confirmation step before the chatbot takes any action that is difficult to reverse.

Evaluate responses systematically

Build an evaluation set of questions and expected answers before you deploy. Include normal questions, edge cases, and out-of-scope requests. Run the evaluation after every significant change to the system prompt, the data sources, or the model version. A human reviewer should check a sample of responses in each evaluation run, not just rely on automated metrics.

Log the inputs and outputs in production. When a user reports a bad response, you need the actual input, the context the chatbot used, and the output. Without logs, you are guessing what went wrong. Review the logs on a schedule, not just when something breaks, because problems often appear gradually rather than all at once.

See AI chatbot for business for how to frame a chatbot project within a business context and the questions to answer before committing to a build.

Maintain approved behavior over time

Model updates, data source changes, and new edge cases will affect your chatbot's behavior after you deploy. Every change to the system prompt, the retrieval configuration, or the model should go through the same evaluation as the original build. Keep a version history of system prompt changes and the evaluation results that accompanied each change.

Document the decisions you make during development, such as why a particular source was excluded, why the chatbot refuses a category of questions, or why a specific response format was chosen. These decisions are invisible in the code and are easily lost when team members change. The next person to maintain the chatbot needs to know not just what it does but why.

How fenbs helps

Keep implementation decisions and progress in one place

fenbs keeps Decisions and rules for the chatbot build, including which sources are approved, what the chatbot must refuse, and the evaluation results that approved each system prompt version. When a team member changes, the decisions travel with the project rather than living in a leaving developer's notes. Connect a supported AI assistant from Integrations and it reads those decisions before proposing changes.

Create a free account

Free for one person · No credit card · Works with Claude, ChatGPT, Cursor and more

How to do this with fenbs

Create a free account and add a project for the chatbot build.
Add tasks for each development stage: scope definition, data source approval, framework selection, access boundary design, evaluation setup, and deployment.
Record each key decision as a Decision and rule: which sources are approved, what the chatbot must not answer, and what constitutes a passing evaluation. Include the person who made the decision and the reason.
When the system prompt is finalized and passes evaluation, save the approved version in Skills and prompts with the evaluation results noted.
Connect a supported AI coding assistant from Integrations and ask it to read the project context before starting work on the chatbot code.

fenbs is for tracking the development decisions and progress, not for running the chatbot itself. It does not import chat logs or connect to your chatbot at runtime. For the broader software delivery lifecycle this chatbot fits into, see software development life cycle.