AI Tech Stack: Choose the Layers Your Project Needs
An AI application is built from several independent layers: model, tools, retrieval, execution and evaluation. Here is what each layer does, what your options are and how to decide what to use.
Updated 4 min read
An AI application is not a single component. It is a set of layers that cooperate: a model that reasons, tools the model can call, retrieval that supplies context, an execution environment that runs everything and an evaluation layer that tells you whether it is working. Choosing the right layer for each job keeps the system maintainable and the decisions reversible.
Tools like fenbs keep your stack decisions and project context outside any one AI app, so the reasoning behind each choice is readable when you revisit a decision or bring in a new contributor.
The five layers
Most AI applications use some combination of these five layers. You do not need all of them for every project; start with the minimum that produces the result you need and add layers only when a real requirement demands it.
| Layer | What it does | When you need it |
|---|---|---|
| Model | Reasons about the input and produces an output: text, structured data or a tool call. | Always. Every AI application calls at least one model. |
| Tools | Functions the model can call: web search, code execution, file reads, API calls. | When the model needs information or actions that are not in its context window. |
| Retrieval | Finds and supplies relevant context from a knowledge base before the model call. | When the relevant information is too large to include in every call, or changes frequently. |
| Execution | Runs the model calls, routes results and coordinates multiple steps. | When a task requires more than one model call in sequence or parallel. |
| Evaluation | Measures whether the system is producing correct and useful outputs. | Always. Without evaluation you are guessing whether the system works. |
Model layer: what to decide
The main decisions for the model layer are which model family to use, whether to call it through an API or run it locally, and how much of your budget each call will consume. These decisions interact: a more capable model costs more per call, so for high-volume applications the cost per call matters as much as quality.
Start with the model your team already uses. Change it when you have a clear reason: the current model consistently fails at a specific task, or the cost at production volume is too high. Record those reasons in Decisions and rules so the next session knows why the switch happened.
Tools layer: what the model can do
Tools extend what the model can do beyond reasoning over text. A model that can call a web search tool can answer questions about current events. A model that can execute code can solve math problems it could not reliably solve by reasoning alone. Each tool you add expands capability and expands the attack surface: a tool that writes files or sends network requests needs careful authorization.
Design each tool to do one thing with a clear description, so the model can choose the right tool reliably. A vague tool description causes the model to call the wrong tool or call it with wrong arguments.
Retrieval layer: when and how to add it
Retrieval, sometimes called RAG (retrieval-augmented generation), fetches relevant content from a knowledge base and adds it to the model's context before the call. It is worth adding when the relevant information is too large to include in every call, changes more often than you can retrain a model, or is specific to your users or customers.
Retrieval is not magic. The model reasons about what retrieval finds, so poor retrieval results in poor answers even with a good model. Evaluate retrieval quality separately from model quality. A pipeline that retrieves unhelpful chunks will produce unhelpful answers regardless of how capable the model is.
Record your stack decisions where the next session can read them
Stack decisions get revisited: a new contributor asks why you chose one retrieval approach over another, or a cost spike prompts a question about which model to use. In fenbs, record each key decision as a Decision with the reason and the alternatives you considered. Every connected AI reads those Decisions before it acts.
Create a free accountFree for one person · No credit card · Works with Claude, ChatGPT, Cursor and more
Evaluation layer: you cannot skip it
Evaluation is how you know the system works. At minimum, write a set of test inputs with expected outputs and run them after every significant change. An AI system without evaluation is one where you find out it was wrong from a user complaint rather than from a test.
For complex pipelines, evaluate each layer separately: does retrieval find the right chunks, does the model reason correctly about what retrieval found, and does the final output match the expected result? Separate evaluations make it easier to find which layer is failing.
Keeping the stack maintainable
A stack that works in a prototype can become hard to change as it grows. The layers that are most often tangled together are model and retrieval: when the prompt format assumes a particular retrieval output format, changing the retrieval approach requires changing the prompt, and the dependency is not obvious from the code.
How to do this with fenbs
- Create a free account and add your AI project.
- Record each stack decision in Decisions and rules as you make it: which model, which retrieval approach, which tools, and why.
- Add a task for each major layer you are building or evaluating, with the current status and next step.
- Add Lessons learned entries when an evaluation reveals that a layer is underperforming in a way the next session should know about.
- Connect your AI over MCP from Integrations and ask it to read the project decisions before proposing changes to the stack.