Building AI Applications: From Use Case to Tested Workflow

Building an AI application is straightforward when you start small. Pick one use case, connect a model to it, and evaluate whether the result meets your requirement. Then expand. Trying to design the whole system before you have evaluated a single workflow produces complexity before you have learned what the system actually needs.

Tools like fenbs keep architecture decisions and project context outside any one coding session, so the reasoning behind each choice is readable when a new contributor joins or when you revisit a decision six months later.

Start with a bounded use case

A bounded use case has a clear input, a clear expected output and a way to check whether the output is correct. "Answer customer support questions" is not bounded. "Given a customer question about our return policy, return the correct policy text" is bounded and testable.

Write the acceptance criterion before you write any code. The criterion describes what a correct output looks like for several representative inputs. This is also the basis for your evaluation later.

Connect the model

Most AI applications start with a model API call. You send a prompt and some context; you receive a response. The first version is usually just a prompt with the use case described in it. Evaluate whether the response meets your acceptance criterion before adding any infrastructure.

Use the model your team is most familiar with for the first version. You can change the model once you have an evaluation in place and can measure whether a different model is actually better.

Add tools and retrieval when you need them

Tools let the model take actions: call an API, search a database, run code. Retrieval supplies relevant context from a knowledge base before the model call. Neither is needed for every application.

• Add retrieval when the relevant information is too large to include in every prompt, or changes frequently.
• Add tools when the model needs to look something up or take an action that is not in its training data.
• Add an execution layer when the task requires more than one model call in sequence.
• Add human review steps when the stakes of a wrong output are high.

Each addition changes the system's failure modes. Retrieval can fail to find the right content. Tools can fail or return unexpected results. Evaluate each layer separately so you know which one is responsible when the system produces a wrong answer.

Evaluate before you scale

Evaluation is the step most often skipped. Without it, you are guessing whether the application works. Build your evaluation set from the acceptance criteria you wrote at the start: a set of inputs with expected outputs that you can run automatically after every significant change.

For a customer support application, the evaluation might check whether the correct policy section is included in the response. For a code generation application, it might run the generated code and check whether the tests pass. The specific check depends on the use case; the principle is the same: measure correctness before shipping.

Common mistakes at each step

At the use case step: choosing a use case that is too broad to evaluate. At the model step: changing models before evaluating whether the first one meets the requirement. At the tools step: adding a tool because it seems useful rather than because you have measured a failure that the tool would fix. At the evaluation step: shipping without one.

The steps an AI agent takes to complete a task guide explains how a well-designed agent breaks work down into verifiable steps, which is the same discipline applied at the application level.

Keeping context across contributors

An AI application project tends to accumulate important decisions that are not visible in the code. Why you chose a particular retrieval approach, why a certain model is used for one step and a different one for another, and what the known failure modes are: all of these shape how the project should be changed. When a new contributor joins or the project is revisited after a gap, that context is harder to reconstruct than the code itself.

Keep the decisions that matter in fenbs Decisions and rules. Every connected AI reads them before it acts. The evaluation set belongs with the project too: a set of test inputs and expected outputs that travels with the code, not just in a single developer's head. When a new contributor joins, pointing them at the evaluation set and the decision log is more useful than a walkthrough from memory.

How to do this with fenbs

1. Create a free account and add your AI application project.
2. Write the use case, the acceptance criteria and the first evaluation set in the project context notes.
3. Record each architecture decision in Decisions and rules as you make it: what you chose, why and what you considered.
4. Add tasks for each layer you are building or evaluating, with the current status and the next step.
5. Connect your AI over MCP from Integrations and ask it to read the project decisions before proposing changes to the design.