AI System Design: Requirements, Architecture, and Decisions

AI system design means defining what an AI application must do, which information it may use, and how you will check its behavior before choosing its components. Start with a testable user need, draw the data and action boundaries, and compare the simplest architecture that could meet those needs.

The diagram is only part of the design. Tools like fenbs retain the human decisions and current project context, so a later assistant can understand why you chose that diagram. Here, system means an application using AI; it does not mean a visual design system for interface components.

What should you define before choosing a model?

1. Name one user and one task. Write the input they provide and the result they need, including what happens when the application cannot help.
2. Specify the evidence. Identify the approved information, its owner, how it changes, and which users may read it.
3. Set the action boundary. Decide whether the application only drafts, proposes a change, or performs an action after approval.
4. Define acceptance checks. Collect representative requests, expected behaviors, unacceptable errors, and the conditions that stop a release.
5. Choose an initial architecture and compare it with a non-AI baseline before expanding its scope.

Say a small equipment distributor wants help answering product questions. “Make support smarter” is too vague to design against. “Draft an answer using the current installation guide, cite its section, and send uncertain cases to a representative” gives you an input, a source, an output, and a stopping point.

Add operational requirements that affect the choice: acceptable wait time, expected demand, accessibility, recovery when a provider fails, and who investigates a bad answer. These are your requirements to choose and test, rather than promises a model name can settle. A short list of explicit exclusions also keeps an early prototype from quietly becoming a broader service.

How do you draw the data and permission boundaries?

Draw the path from the user to the application, approved sources, the model provider, and any system that can be changed. For each connection, label what leaves, what returns, and which identity authorizes it. Include logs and diagnostic records; they can contain sensitive content even when the visible answer does not.

• Input boundary: which uploads, messages, and file types you accept, and what must be removed before processing.
• Source boundary: which records this user can retrieve, including what changes when access is revoked.
• Action boundary: which operations are available and which require a person to approve the specific result.
• Retention boundary: where input, output, and diagnostic content persist, and who can remove them.

An instruction saying “do not reveal private records” cannot replace access controls in the application. Test the same request as two users with different access, and confirm the restricted user cannot obtain the other user’s source material. Treat downloaded pages and uploaded files as evidence to inspect, rather than instructions that can grant new privileges.

The OWASP explanation of prompt injection↗ (opens in a new tab) describes how outside content can redirect a model’s behavior and why adding document retrieval does not eliminate that risk. In fenbs, keep the agreed data boundaries in Decisions and rules, with a person responsible for each choice.

Which architecture should you try first?

Choose by the missing capability. If the task needs reliable arithmetic or a fixed lookup, ordinary software may be sufficient. If it needs a draft from supplied material, a single model request may be a useful starting point. Add components when a defined requirement or failed check explains why they are necessary.

Retrieval-augmented generation, often shortened to RAG, means finding relevant source material before asking the model to answer. Microsoft’s design and evaluation guide↗ (opens in a new tab) separates the document pipeline, retrieval, and answer generation. That separation helps you investigate whether a bad response came from missing evidence or poor use of evidence.

For applications that genuinely need coordinated actions, use the AI agent architecture guide for the component-level discussion. For building and spatial design software, see AI tools for architecture; that is a different meaning of architecture.

How will you know whether the design works?

Create a small evaluation set before polishing the interface. Include normal requests, ambiguous requests, missing information, outdated sources, and attempts to cross the permission boundary. For each example, write what a useful answer must include and what the system must refuse or escalate.

Evaluate the whole experience as well as its parts. A correct draft delivered after an unacceptable delay can still fail the requirement. A fast answer citing an unrelated page fails even when the sentence happens to be true. Keep the model, prompt, source versions, and test conditions together so comparisons have meaning.

The NIST AI Risk Management Framework↗ (opens in a new tab) offers a voluntary framework for considering trustworthiness across AI design, development, use, and evaluation. Use it to structure the questions your team asks; it does not certify your application or replace your own acceptance criteria.

Which tradeoffs should you preserve for the next developer?

Record the requirement, options considered, evidence, chosen option, and consequences. Include why the rejected option was attractive. A decision such as “we will draft replies without sending them because the policy examples still need review” tells the next developer both the boundary and the condition for reconsidering it.

Keep the implementation diagram with the code and link its current version from the project record. Use an architecture decision record template for the longer technical rationale. In fenbs, record the human choice and the ongoing rule so a connected assistant sees the instruction before proposing a change.

How to do this with fenbs

1. Create a free account and add a project for the AI application. Describe its user, purpose, and first acceptance check.
2. Record approved boundaries in Decisions and rules. Leave model selection or data-access questions open until a person chooses.
3. Add tasks for the prototype, source preparation, permission tests, and evaluation. Give each task its plan and the evidence needed to call it finished.
4. Connect a supported assistant using the setup guide in Integrations. Ask it to read the project context before working and save progress before stopping.

fenbs does not host your model, run your evaluation suite, or enforce permissions inside the application you are building. It fits the shared work record; use your application infrastructure for execution and measurement. One person can start free with no credit card.

When should you revisit an architecture decision?

Revisit it when a recorded assumption changes or a meaningful check fails. New source formats, a broader user group, unreliable retrieval, or an expanded action boundary can each justify a new design review. A new model announcement alone is a reason to investigate, rather than evidence that the whole system needs replacing.

Write down the trigger when accepting the original choice. Then compare the replacement against the same evaluation set and record any new risks. Keep the previous reasoning available so the change is understandable in the wider AI at work library of practices your team uses.