LLM APIs: How to Make a Useful First Integration

An LLM API, short for large language model application programming interface, is the connection between your code and a language model. Instead of typing in a chat interface, you send a structured request from code or a tool, and the model returns a response your application can use. Tools like fenbs hold the integration brief, the decisions behind it and the lessons from building it, so any AI assistant you connect reads the current context before helping with the next session.

The major LLM APIs, OpenAI, Anthropic (Claude) and Google (Gemini), follow a similar pattern. This guide uses that common pattern. For provider-specific setup, see OpenAI API: build and review a small project or Claude API.

What an LLM API call actually sends

At its core, an LLM API call sends a list of messages and receives a response. The messages list includes a system message (your instructions), a user message (the input for this call), and optionally earlier assistant messages if you want the model to have conversation history. The model reads what you send and nothing else; it does not remember previous calls.

You also send parameters that control the response: the model name, a temperature value that affects how varied the outputs are, and sometimes a maximum token count. These parameters are your dials. Start with the defaults and only change them when you understand why.

Write a system prompt that constrains the task

The system prompt is your instruction to the model. It sets the scope, the output format and the rules the model must follow on every call. A system prompt that says "you are a helpful assistant" leaves almost everything open. A system prompt that says "you summarize support tickets in under 40 words, in plain text, without adding information not in the ticket" is testable.

• State the task in one sentence: what the model should produce from the given input.
• Specify the output format: plain text, JSON, a numbered list, or a specific structure. A defined format is easier to validate.
• Add constraints: length limits, tone, what to exclude, what to do with an input that is out of scope.
• Include one example of the expected output if the format is non-obvious. An example is more reliable than a description.

Supply project context with each request

The model has no memory of previous calls. Any context it needs to do the task, such as the goal of the project, the rules it must follow or where the work stands, must be included in the request. You can include it in the system prompt for facts that never change, or in a user message for the current state of the work.

For ongoing projects, the context you need to supply is the project's current rules, decisions and progress. An MCP connection to a tool like fenbs lets a supported AI app read that context at the start of a session, rather than requiring you to paste it in each time. The context stays current because the tool is updated as the project moves forward.

Handle errors before they reach users

LLM API errors fall into a few categories. Authentication errors mean your key is wrong or expired. Rate limit errors mean you are sending too many requests too quickly. Context window errors mean the combined length of your messages exceeds the model's limit. Service errors mean the provider is temporarily unavailable.

Each type needs a different response. Rate limit errors: implement exponential backoff and retry. Context window errors: shorten the input or switch to a model with a larger window. Service errors: retry with backoff, then fail with a clear message. Authentication errors: stop and alert; do not retry.

from openai import OpenAI, RateLimitError, AuthenticationError
import time

client = OpenAI()

def call_with_retry(prompt, max_retries=3):
    for attempt in range(max_retries):
        try:
            response = client.chat.completions.create(
                model="gpt-4o-mini",
                messages=[
                    {"role": "system", "content": "Summarize in under 40 words."},
                    {"role": "user", "content": prompt},
                ],
            )
            return response.choices[0].message.content
        except RateLimitError:
            if attempt < max_retries - 1:
                time.sleep(2 ** attempt)
            else:
                raise
        except AuthenticationError:
            raise  # do not retry authentication errors

Validate outputs before acting on them

An LLM output is not guaranteed to match your expected format, even with a clear system prompt. Add a validation step after the API call: does the output have the required structure, is it within the allowed length, does it contain any disallowed patterns? Validation is especially important for outputs that feed into another step, like a structured response that gets parsed as JSON.

If validation fails, decide in advance what happens: retry with a modified prompt, return an error to the caller, or send the output for human review. Write that decision down. Having a defined fallback is better than letting a bad output propagate silently.

How to manage your LLM integration with fenbs

The context you write at the start of a project is the context that helps every subsequent session. Spend five minutes on it before writing the first line of code.

1. Create a free account. A personal board is free with no credit card required.
2. Add a project. Write the integration task in one sentence, the output format, the validation rule and the provider chosen.
3. Add a task for each phase: system prompt, request structure, error handling, output validation. Move each to In Progress when you start.
4. Record each design decision: the model chosen and why, the prompt version, the validation rule, the error handling policy.
5. Connect your AI coding assistant from Integrations over MCP. Ask it to read the AI context before each session so it starts from the current state of the project.