How to Build an AI Chatbot in Java

A Java AI chatbot sends HTTP requests to a model API, maintains a conversation history as a list of message objects, and acts on any tool calls the model requests. The structure is straightforward, but you need to be deliberate about how you manage the message list, handle errors, and restrict what tools the model can call. A project record in fenbs can supply standing rules and task context to a Java chatbot that includes an MCP client, so the assistant starts each session knowing what the project requires.

Dependencies and setup

Java 17 or later and a JSON library are the main requirements. Jackson and Gson are both common choices for serializing the messages list and deserializing the response. For the HTTP call you can use Java's built-in HttpClient (introduced in Java 11) or a library like OkHttp. You will need an API key from your chosen model provider as an environment variable.

The conversation loop

Every call to a chat completions-style API sends the full message history, not just the latest turn. You maintain a List<Map<String, String>> where each entry has a role and a content. The first entry is the system message. Each user turn appends a user message, and each model response appends an assistant message.

List<Map<String, Object>> messages = new ArrayList<>();
messages.add(Map.of("role", "system", "content", "You are a helpful assistant."));

// After each user input:
messages.add(Map.of("role", "user", "content", userInput));

// Build the request body:
Map<String, Object> body = new LinkedHashMap<>();
body.put("model", "gpt-4o-mini");
body.put("messages", messages);

String json = objectMapper.writeValueAsString(body);

Send the JSON body in an HTTP POST to the completions endpoint, parse the response, and append the assistant reply before the next turn. Keep the message list in memory for a single session. To persist it across restarts, serialize to a file or a database when the program exits.

Making the HTTP call

Java's HttpClient is sufficient for most chatbot projects. Set Content-Type: application/json and Authorization: Bearer <your-key> in the request headers. Use BodyPublishers.ofString(json) for the body and BodyHandlers.ofString() to read the response. Parse the response JSON to extract the assistant message content.

Wrap the call in a try-catch that handles IOException and InterruptedException. For rate-limit responses (HTTP 429), a simple retry with a brief wait is usually enough during development. In production, put the chatbot behind a queue and handle retries there rather than in the chatbot loop itself.

Adding tool calls

Tool calls let the model request an action your Java code performs. You define each tool as a JSON object with a name, a description, and a parameter schema, then pass the list of tools in the request body. When the model returns a tool_calls response instead of a content response, you look up the function by name, run it, and append the result as a tool message before calling the API again.

• Define only the tools the chatbot genuinely needs for the current task.
• Validate tool call arguments against your expected types before executing the function.
• Return a JSON string from the tool function, not a Java object, so the append step is consistent.
• Log every tool call name and arguments during development to verify the model is calling the right functions.

Managing context window size

The messages list grows with each turn. All model APIs have a maximum context window measured in tokens. When the list gets long, truncate the oldest non-system messages before the next call. A simple sliding window that keeps the system message and the last N turns is enough for most chatbot use cases.

When you truncate, remove complete user-assistant pairs rather than cutting arbitrarily. Removing a user message without its corresponding assistant reply, or vice versa, can confuse some models. Keep the system message at index 0 and then the most recent N complete exchanges.

Testing your Java chatbot locally

Before connecting the chatbot to any frontend, test the conversation loop by hand: run the program, send a sequence of messages that cover the scenarios you care about, and check each response. A chatbot that works on the happy path may fail on an empty input, a very long message, or a message that triggers a tool call for the first time.

Write unit tests for each tool function using mock inputs. The model generates arguments in varied formats; your function should validate them before use. A tool function that throws a clear exception on bad input is easier to debug than one that silently produces the wrong result.

How to do this with fenbs

1. Create a free account and add a project for the chatbot you are building.
2. Record the key decisions as Decisions and rules: which tools the model may call, what topics are off limits, the expected response style.
3. Add tasks for each conversation flow or feature you are implementing. Note the status and what was tested last.
4. Connect a supported AI coding tool from Integrations. It reads the project context at the start of the session.
5. Before you finish a coding session, ask the assistant to save its progress on each task so the next session continues from Where we left off.

For similar guides in other languages, see how to build a JavaScript AI chatbot and how to build a PHP AI chatbot. The conversation loop pattern is the same; the HTTP client and JSON library differ.