Self-Hosted AI Personal Assistants: Setup and Work Context

A self-hosted AI personal assistant means you run the model yourself, on your own machine or a private server you control, rather than sending queries to a cloud provider. You keep your data local, pay for compute instead of tokens, and choose which model you run. The tradeoff is that you also maintain the setup, manage updates, and accept the capability limits of what runs on your hardware.

The setup question and the context question are separate. Running a model locally does not automatically give it memory of your work or your rules. Tools like fenbs give a local or cloud assistant an external work record it can read through MCP, so your decisions and progress survive across sessions and across model changes. See also the self-hosted AI models guide for choosing an inference platform.

What self-hosted means in practice

Self-hosted covers a range of setups. At the simple end, an app like Ollama or LM Studio runs a model on your laptop and exposes a local API on localhost. At the more complex end, you run inference on a GPU server you own or rent, with your own authentication and access controls.

For a personal assistant, the laptop or workstation setup is the most common starting point. It keeps everything offline and requires no API keys or accounts. The main limits are the size of model that fits in your available memory and the speed of generation on consumer hardware.

Neither case is fully private by default once you add tool access. Any tool the assistant can call is a path to the outside world, or to your local files. Check what each tool reaches before you connect it.

Choosing a local model

The right model depends on your hardware and the tasks you need it for. Smaller models (7B parameters and under) run on most laptops with 8 GB of RAM and are fast enough for most assistant tasks. Larger models need more memory and take longer to generate each response.

For coding and reasoning tasks, look at models with strong benchmark scores on those categories. For writing and summarization, a smaller general model often does the job well. Try a model on the tasks that matter to you before committing to it as your primary assistant. Local models can be swapped without reinstalling your tools.

Check the model's license before using it for work. Some local models are research-only or restrict commercial use. The vendor's model card or release page is the right place to check this, not a third-party summary.

Connecting the assistant to your tools

Most local inference servers expose an OpenAI-compatible API. Chat clients and coding tools that support a custom base URL can point at that local address. MCP-compatible clients, including Claude Desktop and some coding tools, can connect to MCP servers running locally or remotely.

Apply the same access discipline you would with a cloud assistant. Give the assistant only the tools the project actually needs. A file read tool for your notes folder is narrower and safer than full file system access. Review the connections when you change what the assistant is working on.

Some tools require network access even for a local model. A web search tool, a calendar integration, or an external MCP server all make outbound connections. Decide which of those are acceptable for your setup before connecting them.

Writing persistent instructions

A local model has no memory of previous conversations unless you give it one. You can provide persistent instructions through a system prompt that the client sends on every new chat. Write the instructions you want to apply to all sessions there: your name, how you prefer responses structured, any standing rules about the work.

Keep system prompt instructions stable and short. Long system prompts eat into the context window available for your actual work. Put the instructions that genuinely belong to every session in the system prompt. Put project-specific rules and progress in an external record instead.

For the AI personal assistant security considerations, see the AI personal assistant security risks guide before connecting tools to a local model.

What an MCP connection adds to a local setup

MCP is a standard protocol that lets an AI app call outside tools, including servers that hold your work record. An MCP-capable client, pointed at a local model, can still read your project context from a remote MCP server at the start of each session and write progress at the end.

This means a self-hosted assistant can have the same persistent context as a cloud one. The model runs on your hardware; the work record lives in fenbs. Your decisions, rules, tasks, and handoff notes travel with the work rather than with the model.

Switching between local and cloud models

One advantage of keeping your work record outside the model is that model changes do not cost you your context. If a cloud model handles a task better than your current local one, you can switch without re-explaining the project. If you move back to local, the record is still there.

Test the new model against the existing rules and tasks before treating it as a drop-in replacement. Ask it to read the project context and summarize the current state. If its summary misses a rule or misread a task, correct the record or the instructions before continuing.

How to do this with fenbs

1. Create a free account. Add your project to a personal kanban board.
2. Write down the standing rules in Decisions and rules: which tools are allowed, how the assistant should handle uncertain answers, any constraints that apply to every session.
3. Add the tasks you are working through and set Where we left off at the start of each work period.
4. Connect an MCP-capable client to fenbs from Integrations. Use the same connection with both local and cloud models as you switch between them.
5. At the start of each session, ask the assistant to read the project context. At the end, ask it to save progress. The record stays current regardless of which model or client handled the session.