DivaBot: one brain, every channel

In the last post I built a local AI assistant on a handheld PC, learned what a token costs on hardware I can touch, and admitted the API won. That post was about hardware; this one is about architecture. A model alone is not an assistant: it is a function you call. To make DIVA actually live in my life I needed a channel: Discord.

One brain, many channels

The architecture decision that shaped everything was boring and I am proud of it: one shared LLM client (llm_client.py) used by every frontend. Discord and a terminal CLI all call the same ask_llm(prompt) -> str.

If I want a Telegram bot tomorrow, I write the frontend and nothing else. The frontends do not know or care which model answers them.

def ask_llm(prompt: str) -> str:
    if PROVIDER == "deepseek":
        return _ask_deepseek(prompt)
    return _ask_local(prompt)

This is the same instinct I use at work: the model is the part that changes (or gets replaced), so it must be the part that is easy to swap.

Two providers, one switch

DIVA runs on the local llama.cpp server or on the DeepSeek API, selected with one environment variable, LLM_PROVIDER. Both are "OpenAI-compatible" and yet they are not the same endpoint: same shape, different fields. The trick was to make the retry layer take an extractor function, so the only difference between providers is a lambda:

# local llama.cpp
lambda data: data["choices"][0]["text"].strip()
# DeepSeek
lambda data: data["choices"][0]["message"]["content"].strip()

"OpenAI-compatible" is a de facto standard now, and it is a gift: your code does not care who serves the tokens. Local inference is still there, but the brain is swappable — exactly why the API can win without a rewrite.

Retries, because servers fail

Every request goes through a retry loop with exponential backoff: sleep(2**attempt), so 1s, then 2s, then give up and raise a clear error. Three attempts, no more.

With a fixed delay, every client would hit a struggling server with a synchronized wall of requests at the same instant.

What building this taught me

• The model is the commodity, the engineering around it is yours. Two platforms, local or remote, without touching the frontends.
• Do not block the event loop. The LLM call is synchronous, so the Discord handler runs it in a thread (asyncio.to_thread). Otherwise one slow model response would freeze every other command on the bot.

The reliable choice, again

Last post I ended with "the practical choice is the reliable one". Nothing changed: the brain still mostly runs on the DeepSeek API, and the local llama.cpp server is a fallback I can flip to with one variable. The difference is that now it does not matter which one wins, because the software around it is mine.