Eric J Ma's Website

I've been building and testing a Discord bot for the private community around the Learn Anything with AI retreat. It's called Sage, and its job is to be a growable knowledge assistant for the community. It answers the questions that come up over and over (when is the retreat, what does the week look like, what's the deposit policy), points people at the right resources, and does it all in a consistent voice. As the community grows, the knowledge it answers from grows with it.

I had a second goal with this build, and it's the one I care about more. Almost all the energy in coding agents right now goes toward one-on-one use: you at a terminal with your agent. I wanted to find out how hard it is to build the multiplayer version, an agent that an entire community can talk to. I expected that to be the hard part, but it's actually the easy part: just stick a coding agent inside a container, wrap an API around it, and connect it to a chat platform. That's the whole idea. Everything else is detail. You can use very simple tech to build genuinely useful AI applications.

One agent, a whole community

Talk to your own coding agent and the conversation is private: your context, your files. A community agent has to work for everyone at once, and that changes the requirements.

Grounding has to be shared. If I ask Sage about the deposit policy today and you ask tomorrow, we should get the same answer, drawn from the same source of truth. The voice has to stay consistent, because the bot is speaking for the community, not for itself. Spending has to be bounded, because one enthusiastic member shouldn't be able to run up anyone's bill. And the answers happen in public. Anyone can scroll up and check what the bot said, so wrong answers get caught by everyone, every participant in the chat room.

Every one of those requirements is plain code: an allowlist, a budget counter, a reply post-filter. Nothing there takes more than an afternoon to write.

The architecture: a coding agent in a container

Sage runs as a single warm container on Modal. Warm means always on, because the container holds a persistent websocket connection to Discord's gateway; if the container slept, member messages would pile up undelivered. Three things live inside it:

1. A small FastAPI app that gives Modal an HTTP endpoint to route to. It doubles as the health check that CI pings after every deploy.
2. The Discord gateway client on its own thread, listening to the event stream.
3. The pi coding agent, installed with npm exactly like you'd install it on your own laptop.

When a member @mentions Sage, here's the entire path their message travels. Discord delivers it over the websocket to the container. A handful of checks run in plain code: is this channel on the allowlist, is this member under their daily ask budget, is the cap on simultaneous agent runs intact. If the checks pass, the question is piped into the pi coding agent over stdin, with its working directory set to the knowledge base and its tools limited to read, grep, find, and ls. The agent searches the notes itself and drafts an answer. A few more lines of plain code post-filter the draft (strip @everyone pings, cap the word count, check for prompt-leak artifacts), and the container posts the reply back to the channel.

Here is the actual agent invocation, trimmed to the essentials:

result = subprocess.run(
    [
        "pi", "--provider", "zai", "--model", model,
        "--tools", "read,grep,find,ls",  # search and read, never mutate
        "--offline", "--no-session",     # hermetic: no startup network calls
        "--mode", "text",
    ],
    input=member_question,  # the question arrives as stdin, not argv
    cwd="/sage-kb",         # the knowledge base IS the working directory
    capture_output=True,
    text=True,
    timeout=120,
)
reply = post_filter(result.stdout)

Notice what Sage doesn't do: keep its own copy of the conversation. When a question comes in, Sage rebuilds the context on the spot: it walks up the reply chain through Discord's message references, and it reads the recent exchange in the channel straight from Discord's API, capped at a few thousand characters. That context rides along with the question, and once the reply posts, Sage keeps nothing. Discord is already the database of the conversation; replicating it would just mean a second copy to keep in sync. It's the same principle HTMX preaches: use the application state you already have as the data storage.

The whole stack is a websocket, a subprocess, and a markdown file. If you've wired up a Discord bot before, you'll recognize every piece. The only new part is the handler: instead of writing request-matching logic yourself, you install a full coding agent and let it do the searching.

The typing indicator, a chef's kiss detail

Discord shows a little "Sage is typing..." indicator while it works, and getting it right was my favorite implementation detail of the entire build.

The typing indicator has a quirk. You trigger it by hitting Discord's typing endpoint, and it expires 10 seconds after the last trigger. But an agentic answer takes somewhere between 15 seconds and 2 minutes, because the agent runs several model turns: search, read, think, draft. Fire the trigger once and members get 10 seconds of typing, then silence while the agent keeps working.

The fix is to re-fire the endpoint on a loop until the answer is ready. discord.py ships a typing() context manager that does exactly this: it holds the loop open for as long as your code runs inside the context, re-firing the trigger every 5 seconds (comfortably inside the 10-second expiry), then closes it out when the reply posts. In Sage, the entire answer path runs inside that context, and the wrapper swallows failures, because a cosmetic indicator must never be the thing that breaks a reply.

Sixty seconds of silence reads like the bot is broken. Sixty seconds of typing reads like the bot is thinking. Perceived latency is trust.

The knowledge base is one hand-curated markdown file

Sage's knowledge base is a single hand-curated markdown file. No vector database, no embeddings, no chunking strategy, no retrieval pipeline to tune. Every entry is one section with a heading that names the topic, like this one from the real thing:

## The Learn Anything retreat

A one-week, in-person retreat for people who want to get on the AI
curve and learn any field. The announced dates are February 14-20,
2027, and the announced location is San Diego, California...

<!-- Verified: 2026-09-20 (deposit terms doc) -->

When the facts change, I edit the file and redeploy. When the same question keeps coming up and the file can't answer it, that's the signal to add an entry. Knowledge base growth becomes content curation, the thing communities already know how to do, instead of pipeline engineering.

The agent answers by searching this file with grep and reading whole sections, deriving search terms from the entire conversation. That matters for follow-ups. If someone asks "when is the retreat" and then says "yeah, tell me more about the build day", the agent works out from context what "the build day" refers to and searches for that.

One rule is baked into the system prompt: if the notes don't ground the answer, Sage replies, "I don't have that in my community notes yet, so I'd rather not guess. Eric or Daniel will know." For a community assistant that's the right default. A confident hallucination about the refund policy loses the room in one message; "go ask a human" costs nothing.

The same pattern keeps showing up in enterprise AI: hand-curated context, agentic search over it, and an honesty rule. The KB stays small enough that grep finds everything instantly, and if it ever outgrows one file, the agent searches a directory of files the same way.

What it costs

Modal bills per second, and a small container idling at default resources meters about 17 cents a day. I pulled the actual billing rows for this post: Sage went live on September 16, and its container has metered about a dollar in the six days since, which paces to about \$5 a month. Modal's Starter plan hands out \$30 a month in free credits, so my out-of-pocket cost is \$0, and every invoice this year confirms it. The model calls cost pennies on top, because Sage runs on a cheap, fast model and short grounded answers don't need a frontier model. That's an always-on knowledge assistant for an entire community, fully covered by free-tier credits.

Kick the tires

I extracted the pattern into a standalone repo, stripped of everything retreat-specific: github.com/ericmjl/sage-pattern.

Inside you'll find the Modal deployment code, a dummy knowledge base you can swap for your own, a README explaining the architecture, an AGENTS.md written for coding agents, and a paste-ready prompt you can hand to your own agent to test-drive the deployment against your Discord server and Modal account.

The models get the headlines. What makes this buildable for one person is that the surrounding tech got boring: one container holding a websocket, one subprocess doing the searching, one markdown file holding what your community knows. Pick a community and go build one!