Serve your agent with FastAPI

Give your Agent an HTTP endpoint so another program can send it work. Each request names a session; requests with the same name continue a conversation stored on disk.

This guide follows Build your own agent. It uses core.Agent directly, with FastAPI managing the model’s lifetime and each request managing its agent stream.

Install

Use Python 3.12+ and an OpenAI API key. Create a project and install Bub with the web server dependencies:

uv init bub-api
cd bub-api
uv add bub fastapi uvicorn

This service uses the base SDK dependencies together with FastAPI and Uvicorn.

To add an existing Bub plugin, follow Reuse application plugins. You can add its tools or selected hooks while keeping the endpoint and request lifecycle below.

Write the service

Save this as app.py. The order tool returns demonstration data; replace its body when connecting your own service.

import asyncio
from collections import defaultdict
from contextlib import asynccontextmanager
from pathlib import Path

import republic
from fastapi import FastAPI, HTTPException, Request
from pydantic import BaseModel

from bub.core import Agent
from bub.core.store import FileTapeStore
from bub.core.tools import Tool


async def lookup_order(order_id: str) -> dict[str, str]:
    """Look up the status of an order."""
    return {"order_id": order_id, "status": "shipped"}


def create_agent() -> Agent:
    return Agent(
        store=FileTapeStore(Path(__file__).resolve().parent / "sessions"),
        tools=[Tool.from_callable(lookup_order)],
        system_prompt=[
            "You are an order assistant. Reply directly to the user.",
            "Use lookup_order to check order status before answering.",
        ],
    )


@asynccontextmanager
async def lifespan(app: FastAPI):
    # The key comes from REPUBLIC_OPENAI_API_KEY.
    async with republic.get_model("openai:gpt-5-mini") as model:
        app.state.agent = create_agent()
        app.state.model = model
        app.state.session_locks = defaultdict(asyncio.Lock)
        yield


app = FastAPI(lifespan=lifespan)


class Task(BaseModel):
    session_id: str
    prompt: str


@app.post("/tasks")
async def run_task(task: Task, request: Request):
    state = request.app.state
    # Runs on one session's tape must not overlap; different sessions run concurrently.
    async with state.session_locks[task.session_id]:
        parts: list[str] = []
        error: str | None = None
        async with state.agent.run([task.prompt], model=state.model, tape=task.session_id) as stream:
            async for event in stream:
                if event.kind == "text":
                    parts.append(str(event.data.get("delta", "")))
                elif event.kind == "error":
                    error = str(event.data.get("message", "Agent failed"))
        if error is not None:
            raise HTTPException(status_code=502, detail=error)
        return {"session_id": task.session_id, "output": "".join(parts)}

The FastAPI lifespan opens the Republic model once and closes it on shutdown. Each request opens an agent stream with async with, which closes the stream and merges the tape fork even when the request’s agent run fails.

The per-session lock keeps two requests from starting with the same incomplete history. Different sessions can run concurrently.

Run it

Set REPUBLIC_OPENAI_API_KEY in your shell, then start the server:

uv run uvicorn app:app

In another terminal, send two requests with the same session ID:

curl -s localhost:8000/tasks -H 'content-type: application/json' \
  -d '{"session_id": "customer-42", "prompt": "Where is order A123?"}'

curl -s localhost:8000/tasks -H 'content-type: application/json' \
  -d '{"session_id": "customer-42", "prompt": "And when did it ship?"}'

The second answer can refer to the first because both runs read the customer-42 tape under sessions/. The example order data only contains a status, so the agent has no shipping date to report.

Prepare for more sessions

The example keeps a lock for each session in memory. A service that creates many sessions should remove idle locks. These locks work within one process; multiple workers sharing a tape store need coordination across processes. That coordination must cover the whole run, including model calls and tape writes.

FileTapeStore keeps history on one host. To use another backend, implement TapeStore or AsyncTapeStore and pass it to Agent(store=...); see Choose where history lives. Give that store’s connections an application lifespan when they need opening and closing.

Extend the service

Add tools or CodeMode to give the agent new actions. Use event handlers to observe calls or enforce a decision before execution. Choose what the model sees through TapeContext.