FastAPI: Production AI APIs¶

The bar: you can stand up a typed, documented, streaming AI API in Python — with auth, rate limiting, and clean shutdown — and hand another engineer a spec they can generate a client from.

Every AI feature you build eventually needs a server between the browser and the model. That server takes a request, checks who's asking, calls the model, streams the answer back, and doesn't fall over under load. FastAPI is the tool the industry reaches for because it is fast, it is type-safe through Pydantic, and it writes its own API documentation from your code instead of from a wiki nobody updates.

This module takes you from a bare endpoint to a production chat API. The skills carry: the same request/response discipline, streaming, and middleware thinking apply whether you deploy FastAPI on its own or wire a model behind a frontend.

16.1 FastAPI Fundamentals¶

Objective: build real API endpoints with FastAPI.

An app is an APIRouter (or the root app) plus functions decorated with @app.get, @app.post, and friends. Structure matters early: keep routers per feature, not one 800-line file. Path parameters (/users/{id}) and query parameters (?limit=20) are declared as plain function arguments, and FastAPI reads your type hints to parse and validate them for free.

Request bodies are where Pydantic earns its keep. You declare a BaseModel, and an invalid payload gets rejected with a 422 and a precise error before your code runs. Pair that with a response_model so the shape you return is enforced and documented too. Set explicit status_code values — a POST that creates a resource should return 201, not a silent 200.

The common self-taught mistake is skipping the dependency injection system and passing a database handle or the current user around by hand. Depends() lets you declare "this endpoint needs an authenticated user" once and reuse it everywhere, which keeps auth and resource setup out of your business logic.

Drill:

• Scaffold an app with APIRouter and mount it on the root app.
• Read path params (/items/{item_id}) and query params with typed function arguments.
• Define a BaseModel request body and a response_model on the route.
• Set status_code=201 on creates and return typed responses.
• Write a Depends() dependency and inject it into two endpoints.

16.2 OpenAPI Integration¶

Objective: generate and customize the API documentation your consumers actually use.

FastAPI emits an OpenAPI 3.1 spec at /openapi.json and renders interactive docs at /docs with zero extra work. That spec is not decoration — it is the contract. Every type hint and Pydantic field you wrote in 16.1 becomes schema automatically, which is why the typing discipline pays off twice.

You customize it where it counts. Add title, description, and tags so endpoints group sensibly. Attach examples and field descriptions to your Pydantic models so the docs show a real payload, not an empty skeleton. A consumer who can see a working example in /docs files far fewer "how do I call this" messages.

The payoff is client generation: point a tool at /openapi.json and generate a typed TypeScript or Python client. The gotcha is drift — if you hand-write docs alongside the code, they rot. Let the schema be generated and keep your annotations honest.

Drill:

• Open /docs and /openapi.json and read the generated schema.
• Add title, description, and route tags to the app.
• Attach examples and field descriptions to a Pydantic model.
• Generate a typed client from the spec with an OpenAPI generator.

16.3 Streaming Responses¶

Objective: stream LLM output token-by-token instead of making the user wait for the full answer.

A model that takes eight seconds to finish feels broken if the user stares at a spinner the whole time. Streaming fixes the perception and the reality: you send tokens as they arrive. FastAPI's StreamingResponse wraps an async generator, and the standard wire format for this is Server-Sent Events (SSE) — a long-lived HTTP response where each chunk is a data: line.

The pattern is an AsyncIterator: an async def generator that yields chunks as the model produces them. Set the media type to text/event-stream so browsers and clients treat it as a stream. On the client, you consume it with the Fetch API's readable stream or an EventSource, appending each chunk as it lands.

The gotcha nobody warns you about is error handling mid-stream. Once you've sent the first byte, you can't change the status code to 500 — the header's gone. You handle failures inside the generator, emit an error event on the stream, and close cleanly. Test the unhappy path deliberately.

Drill:

• Return a StreamingResponse backed by an async def generator.
• Format chunks as SSE (data: ...\n\n, text/event-stream).
• Build an AsyncIterator that yields model output incrementally.
• Handle an exception raised partway through a stream.
• Consume the stream client-side with fetch / EventSource.

16.4 Middleware and Security¶

Objective: secure and observe your endpoints before they face the open internet.

Middleware runs on every request, which makes it the right place for cross-cutting concerns. First is CORS: a browser will block your frontend from calling the API unless you configure CORSMiddleware with the exact origins allowed — and allow_origins=["*"] is a footgun in production, not a shortcut.

Auth for an AI API usually starts with an API key checked in a Depends() dependency or middleware, rejecting anything without a valid key. Layer rate limiting on top so one caller can't drain your model budget or knock the service over. Add request logging so you can see who called what and how long it took — you cannot debug production traffic you never recorded.

Input sanitization is the step people skip until it bites them. Pydantic validates shape and type, but you still bound the things that cost money and memory: cap prompt length, reject oversized payloads, and never echo raw user input into logs or errors unfiltered.

Drill:

• Configure CORSMiddleware with an explicit origin allowlist.
• Enforce an API key via a Depends() dependency.
• Add rate limiting keyed by API key or client IP.
• Write request logging middleware capturing method, path, and latency.
• Bound and sanitize inputs (max prompt length, payload size).

16.5 Background Tasks¶

Objective: handle work that shouldn't block the response or the shutdown.

Some work doesn't need to finish before you answer the caller — writing an audit record, sending a notification, warming a cache. BackgroundTasks lets you return the response immediately and run that work after. It is not a job queue; for heavy or retryable work you'd reach for a real worker, but for quick post-response side effects it's exactly right.

Startup and shutdown are managed with the lifespan context: open your database pool, model client, or HTTP session once when the app boots, hand it to requests, and close it on the way down. This is also where connection pooling lives — you create one pool at startup, not one connection per request, or you'll exhaust the database under load.

Graceful shutdown is the part that separates a hobby API from a production one. When the platform sends a stop signal, you want in-flight requests to finish and pools to close cleanly. The lifespan teardown block is where that happens; skip it and you leak connections and drop live requests on every deploy.

Drill:

• Queue post-response work with BackgroundTasks.
• Manage startup/shutdown with a lifespan context manager.
• Create a connection pool once at startup and share it via Depends().
• Close pools and drain work in the lifespan teardown for graceful shutdown.

16.6 Hands-On Lab: Chat API¶

Objective: build a production chat API with streaming, auth, and generated docs.