Production Deployment¶

The bar: you can take an AI service running on your laptop, package it into a container, ship it to a cloud platform behind a real URL, and wire up the pipeline so the next change deploys itself — without waking anyone up at 2 a.m.

Getting a model into production is a different job than getting it to run. On your machine you have the right Python version, the weights already downloaded, and every environment variable set by hand. Production has none of that unless you build it in. The gap between "works on my box" and "serves thousands of veterans reliably" is exactly the ground this module covers: containers, cloud runtimes, automated pipelines, and the discipline of a launch checklist.

Treat this as the last mile. The model is the easy part now. Shipping it so it stays up, scales when traffic spikes, and rolls back cleanly when you break something — that is the work that separates a demo from a product.

24.1 Docker for AI¶

Objective: package an AI application into a container that runs the same everywhere.

A container freezes your app and everything it needs — Python, system libraries, model dependencies — into one image that runs identically on your laptop and in the cloud. For AI work this matters more than usual, because your dependency tree is heavy and version-sensitive; a mismatched torch or numpy in production is a whole class of bug you can delete by shipping the exact environment you tested.

The Dockerfile is the recipe. The best ones are boring: pin a slim base image, install dependencies before copying source so the layer cache does its job, and never bake secrets into the image. The mistake self-taught engineers hit is a fat final image — pulling in build toolchains, caches, and dev dependencies that balloon a container to several gigabytes and slow every deploy. Multi-stage builds fix this: compile and install in a build stage, then copy only the finished artifacts into a clean runtime stage.

For dependency management, install from a locked requirements file so the build is reproducible. For local development, Docker Compose lets you stand up your app plus its database and other services with one command, mirroring production topology without cloud costs.

FROM python:3.12-slim AS build
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

FROM python:3.12-slim
WORKDIR /app
COPY --from=build /usr/local/lib/python3.12/site-packages /usr/local/lib/python3.12/site-packages
COPY . .
CMD ["python", "-m", "uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8080"]

Drill:

• Write a multi-stage Dockerfile and compare final image size against a single-stage build.
• Order layers so pip install is cached when only source changes.
• Pin dependencies with a locked requirements.txt and pip install --no-cache-dir.
• Shrink an image with a -slim base and a .dockerignore.
• Bring up app plus database locally with docker compose up.

24.2 Google Cloud Run¶

Objective: deploy a container to Cloud Run and configure it for real traffic.

Cloud Run takes a container image and runs it as a managed service — you hand it an image, it hands you an HTTPS URL, and it scales the number of instances up and down with demand, including down to zero when nobody is calling. You do not manage servers. For an AI backend that sees bursty traffic, this is the cheapest way to get production-grade hosting without babysitting infrastructure.

Service configuration is where the details live. Your container must listen on the port Cloud Run gives it (the PORT env var, typically 8080) or it will never come up — that is the number-one first-deploy failure. Environment variables carry non-sensitive config; secrets like API keys and database passwords come from Secret Manager, never from the image or the repo. Scaling settings — min and max instances, concurrency, memory, and CPU — let you trade cost against latency; a model that needs to stay warm wants a minimum instance count above zero.

Once the service runs, you map a custom domain so users hit your name instead of a generated URL, and you connect Cloud SQL through the built-in connector when your app needs a managed Postgres database.

Drill:

• Deploy an image with gcloud run deploy and confirm the service listens on $PORT.
• Set plain config via --set-env-vars and pull secrets with --set-secrets from Secret Manager.
• Tune --min-instances, --max-instances, --concurrency, and --memory.
• Map a custom domain to the service.
• Attach a Cloud SQL instance with --add-cloudsql-instances.

24.3 CI/CD for AI¶

Objective: automate the path from a merged commit to a running deployment.

Continuous integration and continuous deployment mean the pipeline does what you used to do by hand: run the tests, build the image, push it, and deploy — every time, the same way. Manual deploys drift and get skipped under pressure; an automated pipeline is the only version that stays honest when you are tired.

GitHub Actions is the workhorse. A workflow triggers on push or pull request, sets up Python, installs dependencies, and runs your test suite so nothing broken ever reaches production. On a merge to the main branch, the next stage builds the container, pushes it to a registry, and deploys the new image to Cloud Run automatically.

The piece beginners skip is rollback. Every deploy should be reversible: keep tagged image versions and know the one command that points the service back at the last good revision. A deploy strategy without a tested rollback is not a strategy — it is a hope. Store your cloud credentials as encrypted repository secrets, never in the workflow file.

Drill:

• Write a GitHub Actions workflow that runs pytest on every pull request.
• Gate deploys on the test job passing.
• Build and push a container to a registry from the workflow.
• Deploy to Cloud Run on merge to the main branch.
• Roll back to a previous Cloud Run revision with one command.

24.4 Production Checklist¶

Objective: launch an AI system safely by clearing a checklist, not by crossing your fingers.

A launch is a checklist, not a moment of courage. Before real users touch the system, walk the list. A security review confirms secrets are in a manager and not the repo, endpoints are authenticated, and dependencies have no known critical vulnerabilities. Performance benchmarks establish what "normal" latency and throughput look like, so you can tell later when something has gone wrong.

Monitoring verification means you have actually confirmed that logs, metrics, and alerts fire — not just that you configured them. The classic failure is discovering during an outage that the alert was never wired up. An incident response plan answers, in advance, who gets paged, where you look first, and how you roll back — decided while calm, not invented mid-fire.

Last, documentation completeness: a runbook that lets someone who is not you deploy, debug, and recover the system. If the only operator is the person who built it, you have a single point of failure with a heartbeat.

Drill:

• Run a dependency vulnerability scan and confirm no secrets are committed.
• Capture baseline latency and throughput numbers under load.
• Trigger a test alert and confirm it actually reaches a human.
• Write a one-page incident runbook: who, where, how to roll back.
• Document deploy and recovery steps well enough for a teammate to follow.

Hands-On Lab: Production Launch¶

Objective: deploy a complete AI application end to end, from container to monitored service.