GitHub - PacktPublishing/Operational-AI-with-Docker: Operational AI with Docker · GitHub
/" data-turbo-transient="true" />
Skip to content
Search or jump to...
Search code, repositories, users, issues, pull requests...
-->
Search
Clear
Search syntax tips
Provide feedback
--><br>We read every piece of feedback, and take your input very seriously.
Include my email address so I can be contacted
Cancel
Submit feedback
Saved searches
Use saved searches to filter your results more quickly
-->
Name
Query
To see all available qualifiers, see our documentation.
Cancel
Create saved search
Sign in
/;ref_cta:Sign up;ref_loc:header logged out"}"<br>Sign up
Appearance settings
Resetting focus
You signed in with another tab or window. Reload to refresh your session.<br>You signed out in another tab or window. Reload to refresh your session.<br>You switched accounts on another tab or window. Reload to refresh your session.
Dismiss alert
{{ message }}
PacktPublishing
Operational-AI-with-Docker
Public
Notifications<br>You must be signed in to change notification settings
Fork
Star
main
BranchesTags
Go to file
CodeOpen more actions menu
Folders and files<br>NameNameLast commit message<br>Last commit date<br>Latest commit
History<br>65 Commits<br>65 Commits
chap-01
chap-01
chap-02
chap-02
chap-03
chap-03
chap-04
chap-04
chap-05
chap-05
chap-06
chap-06
chap-07
chap-07
chap-08
chap-08
chap-09
chap-09
.gitignore
.gitignore
LICENSE
LICENSE
README.md
README.md
View all files
Repository files navigation
Operational AI with Docker
This is the code repository for Operational AI with Docker: LLMOps, Agents and Multi-Model Systems with Docker and Kubernetes , published by Packt.
Build, deploy and scale production-ready AI applications using Docker's integrated AI toolkit.
What this book is about
If you've ever wanted to take an AI app from "works on my laptop" to something you can actually run in production, this book is for you. It walks through the full lifecycle running local LLMs, wiring them into real applications, integrating external tools through MCP, building autonomous agents and finally orchestrating fleets of agents on Kubernetes all using Docker's AI tooling.
You'll work hands-on with Docker Desktop, Docker Model Runner, MCP Gateway, Docker Hardened Images, kagent and you'll see how the same containers you already know can carry AI workloads safely and at scale.
What you'll learn
Run and optimize local LLMs with Docker Model Runner
Integrate AI applications with external systems using MCP (Model Context Protocol)
Deploy MCP servers securely with Docker MCP Gateway
Build autonomous AI agents with multi-agent architectures
Implement production security with Docker Hardened Images
Monitor AI workloads with Prometheus and Grafana
Integrate AI with GitHub, Slack, Kubernetes and databases
Scale AI applications from development to production
Implement enterprise security patterns for AI deployments
Automate AI workflows with Docker Compose and orchestration
Chapter guide
Each chapter has its own folder with runnable code and a chapter-specific README.md. Click any chapter title to jump straight to its code.
Chapter<br>What's inside
Introduction to Containerisation for AI<br>Docker fundamentals through an AI/ML lens — images, containers, registries and how containers compare to VMs. Two small examples (tiny-service-container, tiny-training-run) get you comfortable with docker run and docker build before things get serious.
Understanding AI Models in Docker<br>The bridge between "I know Docker" and "I know how to ship models". Covers OCI artifacts, GGUF format, quantization and the new Compose models: provider syntax for declaring model dependencies alongside your services.
Model Serving with Docker Model Runner<br>The heart of the local-LLM workflow. Pull models from Docker Hub, hit them with the OpenAI-compatible API, build a React chatbot and wire up Prometheus, Grafana and Jaeger for observability. Includes Python and JavaScript SDK examples.
Docker Offload<br>Push the heavy stuff — model export, quantization, batch jobs — into purpose-built containers so your main app stays snappy. Includes a working export_and_quantize.py pipeline.
Running ML Container Models on Kubernetes<br>Take your containerized models to a real cluster. Manifests, resource limits, autoscaling and a small ML ecosystem you can deploy end to end.
Protocol-Based AI Integration with MCP<br>Give your models hands. Use Docker MCP Gateway and the MCP Catalog (270+ servers) to connect AI to databases, APIs and tools — with proper isolation, secret management and OAuth.
Building Autonomous AI Agents<br>Move from "AI that answers" to "AI that does". Container-isolated agents, agent-to-agent communication, discovery, memory/state, reasoning, tool access and sandboxing — each in its own subfolder.
Multi-Model and Multi-Agent Architectures<br>When one agent isn't enough. Route tasks by complexity, coordinate specialized models and...