I could build a local AI agent without Docker. A vector database installed locally, a Python script, a local server, all wired together by hand. Plenty of people build it exactly that way, and for many setups, it works fine.

This post isn’t a claim that Docker is the answer for building AI agents in general. It’s one specific case where I think it earns its place, and the reasoning behind that is worth more to you than the agent itself.

The project I’m using to answer that is Mino—a local agent that indexes my own PDFs and lets me ask questions about them, using a model running on my own machine through Ollama.

There aren’t API keys or uploading anything anywhere. Instead, there are three containers orchestrated with Docker Compose:

1. vector database
2. document-ingestion service
3. small API

Each does one job, and all are together in a single YAML file.

This is Part Two of a series. Part One used Portainer as a mirror to actually see Docker’s images and containers instead of trusting docker ps blindly. That gave me the vocabulary. Here, I find out whether I actually understood it by making several pieces talk to each other.

Note 👀
I’m assuming you’ve got Ollama installed already and have at least one model pulled (ollama pull gemma4 or similar). This post treats Ollama as something already running on your machine—we’re not installing it together, we’re teaching Docker to talk to it.

Explore: The Ultimate Guide To Re-Engineering My Portfolio’s RAG Chatbot

The Problem With Building Agent Infrastructure by Hand

An agent like this needs at least three moving pieces:

1. Something to store and search document embeddings
2. Something to watch a folder and process new files
3. And something to actually answer questions

Without Docker, each of those pieces needs its own setup on my machine. This means its own Python version, dependencies, and way of running in the background.

If I ever want to run this on a different machine, or hand it to someone else, I’m now writing a setup guide instead of shipping software.

And that’s the good case, where nothing conflicts 🙂‍↕️

The more common case: my vector database wants one version of a library, and something else on my machine already has a different version installed globally, and now I’m debugging an environment instead of building an agent. This is the exact “dependency hell” every developer eventually runs into, and it gets worse, not better, once AI tooling is involved. (This is also what I run into with my team as shared in Part One!)

There’s a second problem that’s specific to this project: privacy.

If I’m indexing anything remotely sensitive, like financial documents, medical paperwork, whatever, running everything as loose local processes makes it easy to lose track of what’s actually talking to what. You typically end up accidentally leaving a service listening to a port you totally forgot about (happens to me every darn time).

I wanted something where I could look at one file and know exactly what’s running, what it can reach, and what it can’t.

Note 🤌
This is also why I didn’t reach for something like OpenWebUI or a no-code RAG tool. Those are solid options if the goal is “get a chatbot running fast,” but they’d hide the exact plumbing I’m trying to understand. The point of this post isn’t the fastest path to a working agent. It’s seeing what Docker is actually doing for me when I build the plumbing myself.

The Solution: Let Docker Compose Own the System

Docker Compose solves the “several things that need to agree on how they talk to each other” problem by reading one YAML file and standing up (or tearing down) every container it describes, in order, on a shared network it creates for you.

Instead of three separately-managed processes with their own dependency trees, I get three isolated containers—a database, a background service, and an API—each with only the dependencies it needs, each unaware of anything happening on my machine outside itself.

If it works on my laptop, it works anywhere Docker runs. That’s the whole pitch, and it’s also the exact thing Part One never got to demonstrate, because Portainer was watching one container, not orchestrating several.

Here’s the shape of the system before any code:

• The Memory is a vector database. Recall from previous THT projects that a vector database stores documents as long lists of numbers (embeddings) that represent meaning. This lets the system search by “what’s this about” instead of matching exact keywords. I’m using Redis with its vector search module, since I can spin it up with a single official image.
• The Librarian is a custom Python container that watches a local folder for new PDFs, parses them, and sends the text to Ollama to be converted into those embeddings, which then get stored in Redis.
• The Front Door is a small FastAPI service that’s the only thing I actually talk to. I ask it a question, it pulls the most relevant chunks out of Redis, and sends both the question and the chunks to Ollama for an answer.

And then there’s Ollama itself, which isn’t a container in this setup. It’s already running on my host machine, and my containers reach out to it like any other API. Getting that connection right is its own section further down, because it’s the one place Docker’s isolation actually works against you for a second, until you know the trick 💁‍♀️

To sum up the plan: Three containers that each do one job, talking to each other over a network Compose creates automatically, plus one thing running outside Docker that they all depend on.

Everything from here is building each piece and teaching them to find each other.

Hey! This walkthrough creates a working first build which means further refinements and enhancements are yours to explore. In fact, the repo link at the end will show you my own initial enhancements. But if you’re building this out and your intuition is screaming there’s a better way to do something, go ahead and do it. Only way to learn! Plus, revisiting this I can see what I’d change, especially with the data parsing and extraction before sending anything to AI… take a guess if you see it too 🙃

How To Build It

The blueprint above consists of three containers and one outside connection, but the order I build them isn’t random.

1. A container with a bad security posture is still a bad security posture even if it works, so that comes first.
2. Then the network, since nothing should be able to reach vector-db or librarian except the pieces that need to.
3. Then the one deliberate hole in that network: the bridge out to Ollama.
4. Secrets and persistence come last, because they only matter once the first three pieces are actually running and talking to each other.

Before touching a Dockerfile, open your terminal and set up the folder structure everything else in this post lives in:

mkdir mino
cd mino
mkdir librarian api secrets
mkdir librarian/src api/src

You should have:

mino/
├── librarian/
│   └── src/
├── api/
│   └── src/
└── secrets/

Note ⚠️
From here on we run commands from inside the mino/ folder, unless I explicitly say to cd somewhere else. If a command fails with something like “no such file or directory,” the first thing to check is whether you’re actually standing in the folder you think you are—run pwd to confirm.

Explore: How To Build A Wicked Cool App With React + Flask –– pre-AI project with PDF parsing (it might need a nudge on Render for the demo) 🥲

Step 1: Harden the Containers Before Writing Any Logic

Before I write a single line of ingestion logic, I want the container itself built right.

In Part One, I glossed over this entirely, because Portainer’s install command handled it for me. Here, I’m writing my own Dockerfile from scratch, so the mistakes are mine to make or avoid 🙃

I’m building the Librarian service first, since it’s the one that actually turns PDFs into something searchable. Create its dependency file:

touch librarian/requirements.txt

Open librarian/requirements.txt in your editor and add:

requests==2.31.0
pypdf==4.0.1
redis==5.0.1
watchdog==4.0.0
numpy==1.26.4

• requests talks to Ollama’s API
• pypdf reads PDF files
• redis is the client library for talking to our vector database
• watchdog is what lets the Librarian notice when a new file lands in a folder instead of me having to run it manually every time (super important)

Now create the Dockerfile:

touch librarian/Dockerfile

1. Start from a smaller base.

Every Dockerfile starts with a FROM line, and that choice is important.

• FROM python:3.11 pulls in a full Debian-based image with a ton of tooling I don’t need at runtime.
• FROM python:3.11-slim strips most of that out.

Docker also offers Docker Hardened Images (DHI), a catalog of minimal, actively patched base images built specifically to shrink what security folks call the “attack surface” (the total set of code and tools an attacker could exploit if they got in).

For a hobby project, -slim is a fine starting point. If I were shipping this for other people, DHI would be the next step up.

2. Don’t run as root.

By default, a container runs as root unless you tell it otherwise. This means that if something inside that container gets compromised, whatever broke in has root-level access to everything the container can touch.

I confess to skipping this step on other projects for a long time, and I’m not proud of it. It’s one of the most common, and most avoidable, container security mistakes out there 😬

We’ll build best practices here; paste this into librarian/Dockerfile:

# Stage 1: build dependencies
FROM python:3.11-slim AS builder
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir --user -r requirements.txt

# Stage 2: the actual runtime image
FROM python:3.11-slim
RUN useradd --create-home librarian
WORKDIR /app
COPY --from=builder /root/.local /home/librarian/.local
COPY src/ ./src/
USER librarian
ENV PATH=/home/librarian/.local/bin:$PATH

CMD ["python", "src/ingest.py"]

Here’s what each line is doing, top to bottom:

• FROM python:3.11-slim AS builder starts a temporary stage named builder, just for installing dependencies
• WORKDIR /app sets the working directory inside the container; everything after this runs relative to /app
• COPY requirements.txt . copies just that one file in first, so Docker can cache this layer and skip reinstalling packages if only your code changes later
• RUN pip install --no-cache-dir --user -r requirements.txt installs the packages into the builder stage
• The second FROM python:3.11-slim starts a brand-new, clean stage; none of the build tools from builder carry over automatically
• RUN useradd --create-home librarian creates the non-root user
• COPY --from=builder /root/.local /home/librarian/.local pulls only the installed packages from the first stage, nothing else
• COPY src/ ./src/ copies your actual application code in
• USER librarian switches to the non-root user for everything after this line
• CMD [...] is what runs when the container starts

This is a multi-stage build in action, using more than one FROM line, where the first stage installs build tools and compiles dependencies, and the second, leaner stage copies over only the finished result.

The Librarian container doesn’t need a compiler sitting around once it’s built, and skipping that step is the difference between an image that’s 900MB and one that’s a fraction of that.

Tip 💯
If your app needs to write files (logs, temp files, downloaded models), make sure the non-root user actually owns those directories, or you’ll spend twenty minutes debugging a “permission denied” error before you remember why. Ask me how I know.

3. Clean the build context.

When you run docker build, Docker sends your entire project folder to the build process as context, unless you tell it not to.

That means .env files, .git history, and anything else sitting in that folder can end up readable inside your image layers, even if you never explicitly COPY them.

To avoid this, let’s create a .dockerignore right next to the Dockerfile:

touch librarian/.dockerignore

Add this to it:

.env
.git
__pycache__/
*.pyc
.venv/

Note 👁️
I didn’t fully appreciate this one until I went digging through an old image with docker history and found a .env file baked into a layer from months back. It’s a quiet mistake—nothing breaks, nothing errors, it’s just sitting there. A .dockerignore file next to your Dockerfile is the fix, and it’s this easy to forget.

I’m not writing ingest.py‘s actual logic yet since it’ll come together properly once the network and Ollama bridge exist in Step 3 (the script needs both to do anything useful).

For now, drop in a placeholder so the build doesn’t fail on a missing file:

touch librarian/src/ingest.py

# librarian/src/ingest.py
# Placeholder — real ingestion logic gets added once networking is wired up

print("Mino's Librarian is alive. Ingestion logic comes next.")

Then build the Librarian image on its own before moving forward, so if something’s wrong, you know it’s this piece and not something downstream:

docker build -t mino-librarian ./librarian

Verify that the build finishes with Successfully tagged mino-librarian:latest (or the newer Buildx equivalent, naming to docker.io/library/mino-librarian:latest).

If instead you see COPY failed: file not found, double-check that requirements.txt and the src/ folder actually exist inside librarian/ as this almost always means a file got created in the wrong directory.

Step 2: Wire the Network So Only the Front Door Is Exposed

This is the part of the project that actually forced me to understand what Docker Compose networking is doing under the hood, instead of assuming it “just works.”

Before writing the Compose file, I need the API service’s own dependency and Dockerfile too, since Compose is going to reference both services. Create its requirements file:

touch api/requirements.txt

fastapi==0.109.0
uvicorn==0.27.0
redis==5.0.1
requests==2.31.0
numpy==1.26.4

And its Dockerfile with the same hardening pattern as the Librarian:

touch api/Dockerfile

# Stage 1: build dependencies
FROM python:3.11-slim AS builder
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir --user -r requirements.txt

# Stage 2: the actual runtime image
FROM python:3.11-slim
RUN useradd --create-home apiuser
WORKDIR /app
COPY --from=builder /root/.local /home/apiuser/.local
COPY src/ ./src/
USER apiuser
ENV PATH=/home/apiuser/.local/bin:$PATH
CMD ["uvicorn", "src.main:app", "--host", "0.0.0.0", "--port", "8000"]

The CMD here is different from the Librarian’s:

• uvicorn is the server that actually runs a FastAPI app
• --host 0.0.0.0 tells it to accept connections from outside the container (not just from inside it)
• --port 8000 is the port it listens on

I’ll add a placeholder for the API code, same reasoning as before since the real logic comes in Step 3:

touch api/src/main.py

# api/src/main.py
# Placeholder — real query logic gets added once networking is wired up

from fastapi import FastAPI

app = FastAPI()

@app.get("/health")
def health():
    return {"status": "Mino's front door is open"}

And its own .dockerignore:

touch api/.dockerignore

.env
.git
__pycache__/
*.pyc
.venv/

Now the piece that ties everything together. Create the Compose file at the top level of mino/, not inside librarian/ or api/:

touch docker-compose.yml

By default, every service in a docker-compose.yml file can talk to every other service by name. Compose creates a private network behind the scenes and gives each container a DNS entry (an internal address other containers can look up by name) matching its service name.

But “can talk to everything” isn’t the same as “should be reachable from outside” 💁‍♀️

I only want the API exposed to my own machine; the vector database and the ingestion service should only be reachable from inside that internal network.

services:
  vector-db:
    image: redis/redis-stack:latest
    networks:
      - internal

  librarian:
    build: ./librarian
    networks:
      - internal
    depends_on:
      - vector-db

  api:
    build: ./api
    ports:
      - "8000:8000"
    networks:
      - internal
    depends_on:
      - vector-db

networks:
  internal:

Only api has a ports mapping. vector-db and librarian are only reachable by other containers on the internal network, not from my browser or anything outside Docker.

Tip: Use this pattern any time a service is a backend dependency other containers need but you personally never touch directly. In such a case, it should never get a ports line at all.

To check our work so far, from the mino/ folder, bring the whole system up for the first time:

docker compose up --build

You should see three containers start in your terminal output: mino-vector-db-1, mino-librarian-1, and mino-api-1 (Compose names them using your folder name as a prefix).

Open a second terminal tab and run:

# RUN: curl <http://localhost:8000/health>

You should get back {"status":"Mino's front door is open"}.

Tip 🪲
If instead you get curl: (7) Failed to connect, the API container likely isn’t running. Check docker compose ps to see its status, and docker compose logs api to see why it exited.

Once you’ve confirmed that, stop everything with Ctrl+C in the first terminal, or run docker compose down from the second.

Step 3: Bridge the Gap to Ollama on the Host

Ollama isn’t a container here because it’s a process running on my host machine, outside Docker entirely.

So how does a container reach something that isn’t part of its own network? 🤔

The answer is a special DNS name Docker provides for exactly this situation: host.docker.internal. It resolves to your host machine’s IP address from inside any container, so instead of hardcoding an IP that might change, my API service points at http://host.docker.internal:11434 (Ollama’s default port) and Docker handles the rest.

Update docker-compose.yml to add an environment block to the api service:

services:
  api:
    build: ./api
    ports:
      - "8000:8000"
    networks:
      - internal
    depends_on:
      - vector-db
    environment:
      - OLLAMA_HOST=http://host.docker.internal:11434

Tip 👇
host.docker.internal works out of the box on Docker Desktop for Mac and Windows. On Linux, you may need to add extra_hosts: - "host.docker.internal:host-gateway" to the service in your Compose file for it to resolve. I’m on Mac for this build, so I can’t personally confirm the exact behavior on a Linux host. If you hit a connection refused error there, that line is the first thing I’d check!

This is the moment Docker’s isolation, which is the whole reason to use it, briefly looks like it’s working against you.

In fact, it isn’t. It’s doing exactly what it’s supposed to. You just have to explicitly open one door.

Now the real logic can go in. Replace the placeholder in api/src/main.py:

# api/src/main.py

import os
import requests
from fastapi import FastAPI

app = FastAPI()
OLLAMA_HOST = os.environ.get("OLLAMA_HOST", "<http://host.docker.internal:11434>")

@app.get("/health")
def health():
    return {"status": "Mino's front door is open"}

@app.post("/ask")
def ask(question: str):
    response = requests.post(
        f"{OLLAMA_HOST}/api/generate",
        json={"model": "gemma4", "prompt": question, "stream": False},
    )
    return {"answer": response.json().get("response")}

OLLAMA_HOST is read from the environment variable we just set in Compose, with a fallback in case it’s ever missing.

The /ask route sends the question straight to Ollama for now. This is the bare-minimum version, before Mino actually pulls anything from Redis. That’s coming up in the next section.

Let’s rebuild just the API service and bring the system back up:

docker compose up --build api

Verify from your second terminal:

# RUN: curl -X POST "<http://localhost:8000/ask?question=hello>"

You should get a JSON response with an answer field containing whatever Ollama replied. In my case: {"answer":"Hello! How can I help you today?"}.

Tip: If you instead get a Connection refused or a timeout, confirm Ollama is actually running on your host machine (ollama list in a terminal outside Docker should show your installed models). The container can only bridge to Ollama if Ollama itself is up!

Step 4: Give the Librarian Something to Actually Index

The placeholder in ingest.py proves the container runs. It doesn’t do anything yet.

But we’re about to change that. The Librarian:

1. Watches a folder for PDFs
2. Pulls text out of them
3. Turns that text into embeddings via Ollama
4. Stores everything in Redis so the API can search it later

First, the watched folder needs to exist and be reachable from inside the container, which it currently isn’t.

Create a documents/ folder at the top level of mino/, next to librarian/ and api/:

mkdir documents

Mount that folder into the Librarian container, and give the vector database a real network alias while you’re at it, by updating docker-compose.yml:

services:
  vector-db:
    image: redis/redis-stack:latest
    networks:
      - internal

  librarian:
    build: ./librarian
    networks:
      - internal
    depends_on:
      - vector-db
    volumes:
      - ./documents:/app/documents
    environment:
      - OLLAMA_HOST=http://host.docker.internal:11434
      - REDIS_HOST=vector-db

./documents:/app/documents is a bind mount, which is different from the named volume you’ll add for Redis in Step 6.

A bind mount links a specific folder on your machine directly into the container, so dropping a PDF into mino/documents/ on your laptop makes it instantly visible at /app/documents inside the Librarian container.

REDIS_HOST=vector-db works because Compose’s internal DNS automatically resolves service names. vector-db is reachable by that name from any other service on the internal network, no IP address needed.

Now replace the entire placeholder in librarian/src/ingest.py:

# librarian/src/ingest.py

import os
import time
import requests
import numpy as np
from pypdf import PdfReader
from redis import Redis
from redis.commands.search.field import TextField, VectorField
from redis.commands.search.indexDefinition import IndexDefinition, IndexType
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler

OLLAMA_HOST = os.environ.get("OLLAMA_HOST", "<http://host.docker.internal:11434>")
REDIS_HOST = os.environ.get("REDIS_HOST", "vector-db")
WATCH_FOLDER = "/app/documents"
EMBED_MODEL = "nomic-embed-text"
VECTOR_DIM = 768
INDEX_NAME = "mino_docs"
DOC_PREFIX = "doc:"

redis_client = Redis(host=REDIS_HOST, port=6379, decode_responses=False)

def ensure_index():
    """Create the vector index once, if it doesn't already exist."""
    try:
        redis_client.ft(INDEX_NAME).info()
        print(f"Index '{INDEX_NAME}' already exists, skipping creation.")
    except Exception:
        schema = (
            TextField("content"),
            TextField("source"),
            VectorField(
                "embedding",
                "HNSW",
                {
                    "TYPE": "FLOAT32",
                    "DIM": VECTOR_DIM,
                    "DISTANCE_METRIC": "COSINE",
                    "M": 16,
                    "EF_CONSTRUCTION": 200,
                },
            ),
        )
        definition = IndexDefinition(prefix=[DOC_PREFIX], index_type=IndexType.HASH)
        redis_client.ft(INDEX_NAME).create_index(fields=schema, definition=definition)
        print(f"Created index '{INDEX_NAME}'.")

def chunk_text(text: str, max_chars: int = 1000) -> list[str]:
    """Split text into rough chunks so embeddings stay meaningful and small."""
    paragraphs = [p.strip() for p in text.split("\n\n") if p.strip()]
    chunks = []
    current = ""
    for para in paragraphs:
        if len(current) + len(para) < max_chars:
            current += " " + para
        else:
            if current:
                chunks.append(current.strip())
            current = para
    if current:
        chunks.append(current.strip())
    return chunks

def get_embedding(text: str) -> list[float]:
    """Call Ollama's embed endpoint and return a single embedding vector."""
    response = requests.post(
        f"{OLLAMA_HOST}/api/embed",
        json={"model": EMBED_MODEL, "input": text},
        timeout=30,
    )
    response.raise_for_status()
    return response.json()["embeddings"][0]

def index_pdf(filepath: str):
    """Read a PDF, chunk it, embed each chunk, and store it in Redis."""
    filename = os.path.basename(filepath)
    print(f"Indexing {filename}...")

    reader = PdfReader(filepath)
    full_text = "\n\n".join(page.extract_text() or "" for page in reader.pages)

    chunks = chunk_text(full_text)
    if not chunks:
        print(f"No extractable text found in {filename}, skipping.")
        return

    for i, chunk in enumerate(chunks):
        embedding = get_embedding(chunk)
        vector_bytes = np.array(embedding, dtype=np.float32).tobytes()
        key = f"{DOC_PREFIX}{filename}:{i}"
        redis_client.hset(
            key,
            mapping={
                "content": chunk,
                "source": filename,
                "embedding": vector_bytes,
            },
        )

    print(f"Indexed {filename} as {len(chunks)} chunk(s).")

class PDFHandler(FileSystemEventHandler):
    def on_created(self, event):
        if not event.is_directory and event.src_path.lower().endswith(".pdf"):
            # Give the file a moment to finish writing to disk before reading it
            time.sleep(1)
            index_pdf(event.src_path)

def index_existing_files():
    """Catch any PDFs already sitting in the folder before watching started."""
    for filename in os.listdir(WATCH_FOLDER):
        if filename.lower().endswith(".pdf"):
            index_pdf(os.path.join(WATCH_FOLDER, filename))

if __name__ == "__main__":
    ensure_index()
    index_existing_files()

    event_handler = PDFHandler()
    observer = Observer()
    observer.schedule(event_handler, WATCH_FOLDER, recursive=False)
    observer.start()
    print(f"Mino's Librarian is watching {WATCH_FOLDER} for new PDFs.")

    try:
        while True:
            time.sleep(1)
    except KeyboardInterrupt:
        observer.stop()
    observer.join()

That’s a lot of new code, so here’s what each piece is actually doing:

• ensure_index() checks whether the Redis vector index already exists before creating it. Without this check, restarting the container would attempt to create the same index twice and fail. The VectorField schema is where 768 and COSINE matter—this has to match whatever nomic-embed-text actually outputs, or every search later returns nonsense.
• chunk_text() breaks a full PDF’s text into smaller pieces along paragraph breaks. This matters because embedding an entire 20-page PDF as one vector loses all specificity. Chunking means Mino can later find the one paragraph that answers your question, not just “some document, somewhere, might be relevant.”
• get_embedding() calls Ollama’s /api/embed endpoint. Note this is /api/embed, not the older /api/embeddings, which Ollama has deprecated in favor of this one. It returns a list of embeddings even for a single input, so ["embeddings"][0] pulls out the one we want.
• index_pdf() does the actual work of extracting the text with pypdf, chunking it, embedding each chunk, and storing it in Redis as a hash with the raw text, the source filename, and the embedding encoded as raw bytes (np.float32().tobytes()). Redis’ vector fields expect vectors in that exact byte format, not as a plain list.
• PDFHandler and Observer are watchdog‘s way of watching a folder in real time. The one-second time.sleep(1) in on_created exists because a file system can fire a “file created” event before the entire file has finished writing to disk. Without that pause, you’ll occasionally try to read a PDF that’s still half-copied 🙈
• index_existing_files() handles PDFs that were already sitting in documents/ before the container started, since watchdog only notices new events, not files that already existed.

To check, put a real PDF into the folder you created and bring the Librarian up on its own:

docker compose up --build librarian

Now, drop any PDF into mino/documents/ from a separate terminal (or before running the command above), then check the logs:

docker compose logs librarian

You should see Mino's Librarian is watching /app/documents for new PDFs. followed by Indexing yourfile.pdf... and Indexed yourfile.pdf as N chunk(s).

Mini troubleshooting:

• If instead you see a ConnectionError pointing at Ollama, double-check ollama pull nomic-embed-text actually completed on your host by running ollama list to confirm the model is there.
• If you see nothing happen at all when you drop in a PDF, confirm the bind mount is correct by running docker compose exec librarian ls /app/documents and your file should show up in that list.

Now the API needs to actually use what the Librarian indexed, instead of just forwarding raw questions to Ollama. Replace api/src/main.py again:

# api/src/main.py

import os
import numpy as np
import requests
from fastapi import FastAPI
from redis import Redis
from redis.commands.search.query import Query

app = FastAPI()

OLLAMA_HOST = os.environ.get("OLLAMA_HOST", "<http://host.docker.internal:11434>")
REDIS_HOST = os.environ.get("REDIS_HOST", "vector-db")
EMBED_MODEL = "nomic-embed-text"
CHAT_MODEL = "llama3.2"
INDEX_NAME = "mino_docs"

redis_client = Redis(host=REDIS_HOST, port=6379, decode_responses=False)

def embed_question(question: str) -> bytes:
    response = requests.post(
        f"{OLLAMA_HOST}/api/embed",
        json={"model": EMBED_MODEL, "input": question},
        timeout=30,
    )
    response.raise_for_status()
    vector = response.json()["embeddings"][0]
    return np.array(vector, dtype=np.float32).tobytes()

def search_documents(question: str, k: int = 3) -> list[str]:
    query_vector = embed_question(question)
    query = (
        Query(f"*=>[KNN {k} @embedding $vec AS score]")
        .sort_by("score")
        .return_fields("content", "source", "score")
        .dialect(2)
    )
    results = redis_client.ft(INDEX_NAME).search(query, query_params={"vec": query_vector})
    return [doc.content for doc in results.docs]

@app.get("/health")
def health():
    return {"status": "Mino's front door is open"}

@app.post("/ask")
def ask(question: str):
    relevant_chunks = search_documents(question)
    context = "\n\n".join(relevant_chunks) if relevant_chunks else "No matching documents found."

    prompt = f"""Answer the question using only the context below. If the context doesn't contain the answer, say so.

Context:
{context}

Question: {question}"""

    response = requests.post(
        f"{OLLAMA_HOST}/api/generate",
        json={"model": CHAT_MODEL, "prompt": prompt, "stream": False},
    )
    return {"answer": response.json().get("response"), "sources_used": len(relevant_chunks)}

The important addition is search_documents(). It embeds the incoming question with the same model used for indexing. And, yes, this has to be the same model on both sides, or the vectors live in different mathematical spaces and comparing them means nothing.

• Query(f"*=>[KNN {k} @embedding $vec AS score]") is Redis’ KNN (k-nearest-neighbors) search syntax and basically says: find the k chunks whose embedding field is closest to $vec, the question’s own embedding.
• .dialect(2) is required since Redis’ newer query syntax needs it explicitly set or the query fails outright.

The rest of /ask builds a prompt that hands Ollama only the relevant chunks instead of the whole document collection, which is the actual point of doing any of this.

Let’s bring the full system up together now that both services have real logic:

docker compose up --build

Verify with a PDF already indexed from the previous checkpoint, by asking it something the document would actually answer:

# RUN: curl -X POST "<http://localhost:8000/ask?question=your+question+here>"

You should get back an answer that references the actual content of your PDF, plus a sources_used count greater than 0.

When I add my resume in documents and ask “who is klea,” I get:

{"answer":"Klea Merkuri is a Full-Stack Engineer with 5 years of experience.","sources_used":1}

Tip ⛔
If sources_used comes back as 0, the most likely cause is that the API and Librarian used different embedding models at some point. This is when you stop everything, delete the Redis volume with docker compose down -v, and re-run ingestion from a clean index to rule that out.

Step 5: Keep Secrets Out of Plain Environment Variables

If Mino needed an authentication token for anything, such as a locked-down Ollama instance or a third-party embedding fallback, the instinct is to drop it into an environment: block as plain text.

Don’t 🙅‍♀️

Environment variables show up in docker inspect, logs, crash reports, and anywhere the process’s environment gets dumped. Don’t go this way for anything you wouldn’t want sitting in a log file: tokens, passwords, connection strings with credentials baked in.

Compose Secrets solves this by mounting the value as a file instead, at /run/secrets/<secret_name> inside the container, readable only by the service that explicitly asks for it.

Create the secret file inside the secrets/ folder you made at the very start:

touch secrets/ollama_token.txt

Put a placeholder value in it for now (or a real token if you have one):

placeholder-token-replace-me

Update docker-compose.yml to reference it:

services:
  api:
    build: ./api
    ports:
      - "8000:8000"
    networks:
      - internal
    depends_on:
      - vector-db
    environment:
      - OLLAMA_HOST=http://host.docker.internal:11434
    secrets:
      - ollama_token

secrets:
  ollama_token:
    file: ./secrets/ollama_token.txt

Reading it from inside the app takes a small helper function rather than pulling from os.environ:

def read_secret(name: str) -> str:
    with open(f"/run/secrets/{name}") as f:
        return f.read().strip()

Note: Add ./secrets/ to your .gitignore immediately, before you even create the folder. A secrets file that gets committed to git isn’t a secret anymore. Create that file now if you haven’t yet!

touch .gitignore

.env
secrets/
__pycache__/
*.pyc

Step 6: Make the Indexed Data Survive a Restart

None of this indexing work matters if it evaporates every time I stop the containers.

Named volumes give the vector database a piece of storage on my actual disk, separate from the container’s own filesystem, so the data survives a docker compose down and comes right back on docker compose up.

Update docker-compose.yml one more time to add a volumes line to vector-db, and declare the volume at the bottom of the file:

services:
  vector-db:
    image: redis/redis-stack:latest
    networks:
      - internal
    volumes:
      - vector_data:/data

volumes:
  vector_data:

The difference between a container and a volume in this context is that the container is disposable; the volume isn’t.

Tip: Reach for a named volume any time a container holds data you’d be upset to lose, such as a database, an upload folder, or anything that isn’t fully reproducible from your source code.

Bring the full system up one more time with everything now in place:

docker compose up --build

And verify by checking that the volume actually got created:

docker volume ls

You should see mino_vector_data in the list. To confirm persistence works, stop everything (docker compose down, not docker compose down -v since that flag deletes volumes), then bring it back up. The volume, and anything Redis had stored in it, should still be there.

I can delete and rebuild vector-db a hundred times, and as long as vector_data still exists, my indexed documents come back with it. Small thing, but it never stops feeling like a magic trick 🙌

At this point, here’s the complete mino/ folder:

mino/
├── docker-compose.yml
├── .gitignore
├── secrets/
│   └── ollama_token.txt
├── librarian/
│   ├── Dockerfile
│   ├── .dockerignore
│   ├── requirements.txt
│   └── src/
│       └── ingest.py
└── api/
    ├── Dockerfile
    ├── .dockerignore
    ├── requirements.txt
    └── src/
        └── main.py

OPTIONAL: Give Mino an Actual Interface

Everything up to this point works, but “works” currently means typing curl commands into a terminal. That’s fine for verifying the plumbing, but it’s not how anyone would actually want to use this day to day.

Dropping PDFs into a folder by hand and reading raw JSON back isn’t an interface, it’s a workaround. Even for a developer.

So, for better UX sake, let’s wire a small, plain HTML/CSS/JS frontend:

• A chat window that feels familiar
• An upload button that puts files where the Librarian is already watching
• A scrollable history, with a hard limit, so old conversations don’t quietly turn into unbounded storage

Give the API Something to Talk To

The frontend needs two things main.py doesn’t have yet:

1. a way to remember a conversation
2. a way to accept file uploads

Both go in Redis, next to the vectors, using a different key prefix so they never collide with document data.

Note: The snippets below modify the existing main.py. For the full updated file, see Mino’s repo.

We’ll start making changes to api/src/main.py by adding the new imports and initializing CORSMiddleware because the frontend and the API technically run on different origins from the browser’s perspective (even though they’re on the same machine).

Without it, the browser blocks the frontend’s API requests outright 💁‍♀️

import json
import shutil
import uuid
from fastapi import FastAPI, UploadFile
from fastapi.staticfiles import StaticFiles
from fastapi.middleware.cors import CORSMiddleware

app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],
    allow_methods=["*"],
    allow_headers=["*"],
)

Two Redis clients will now exist:

1. redis_client with decode_responses=True for chat history (plain text in, plain text out)
2. vector_client with it set to False for the embeddings, which need to stay as raw bytes.

Mixing these up is an easy mistake because if you decode a vector as a string, it silently corrupts!

redis_client = Redis(host=REDIS_HOST, port=6379, decode_responses=True)
vector_client = Redis(host=REDIS_HOST, port=6379, decode_responses=False)

Moving on, for the history we’ll introduce get_history() and append_to_history(). These use a Redis list (RPUSH/LRANGE), not a hash like the documents.

Tip: A list keeps message order, which a hash doesn’t guarantee.

ltrim(key, -MAX_HISTORY_MESSAGES, -1) sets the bound. After every message, Redis only keeps the most recent 20 entries and silently drops anything older. This runs on every single append, preventing the list from growing past that cap, thus avoiding a cleanup job, manual pruning, and risk of Redis quietly filling up with years of chat logs.

For the /ask endpoint, we now take a conversation_id alongside the question, pull prior messages into the prompt, so Mino has short-term memory of the conversation, and log both sides of the exchange to history after responding.

@app.post("/ask")
def ask(question: str, conversation_id: str):
    relevant_chunks = search_documents(question)
    context = "\n\n".join(relevant_chunks) if relevant_chunks else "No matching documents found."

    history = get_history(conversation_id)
    history_text = "\n".join(f"{m['role']}: {m['content']}" for m in history)

    prompt = f"""Answer the question using only the context below. If the context doesn't contain the answer, say so.

Conversation so far:
{history_text}

Context:
{context}

Question: {question}"""

    response = requests.post(
        f"{OLLAMA_HOST}/api/generate",
        json={"model": CHAT_MODEL, "prompt": prompt, "stream": False},
    )
    answer = response.json().get("response")

    append_to_history(conversation_id, "user", question)
    append_to_history(conversation_id, "assistant", answer)

    return {"answer": answer, "sources_used": len(relevant_chunks)}

Let’s also introduce two new endpoints (technically three if you account for one having a GET route as well):

@app.post("/conversations")
def new_conversation():
    conversation_id = str(uuid.uuid4())
    return {"conversation_id": conversation_id}

@app.get("/conversations/{conversation_id}")
def get_conversation(conversation_id: str):
    return {"messages": get_history(conversation_id)}

@app.post("/upload")
def upload_pdf(file: UploadFile):
    destination = os.path.join(UPLOAD_FOLDER, file.filename)
    with open(destination, "wb") as f:
        shutil.copyfileobj(file.file, f)
    return {"filename": file.filename, "status": "uploaded, indexing will pick it up shortly"}

• /conversations (POST) hands back a fresh UUID. This is what lets the frontend track “which conversation is this” without a login system.
• /upload accepts a file directly through FastAPI’s UploadFile and writes it straight into /app/documents, the same folder the Librarian is already watching from Step 4. No new ingestion code needed; dropping a file here is functionally identical to dragging it into the folder by hand.

And the last line, app.mount("/", StaticFiles(...)), is what actually serves the frontend. It has to come after every other route since FastAPI checks routes in the order they’re defined, and mounting static files first would swallow every request before your API routes ever saw them.

Tip: Update api/requirements.txt to add python-multipart, which FastAPI needs specifically to handle file uploads!

Build the Frontend Itself

Create the static folder and its files:

mkdir api/static
touch api/static/index.html
touch api/static/style.css
touch api/static/app.js

In api/static/index.html add:

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <title>Mino</title>
  <link rel="stylesheet" href="/style.css">
</head>
<body>
  <div id="app">
    <header>
      <h1>Mino</h1>
      <label class="upload-btn">
        Upload PDF
        <input type="file" id="fileInput" accept="application/pdf" multiple hidden>
      </label>
    </header>
    <div id="chat"></div>
    <form id="chatForm">
      <input type="text" id="questionInput" placeholder="Ask Mino something..." autocomplete="off">
      <button type="submit">Send</button>
    </form>
  </div>
  <script src="/app.js"></script>
</body>
</html>

For api/static/style.css add:

* { box-sizing: border-box; font-family: system-ui, sans-serif; }
body { margin: 0; background: #1a1a1a; color: #e8e8e8; }

#app { max-width: 700px; margin: 0 auto; height: 100vh; display: flex; flex-direction: column; }

header {
  display: flex;
  justify-content: space-between;
  align-items: center;
  padding: 1rem;
  border-bottom: 1px solid #333;
}

.upload-btn {
  background: #2d6cdf;
  padding: 0.5rem 1rem;
  border-radius: 6px;
  cursor: pointer;
  font-size: 0.9rem;
}

#chat {
  flex: 1;
  overflow-y: auto;
  padding: 1rem;
  display: flex;
  flex-direction: column;
  gap: 0.75rem;
}

.message {
  max-width: 75%;
  padding: 0.6rem 0.9rem;
  border-radius: 10px;
  line-height: 1.4;
}

.message.user { align-self: flex-end; background: #2d6cdf; }
.message.assistant { align-self: flex-start; background: #2a2a2a; }

#chatForm {
  display: flex;
  gap: 0.5rem;
  padding: 1rem;
  border-top: 1px solid #333;
}

#questionInput {
  flex: 1;
  padding: 0.6rem;
  border-radius: 6px;
  border: 1px solid #444;
  background: #111;
  color: #e8e8e8;
}

button {
  padding: 0.6rem 1.2rem;
  border: none;
  border-radius: 6px;
  background: #2d6cdf;
  color: white;
  cursor: pointer;
}

Lastly, api/static/app.js:

let conversationId = null;

async function initConversation() {
  const res = await fetch("/conversations", { method: "POST" });
  const data = await res.json();
  conversationId = data.conversation_id;
}

function addMessage(role, content) {
  const chat = document.getElementById("chat");
  const bubble = document.createElement("div");
  bubble.className = `message ${role}`;
  bubble.textContent = content;
  chat.appendChild(bubble);
  chat.scrollTop = chat.scrollHeight;
}

document.getElementById("chatForm").addEventListener("submit", async (e) => {
  e.preventDefault();
  const input = document.getElementById("questionInput");
  const question = input.value.trim();
  if (!question) return;

  addMessage("user", question);
  input.value = "";

  const params = new URLSearchParams({ question, conversation_id: conversationId });
  const res = await fetch(`/ask?${params}`, { method: "POST" });
  const data = await res.json();

  addMessage("assistant", data.answer);
});

document.getElementById("fileInput").addEventListener("change", async (e) => {
  const files = Array.from(e.target.files);
  if (!files.length) return;

  const formData = new FormData();
  for (const file of files) {
    formData.append("files", file);
  }

  const label = files.length === 1 ? files[0].name : `${files.length} PDFs`;
  addMessage("assistant", `Uploading ${label}...`);

  const res = await fetch("/upload", { method: "POST", body: formData });
  const data = await res.json();

  if (!res.ok) {
    addMessage("assistant", data.detail || "Upload failed.");
    e.target.value = "";
    return;
  }

  const names = data.filenames.join(", ");
  addMessage("assistant", `${names} uploaded — ${data.status}`);
  e.target.value = "";
});

initConversation();

Nothing crazy here—on purpose. No framework, no build step, just three files a browser can run as-is.

• initConversation() fires once on page load and gets a conversation_id from the API before anything else happens, since every question and upload after that needs it.
• addMessage() is the one function both the chat form and the upload handler share, which is why uploads show up as system-style bubbles in the same chat window instead of a separate notification area.

Tip 🪲
If you open this and the chat window does nothing when you hit Send, open your browser’s dev tools console first. A blocked CORS request or a JavaScript typo will show up there immediately and it’s the fastest way to tell “the frontend is broken” from “the API is broken.”

We’re not quite done yet 👇

Wire the Frontend Into the Container and Compose File

The static files need to ship inside the API image. Update api/Dockerfile with the only change being the new COPY static/ ./static/ line.

It puts the frontend files where StaticFiles(directory="/app/static", ...) in main.py expects to find them.

The only change in docker-compose.yml is to add volumes ./documents:/app/documents to mount the shared ./documents volume on the api service so uploads land where the librarian watches.

Nothing else changes since the api service already builds from ./api, exposes port 8000, and has everything it needs. This is the actual payoff of Compose owning the whole system from the start!

Adding a full frontend didn’t mean touching much of the orchestration, just the one container that was already responsible for serving requests.

To check, bring the full stack up one more time, then:

1. Open http://localhost:8000 in a browser — you should see Mino’s chat interface, not raw JSON.
2. Send a question about a PDF you’ve already indexed, and you should see both your message and Mino’s answer appear as chat bubbles.
3. Upload a new PDF through the button, wait a few seconds, then ask about its contents in the same window. It should show up in the response without having to touch the terminal.

Tip: To take this to a next level, I’d certainly add an upload progress indicator for the new PDFs pronto. For a refined version, check the repo, clone, and launch it. Then compare against your first scaffolded version.

To verify the history cap specifically, run 25 questions through the same conversation, then check what’s actually stored:

docker compose exec vector-db redis-cli LLEN history:<paste-your-conversation_id-here>

That should return 20, not 50, confirming ltrim is doing its job instead of quietly letting the list grow forever.

Tip 💬
If you refresh the browser page and lose your chat window, that’s expected: conversationId currently only lives in the page’s JavaScript memory. The history itself is safe in Redis under that same ID; it’s just not reloaded automatically. It’s a reasonable next extension, but outside what this post walks through. However, feel free to check the refined source files in Mino’s repo for how this is addressed.

Real-World Use: Where This Setup Actually Pays Off

At this point, the system runs, which answers the “does it work” question. What it doesn’t answer yet is whether containerizing all of this was worth the extra setup.

Was it worth it? Well, that’s something that shows up less in the initial build than in what happens after, when something breaks, or the code needs to change.

These are the three moments where I directly felt the most difference:

Debugging With Gordon Instead Of Guessing

Gordon is Docker’s AI agent, built into Docker Desktop and the CLI, and it’s specifically useful here because it already has context a general chatbot doesn’t. It knows your running containers, images, Compose files, and logs.

Note: Gordon is included free with any Docker account, though the free tier caps how much you can use it before it asks you to upgrade. For occasional debugging like this, I never hit that ceiling.

Run docker ai from your project directory, and it reads that context automatically. When my api container couldn’t reach Ollama the first time, I asked it directly:

docker ai "my api container can't reach Ollama on the host, what's wrong"

It checked my Compose file, noticed there was no host.docker.internal mapping, and proposed the extra_hosts fix, showing me the exact change before making it 👀

Gordon proposes every action and waits for your approval before it touches anything. It’s not autonomously editing your files; it’s a second set of eyes with more context than a search engine has, because it can actually see the system I built instead of guessing at it from a description.

Tip 📍
Reach for Gordon over a general coding assistant specifically when the problem is Docker-shaped. Things like container-to-container or container-to-host communication, image size, and startup order, since that’s the exact context it has that a general assistant doesn’t.

Editing Code Without Rebuilding the World

Every time I tweaked my PDF-parsing logic in the Librarian service, I had to manually stop the container, rebuild the image, and start it again.

Slow, and it breaks the darn flow.

Compose Watch fixes this by syncing file changes into a running container automatically, without a full rebuild, using a develop.watch block:

services:
  librarian:
    build: ./librarian
    develop:
      watch:
        - path: ./librarian/src
          action: sync
          target: /app/src
        - path: ./librarian/requirements.txt
          action: rebuild

Run it with docker compose watch. Edits to my parsing code inside ./librarian/src get copied straight into the running container without a rebuild or restart.

Changing requirements.txt triggers a full rebuild instead, since a new dependency can’t just be “synced” into a running process.

Use sync for source code you’re actively iterating on, and rebuild for anything that changes what the image itself needs to contain.

Note: sync only works for services built with the build attribute, not ones pulling a pre-built image. That’s the Librarian and the API—not vector-db, since Redis isn’t something I’m editing source code for.

A Door To More, Without More Custom Code

I’d like to quickly mention this despite being outside the scope of today’s build: Docker’s MCP Toolkit is a catalog of containerized MCP (Model Context Protocol) servers that provide pre-packaged, isolated tools an agent can call, like calendar access or local file search, each shipped and versioned as its own image.

If I eventually wanted Mino to do more than answer questions about PDFs, this is the mechanism I’d reach for instead of writing custom integrations by hand.

Definitely not part of this project, but it’s the natural next step, and it only exists as an option because everything here is already containerized.

Pro Tips & Gotchas Before You Trust It With Real Documents

I can’t help it, I’ve got to mention these little tips:

1. Don’t Let Services Start In The Wrong Order

Compose starts containers in the order you define with depends_on, but “started” doesn’t mean “ready.”

Redis’ container can report as running before it’s actually finished initializing, and if my API tries to connect a half-second too early, it fails.

A HEALTHCHECK runs a command inside the container on a schedule, and depends_on can wait for that check to pass, not just for the container to start, before starting a dependent service:

services:
  vector-db:
    image: redis/redis-stack:latest
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 5s
      timeout: 3s
      retries: 5
      start_period: 10s
    networks:
      - internal

  api:
    build: ./api
    depends_on:
      vector-db:
        condition: service_healthy
    networks:
      - internal

The API now waits for vector-db to answer PONG to redis-cli ping, not just for the container to exist.

Tip: start_period gives Redis a 10-second grace window before failed checks start counting against it, since the first few seconds of any database container are usually just it warming up.

Reach for a healthcheck any time one service’s readiness, not just its existence, matters to another.

2. Don’t Let Ingestion Eat Your Whole Machine

Document ingestion (parsing PDFs and generating embeddings) is exactly the kind of process that can quietly eat every CPU core and all your RAM if you let it, especially on a personal machine that’s also trying to run everything else you have open 😬

services:
  librarian:
    build: ./librarian
    deploy:
      resources:
        limits:
          cpus: "1.5"
          memory: 1G

This caps the Librarian at 1.5 CPU cores and 1GB of memory. If it tries to exceed the memory limit, Docker kills and restarts the container rather than letting it take the rest of your system down with it.

If you index a truly massive PDF, you might see that container restart. Better than your laptop fan taking off, in my opinion, but bump the limit if you hit it!

3. Scan Before You Trust It

Last step before I trust this thing with actual documents: scanning the images for known vulnerabilities.

Docker Scout is built into Docker Desktop and the CLI, and it checks your image’s packages against a live vulnerability database (CVE). It does need a Docker account, so run docker login first if you haven’t already, or the scan will prompt you to.

docker scout cves mino-librarian:latest

That returns every known CVE in the image, ranked by severity, along with which package introduced it.

If I want to gate on it by failing if anything critical shows up, there’s a flag for that too:

docker scout cves --exit-code --only-severity critical mino-librarian:latest

It’s a surprisingly low-effort last check that doesn’t need an enterprise plan for local scans like these (the free Docker Personal tier covers it), so there’s no excuse for skipping it.

It’s a Wrap

What I actually walked away with here isn’t a working RAG pipeline for its own sake but an answer to the question I opened with: Why does Docker matter once you’re the one building the thing, not just running someone else’s container?

The answer, for me, is that Docker stopped being a way to run one tool and became the layer that let three independent pieces trust each other:

• A private network they share but nothing outside can see
• Storage that outlives any single container
• Secrets that never show up in a log
• A startup order that’s enforced instead of assumed

None of that comes free if you build the same system as loose local processes. It comes standard the moment you describe it in a Compose file.

And there’s nothing better than a local AI agent to put all this in perspective (since, you know, it uses basically all different services).

If you build a version of this yourself, the one thing I’d check first is your host.docker.internal connection to Ollama. That was the single spot where I stopped, staring at a connection refused error and realizing the isolation I’d spent this entire post building was, for a beat, working exactly as intended against me too 🤦‍♀️

Docker doesn’t know the difference between “a stranger trying to reach my database” and “me, trying to reach my own Ollama install”—it simply enforces the boundary. I had to be the one to decide how narrow to make that deliberate exception.

Give your own Mino a try, play with Docker, and drop your experience in the comments!

I’ll catch ya next time.

Bye.