For weeks now, my YouTube feed and the weekly articles I read to stay current have been circling the same thing: developers setting up OpenClaw or Hermes Agent, and turning them loose to run their lives. Check the inbox. Watch the pipelines. Message on Telegram when something needs attention.
It’s not one viral video. It’s a pattern, showing up over and over, from enough different creators that I started assuming I was missing something everyone else already understood.
So I sat down and made an actual list. Not “would this be cool,” but specific tasks I’d hand to an agent if I set one up, things I’d want automated, checked on, or generated while I wasn’t watching.
What would I actually use this for, if I had one running right now?
I wanted to see the thing on paper before deciding whether it was worth building. So I looked at what it would actually cost to run OpenClaw or Hermes the way the setup videos show it, local model included, since that’s what most of those videos imply is doing the work.
Once I priced that out, I couldn’t justify a single item on that list 😑
I’m not saying the hype is fake, nor am I pretending I’m new to any of this.
I’ve built local agent tooling before. Mino, my local RAG setup, runs on my own machine right now, doing exactly what I built it to do (and even evolved into something much bigger on my HOA Connect web app that I’ve been talking about on LinkedIn).
So this post isn’t “I don’t get why people are doing this.” It’s more like I tried to find my own reason to do this too, and the math kept refusing to add up. I want to walk through where it broke down, because most of the confusion around this whole trend comes from three separate decisions that are flattened into one exciting reveal.
Why the AI Agent Hype Doesn’t Add Up: Framework vs. Local Model, Explained
Local models that get close to frontier quality, the kind of output you’d get from Claude or GPT-4, need hardware most people don’t own.
A Mac Mini isn’t it. You’re looking at Mac Studio territory, and that’s before you’ve factored in what happens once you actually try to run something on top of the model, not just chat with it.
Can I Use VPS to Host My Agent?
VPS hosting looks like the workaround. Rent someone else’s server, skip the hardware purchase, run the thing remotely (or so the pitch goes).
Except GPU access on a VPS isn’t the same product as the general-purpose VPS most people already know how to spin up 💁♀️
Mainstream hosts that popularized cheap VPS hosting are still catching up on the GPU side. Even where GPU tiers now exist, they sit in a separate, pricier product line from the $4-a-month box most setup videos assume.
The mainstream, cheap, everyone’s-heard-of-it hosting tier wasn’t really built with GPU workloads in mind.
And once you’re paying for a rented GPU server, the privacy pitch that supposedly justified all of this in the first place gets murky. Your data is still leaving your machine; it’s just landing on a different company’s server instead of Anthropic’s or OpenAI’s.
So what did you actually gain?
What “Run It Locally” Actually Means
None of that VPS and GPU talk is me overthinking it. The math doesn’t work the way “just run it locally” implies, and I spent enough time confirming that to trust it.
The confusion mostly clears up once you separate two things that get talked about like they’re the same decision 👇
Running an agent framework and running a local model are not the same choice.
One of those is cheap and light enough to run almost anywhere. The other one is the thing that demands the hardware and the cost.
Most of what I was watching on YouTube blurred the two together into a single “here’s my agent” reveal. Untangling them is what actually resolved most of my confusion.
Related: I Tried the ‘Code for Free with Local AI’ Setup. Here’s What Actually Happened.
OpenClaw vs. Hermes Agent: They’re Not Actually Competing on the Same Thing
OpenClaw and Hermes Agent are the two open-source frameworks driving most of this discourse right now, and they’re not really competing on the same thing, even though the videos tend to present them that way.
- OpenClaw, built by Peter Steinberger, leans into breadth: messaging platform integrations, a skill marketplace to pull from instead of building everything yourself, and the most accessible setup for someone who isn’t deep into the Python ecosystem.
- Hermes Agent, from Nous Research, leans the other direction: a self-improving memory loop with more direct control over the underlying architecture, aimed at developers who want to shape how the agent reasons over time instead of just plugging it into their existing tools.
Neither of these is a model.
Think of them like a phone’s operating system versus the apps running on it. OpenClaw and Hermes are the operating system, the piece that decides what to do, when to check in, and which tool to call. The model is the app doing the actual thinking.
You point it at whatever model you want to answer the requests it routes, be it Claude, GPT, DeepSeek, or a model you’re running yourself through Ollama.
The framework and the model are two separate budget decisions, and almost none of the setup videos I watched said which one they were paying for.
Tip 🤨
For a setup video that shows someone running OpenClaw or Hermes and never mentions which model it’s calling, they’re very likely pointing it at a cloud API like Claude or GPT, not running anything locally. That single detail changes the entire cost conversation, and it’s the detail that gets skipped most often, probably because “I hooked this up to Claude” is a less exciting reveal than “I built my own AI.”
Local LLM Hardware Requirements: Why a Mac Mini Isn’t Enough
Mino, my local RAG agent, retrieves and reasons over documents I’ve stored. I built and run it on my MacBook Pro (24GB of RAM, M4 Pro chip). Even at that spec, and even with a scope as contained as Mino’s, I ran into disk space problems once I moved past a simple prototype and started layering the pieces that make an agent actually useful.
Local models take up space on their own. Then you add whatever you’re using for parsing or extraction, and then space for persistent memory through local embeddings (numerical representations of text that let the agent “remember” and search past context instead of starting fresh every time).
The storage math climbs faster than the “just spin up Ollama” tutorials make it sound.
I also tried the same setup on an old MacBook Air (8GB of RAM) I was about to sell, mostly out of curiosity about how low I could go. Sadly, it didn’t just struggle; it crashed the entire machine, and capped what I could even attempt for parsing, extraction, or generating embeddings for memory before it gave out.
Hey! Though it might be tempting to say, that’s not a Mino-specific problem. That’s what happens when you try to run any local model workload on hardware that was never built for it.
Local LLM RAM Requirements by Model Size: The Actual Numbers
Time for the maths! You’ve probably heard discussions or whispers of “quantization,” and if you don’t know what it means, you need to.
Quantization is a way of shrinking a model down so it takes up less memory by storing its internal numbers with less precision. Q4 is one common level of that shrinking.
Picture a photo saved at full resolution versus compressed for the web. The compressed version is smaller and loads faster, even though you lose a little detail, but it still looks like the same photo. That’s what Q4 quantization does to a model.
A local model needs roughly 0.6GB of RAM per billion parameters at that Q4 level.
A 7B or 8B model, meaning 7 or 8 billion parameters, needs roughly 5 to 6GB of RAM, comfortable on most modern laptops.
Note: The “B” stands for billion parameters, roughly the model’s size and complexity. More parameters generally means a smarter model, but also a bigger one.
Getting into the 14B to 32B range needs roughly 11 to 25GB, which is where quality starts approaching something you’d actually want to use daily instead of just testing.
That range is also where a base Mac Mini starts running out of headroom, especially once you add the other things an agent needs alongside the model itself.
70B-class models, the ones that start to meaningfully close the gap with frontier quality, need around 42GB just to load into memory, before you’ve asked it a single question 😳
70B Model Speed and Cost: Why the DIY Route Doesn’t Save You
Even once you clear that memory bar, speed doesn’t follow automatically.
Let’s put “tokens per second” in perspective. A token is roughly three-quarters of a word, so reading speed for most people lands somewhere around 5 tokens per second.
Community benchmarks on 70B-class models land anywhere from roughly 10 to 28 tokens per second depending on the chip, which is still far from the near-instant feel of a cloud API.
One developer stacked eight Mac Minis into a DIY cluster to get DeepSeek-V3 running locally: roughly $20,000 in hardware, for 5 tokens per second.
Commercial setups hit the same benchmark for $200,000 or more, so by that comparison the DIY cluster is objectively the better deal. But it’s still proof that the cost doesn’t disappear when you go the scrappy route. It simply changes shape, and moves onto your kitchen counter.
Note 😓
This is where my original confusion was warranted, not something I was overthinking. The “frontier-adjacent local model on a normal laptop” pitch doesn’t hold up under the actual hardware math. There’s a real gap between the marketing language (“run it locally!”) and what running it locally actually costs to do well, and I hit it on two different machines before I ever opened a setup guide for OpenClaw or Hermes.
VPS GPU Hosting for AI Agents: The Catch Most Setup Guides Skip
Most people deploying OpenClaw or Hermes to a VPS aren’t running local inference at all. They’re pointing the framework at a cloud model and need somewhere for the framework itself to live around the clock.
A cloud-API agent runs comfortably on a $4 to $6 a month VPS with no GPU anywhere in the picture, because the actual “thinking” is happening on Anthropic’s or OpenAI’s servers, not on the box you’re renting.
Local inference is the thing that actually demands GPU-specific hosting.
Even providers like Hostinger or Bluehost have started adding GPU VPS tiers, but that’s a separate product line from their standard VPS plans, priced and provisioned differently.
Whether you’re going through a mainstream host’s GPU tier or a GPU-specialized platform like Vast.ai, you’re not getting there through the cheap general-purpose VPS setup guides usually mean.
A dedicated GPU VPS, built specifically for machine learning workloads, starts at a higher price point than the general-purpose VPS everyone already knows how to spin up. That’s the split I kept missing when I first started looking into this, resulting in a few lost hairs 😔
“Get a VPS” sounds like one decision, but it’s actually two very different products depending on whether a GPU is involved.
Why Most Agent VPS Setups Are Actually About Uptime, Not Privacy
That split also answers the privacy question I kept getting stuck on back when I was first working through this. If someone rents a GPU VPS to run local inference, their data isn’t private in the way “self-hosted” language implies.
Why? Because it’s now sitting on someone else’s server, even if it’s someone other than Anthropic or OpenAI.
The privacy argument is really only valid when the hardware is physically sitting in your own home or office, plugged into your own network, under your own control.
Rent a GPU box from a cloud provider, and you’ve swapped one third party for another. You haven’t removed the third party from the equation 🤦♀️
Once I looked past the privacy framing, what’s actually driving most of the VPS adoption is uptime, not privacy or model quality. These frameworks are built to run background loops.
OpenClaw, for example, checks in roughly every 30 minutes to see if there’s anything it should be doing, following up on reminders or reviewing an inbox on its own without any prompting.
A laptop that goes to sleep, or closes because you’re done for the day, breaks that loop completely. A cheap CPU VPS solves that specific problem without needing a GPU anywhere near it, which is a different justification than “smarter local model.”
Conflating the two, uptime versus model quality, was a big part of the confusion.
The AI Agent Cost-Benefit Test: What I Actually Needed vs. What I Thought I Needed
Once I’d separated the costs (framework vs. model vs. hosting) in my head, I went back to the list I made and sorted each item instead of lumping them all into one vague “would be nice” pile.
1. Recurring Content Check
The first item I wanted to set up is a recurring content check. Basically, something that would surface a set number of news topics or discussions each week that are worth researching for THT or just worth consuming for myself to stay current.
That’s a background, low-stakes, scheduled task. It doesn’t need frontier-quality local inference to do its job well, and nothing about it touches sensitive data that I’d want kept off a cloud model’s servers.
A cheap CPU VPS running the framework against a cloud API, no GPU anywhere in the setup, would cover this completely. I don’t need a smarter model here; I need something that’s still checking on a schedule when my laptop is closed (hint: uptime).
2. Project Ideation
The second item was all about project ideation. This setup would help me think through potential apps or projects worth building, surfacing ideas or angles I hadn’t considered.
Same story as the first. Nothing about that task needs to stay off a cloud model’s servers, and nothing about it benefits meaningfully from local inference over a strong cloud model either.
It needs a framework that can hold context across sessions and prompt me periodically, not a GPU sitting somewhere generating tokens slower than Claude or GPT already would.
3. Opportunity Tracker
Now, the third item was more ambitious. I was considering a comprehensive, intentional job list, something that could track opportunities, cross-reference them against my own criteria, and act as a real ongoing filter rather than a static spreadsheet I forget to update.
You probably guessed it, but this one sits closer to the edge than the first two.
It’s still not obviously a local-inference problem on its face, but it’s the first item on the list where I could see an argument forming for something more capable than a lightweight cloud-API loop, if the volume or complexity of what it needed to track ever grew past what a standard API call handles comfortably.
Related: This Is The Free Browser Extension Your Job Search Needs
What My List Actually Proved About Local vs. Cloud AI Agents
What I didn’t find, across the whole list, was a task that actually needed a frontier-quality model running entirely on hardware I own, with nothing leaving my machine (which is the thing the YouTube thumbnails are implying).
Was there anything on that list I’d trust enough to spend money making local?
Every single item wanted one of two things:
- uptime, which is cheap and doesn’t need a GPU
- a smarter model, which is solved by pointing at a cloud API rather than hosting one myself at real cost.
Nothing on my list justified a Mac Studio or a GPU-tier VPS bill.
Note 👀
I’m not going into the costs associated with these cloud API models right now, because that’s of course something else to consider. My goal here is to identify examples of the sort of agentic workflows that I would actually set up to see the extent to which they would need local inference, and get a feel of the level of compute.
Is Self-Hosting an AI Agent Worth It? Where the Hype Isn’t Wrong
I don’t think the online discourse is lying, and I don’t want this post to read like I think everyone setting one of these up got fooled. The framework-plus-cloud-API-plus-cheap-VPS combo is affordable and useful for the exact kind of background automation I put on my list.
If that’s the version someone’s actually running, the enthusiasm makes sense.
What’s misleading is the way it gets marketed, showing off “an agent” without saying which of the three cost decisions is actually doing the work behind it. That gap is what makes it look like everyone’s cracked frontier-quality local AI on a personal budget, when most of them haven’t, because most of them didn’t need to.
Note: The appeal of running everything yourself tends to wear off once the actual maintenance and cost show up in practice instead of in theory.
People spin up their own servers, their own nodes, their own infrastructure, excited about the control it gives them, and most of them eventually fall back toward whatever’s cheapest and least annoying to keep running long-term.
Self-hosting has a novelty period, and the bill and the upkeep are usually what end it.
I’m not against setting up OpenClaw or Hermes. I’m against buying hardware or paying for a GPU-tier VPS to chase what a thumbnail implied when the actual task on my list doesn’t need it.
If something on that list ever grows into a task that needs local, frontier-adjacent inference badly enough that the cost stops being the deciding factor, I’ll revisit this. Right now, none of them do, and I want to be upfront about it because it’s easy to fall prey to the viral agent excitement.
It’s a Wrap
The real decision was never “agent or no agent.” It’s three smaller decisions:
- Which framework
- Which model
- Where the model lives
Setup videos collapse all three into one exciting reveal because that’s a better thumbnail than “I’m pointing this at Claude and hosting it on a $5 VPS.”
If you’ve made your own list, I’d be curious what’s actually on it. Does anything on it actually need local, frontier-adjacent inference, or is it really an uptime problem wearing a bigger price tag?
Thanks for sticking with me through the math on this one.
I’ll see ya on the next ✌️
