By Rolf Versluis ยท Published [DATE] on Uncommon Knowledge in Business and Life
Open-source software is almost always 95% ready to go. You download it. You run it. It mostly works. Then you need to install it, configure it, integrate it with whatever else you have running, lock it down, and tune it for your situation. That last 5% is where most projects die. Not because the work is hard. Because the work is fiddly, undocumented, and never quite the same twice.
I have been running into this for 20 years. The most popular web server in the world is nginx. You can go download it and install it in a few minutes. But it still requires setup and integration, and things to optimize it and make it secure. Those are the small decisions nobody writes down.
That is the work I do, and the work I want an agent doing with me.
The farm camera example
I bought an IP security camera for the farm and mounted it on the building. I wanted it to do one thing: when somebody drives onto the property, send me a Telegram message with a picture, and tell me whether it is someone I know or someone I do not.
That sounds simple. It is not simple. The hard part is the "someone I know" piece. A trail camera with a SIM card does the first half. But "is this person known?" requires looking at the picture, comparing it to the people I have seen before, and making a judgment. That is what a person does. That is not what a sensor does.
The agent and I looked at the open-source options first, but none of them did what I wanted. So we built the system from scratch. The project is open source at github.com/blockops1/ai_camera_monitor. MIT-licensed, runs on a Mac Studio or a Linux box, the README has the architecture. Took a few months. Not because any single piece was hard. Because the fiddly bits add up.
What the pipeline actually does
Four stages, each one filtering out the noise before the next stage has to deal with it.
A Reolink camera detects motion and fires an HTTP webhook at a small Flask listener running on port 8090. The listener peels frames from the camera's RTSP ring buffer and routes by camera and event type.
The first stage is a motion gate running a small YOLOv8n convolutional neural network on each frame's motion crop. If there is no real object in the frame, the alert is dropped here. This is where roughly 99% of false positives die. The expensive vision model is never called for events that do not matter.
The second stage is a vision model, a local Qwen3-VL running on port 8093. It receives a three-image payload (two streak crops plus the pairwise differential) and returns structured output: make, model, color, body style, and confidence. This is what turns "vehicle" into "black Ford F-150 pickup."
The third stage is the matcher. It scores the vision signature against data/vehicles/known_vehicles.json, an enrolled store of vehicles I have shown the system before. This is the part that answers the actual question: known or unknown.
The fourth stage is a small text model running locally on port 8081. It writes the human-readable alert body. This is what makes the alert read like a person wrote it instead of "VEHICLE DETECTED."
Then the alert goes to Telegram through the bot API. Two seconds from motion event to phone.
How I built it
Architecture came out of a back-and-forth between me and my Hermes agents. I knew what I wanted the system to do. Jill, a Hermes instance running on a local model, proposed how to break the problem into stages, picked the open-source pieces for the motion gate and the vision model, wrote the integration code, and debugged the connections between stages. We worked on it for a few months. I made the design calls. The agent did most of the implementation.
The first version was bad. The motion gate let tires through. The vision model mistook shadows for trucks. The matcher said everything was unknown because the enrolled store was empty.
We kept a log of the wrong cases. Each one pointed at a small fix. The YOLO confidence threshold needed adjusting. The differential crop had to hug the motion tighter. The enrolled store needed the truck added. Each fix taught the system something the next alert would benefit from.
The tuning was not a one-time event. After a few months of operating, the false positives were gone. The system figured out who was at the property, took a picture, and sent it to me. No tire alerts. No shadow alerts. No branch-blowing-in-the-wind alerts.
Why this is the actual value
Open-source software was 95% there. The 5% that was missing was the part I actually needed: a system that knew the difference between a known vehicle and a stranger, in my specific setting, with no false positives.
That 5% is also the part no vendor will sell you. It is too specific to your property, your cameras, your lighting, your definition of "known." A vendor sells a generic product. I built the specific thing.
This is the pattern I keep running into. The hard 5% is always the part where the work becomes yours. It is the part where the open-source README stops being useful and the project becomes whatever it is in your specific situation.
Most people quit at 95%. I do not.
What this looks like as a service
When someone asks me to implement a Hermes agent for a piece of work, this is the model. Look at what already exists. Pick the best option. Install and configure it. Then build the 5% that turns the open-source thing into your thing.
For some projects, the 5% is configuration. For some, it is integration with the rest of your stack. For some, it is writing a custom classifier or a custom workflow. The shape of the work changes. The pattern is the same.
I am not pitching "AI builds your business in five minutes." I am pitching the other way around: I sit down with you, learn the business, build the thing that fits it, and keep it running.
If you have an open-source tool that is 95% there and the 5% is the part keeping you from using it, that is the kind of problem I work on. If you want to see how a system like this gets built, the camera monitor is at github.com/blockops1/ai_camera_monitor, MIT-licensed. My address is localhermesagent.com.