All Posts

A thumbnail image

Trivy Scanned My Compromised Server This Morning. It Found Two OpenSSL Bugs.

October 2026: the scanner was healthy, up to date, and looking straight at it.

Last weekend I finally got around to a project that sat on the backburner for months: proper vulnerability scanning for the cluster. Trivy Operator, a built-in server so scans avoid re-downloading the vulnerability database every run, a UI, alert rules, and an accepted-risk register with review dates. The works. I felt pretty good about it. The cluster was finally observed.

A thumbnail image

Home DNS failover, taken too far on purpose

September 2026 — you would rather read this than do it. Good. I did it so you don’t have to.

Every home network has one thing that, when it breaks, makes everything look broken: DNS. The router still routes, the fibre still carries bits, the NAS still serves — and nobody in the house can open a page. If you run an ad-blocking resolver (AdGuard Home, in my case, in Kubernetes), you have also made DNS depend on the most complicated thing you own.

A thumbnail image

Meten is Weten: How Installing Plausible on My Hugo Blog Led to a Three-Node BGP ECMP Varnish DaemonSet

It started with a single, innocent question: does anyone actually read this?

I’d been running this Hugo blog for a while, writing posts about the homelab, the cluster, the occasionally catastrophic self-inflicted incidents. At some point the thought surfaced that it would be nice to know whether the words were reaching anyone beyond me and the search indexer. So I installed Plausible — privacy-respecting analytics, no cookies, one config line in Hugo — and moved on.

A thumbnail image

The NPU Nobody Talks To: Mapping the CIX P1's Hidden AI Engine

The Orange Pi 6 Plus sits in the rack, humming along with the quiet confidence of a board that knows it is being underutilised. Inside the CIX P1 SoC is a dedicated Neural Processing Unit (NPU)—a piece of silicon specifically etched to crunch tensors and accelerate inference. The hardware is real, the chip is powered, and the transistors are awake. But in the world of the Linux kernel, it is a ghost. There is no driver to wake it up, no sysfs entry to query it, and no user-space API to feed it data. It is a high-performance engine idling in a vacuum; the chip is awake, but nobody’s home.

A thumbnail image

700ms to 2ms: What a Cluster Fire Taught Me About Embedding

700ms. That was the number that haunted my Kubernetes cluster, slowly burning it to the ground. Every alert the cluster generated, every log line it processed for AI-driven feedback, triggered an embedding operation. Each of these embeddings, we thought, took 700ms, saturating a CPU core, which in turn triggered more alerts, creating a truly spectacular, self-immolating feedback loop. The load average climbed to 153. It was a cluster fire, quite literally. Then, in the chaotic aftermath of patching the inferno and moving that “expensive” embedding workload to a humble 15-watt ARM board, something remarkable emerged: the warm latency was a mere 2ms. Even a cold start, including model loading, clocked in at around 100ms. The sobering discovery? The 700ms was never about the embedding operation itself. It was the embedding struggling under full CPU saturation, choked for headroom. On a quiet, dedicated machine with a warm model, the exact same task takes 2 milliseconds.

A thumbnail image

The AI That Monitored Your Cluster Just Brought It Down

April 2026 — on the sentinel that decided to burn the house down

“Why can’t I see the new photos?”

That’s how the outage started. Not with a PagerDuty alert or a Grafana dashboard turning red, but with a casual question from my wife. I was already deep in the weeds debugging a glitch in Nextcloud Talk, but as I tried to refresh my own dashboard, the latency didn’t just spike—it vanished. Immich was gone. Mail was gone. The search index was a black hole.