All Posts
Sentinel: Solving the A8's 'Storage Sink' Problem
April 2026 — on building a survival loop for a car that spends its summers in a greenhouse, and why ‘propping it up’ is an engineering challenge
JAPIE: How an Incoherent Mess Became a Self-Improving AI Orchestrator
April 2026 — on shipping fast, discovering chaos, systematically fixing it, and then building a system that improves itself
There’s a particular kind of technical debt that emerges when you ask an LLM to design an AI orchestrator without a clear spec. The result lands in your codebase looking plausible: proper error handling, metrics collection, a learning loop. But when you actually try to run it against real workflows, you discover the wires are loose, the assumptions are broken, and half the system assumes the other half already exists.
The NPU Nobody Talks To: Mapping the CIX P1's Hidden AI Engine
The Orange Pi 6 Plus sits in the rack, humming along with the quiet confidence of a board that knows it is being underutilised. Inside the CIX P1 SoC is a dedicated Neural Processing Unit (NPU)—a piece of silicon specifically etched to crunch tensors and accelerate inference. The hardware is real, the chip is powered, and the transistors are awake. But in the world of the Linux kernel, it is a ghost. There is no driver to wake it up, no sysfs entry to query it, and no user-space API to feed it data. It is a high-performance engine idling in a vacuum; the chip is awake, but nobody’s home.
700ms to 2ms: What a Cluster Fire Taught Me About Embedding
700ms. That was the number that haunted my Kubernetes cluster, slowly burning it to the ground. Every alert the cluster generated, every log line it processed for AI-driven feedback, triggered an embedding operation. Each of these embeddings, we thought, took 700ms, saturating a CPU core, which in turn triggered more alerts, creating a truly spectacular, self-immolating feedback loop. The load average climbed to 153. It was a cluster fire, quite literally. Then, in the chaotic aftermath of patching the inferno and moving that “expensive” embedding workload to a humble 15-watt ARM board, something remarkable emerged: the warm latency was a mere 2ms. Even a cold start, including model loading, clocked in at around 100ms. The sobering discovery? The 700ms was never about the embedding operation itself. It was the embedding struggling under full CPU saturation, choked for headroom. On a quiet, dedicated machine with a warm model, the exact same task takes 2 milliseconds.
The AI That Monitored Your Cluster Just Brought It Down
April 2026 — on the sentinel that decided to burn the house down
“Why can’t I see the new photos?”
That’s how the outage started. Not with a PagerDuty alert or a Grafana dashboard turning red, but with a casual question from my wife. I was already deep in the weeds debugging a glitch in Nextcloud Talk, but as I tried to refresh my own dashboard, the latency didn’t just spike—it vanished. Immich was gone. Mail was gone. The search index was a black hole.
€200 Claude.ai bill in one week — so I built a cheaper alternative
April 2026 — one week of intensive AI-assisted work, one surprising bill, and one decision to do something about it
The Claude.ai usage screen showed €169.51 spent in a single week. That number included €23 for a Pro subscription and four separate top-ups of €50 each in “extra usage.” One hundred and sixty-nine euros. In seven days. On a chat interface.