All Posts
The NPU Nobody Talks To: Mapping the CIX P1's Hidden AI Engine
The Orange Pi 6 Plus sits in the rack, humming along with the quiet confidence of a board that knows it is being underutilised. Inside the CIX P1 SoC is a dedicated Neural Processing Unit (NPU)—a piece of silicon specifically etched to crunch tensors and accelerate inference. The hardware is real, the chip is powered, and the transistors are awake. But in the world of the Linux kernel, it is a ghost. There is no driver to wake it up, no sysfs entry to query it, and no user-space API to feed it data. It is a high-performance engine idling in a vacuum; the chip is awake, but nobody’s home.
700ms to 2ms: What a Cluster Fire Taught Me About Embedding
700ms. That was the number that haunted my Kubernetes cluster, slowly burning it to the ground. Every alert the cluster generated, every log line it processed for AI-driven feedback, triggered an embedding operation. Each of these embeddings, we thought, took 700ms, saturating a CPU core, which in turn triggered more alerts, creating a truly spectacular, self-immolating feedback loop. The load average climbed to 153. It was a cluster fire, quite literally. Then, in the chaotic aftermath of patching the inferno and moving that “expensive” embedding workload to a humble 15-watt ARM board, something remarkable emerged: the warm latency was a mere 2ms. Even a cold start, including model loading, clocked in at around 100ms. The sobering discovery? The 700ms was never about the embedding operation itself. It was the embedding struggling under full CPU saturation, choked for headroom. On a quiet, dedicated machine with a warm model, the exact same task takes 2 milliseconds.