Build Logs from the Lab
Real engineering stories. Kernel panics at midnight, RAID failures on Christmas, and the occasional moment where everything just works.
The RAID That Almost Ate Christmas
Four HGST drives. One aging Synology. USB docks that struggled under sustained recovery load. The week between Christmas and New Year's became a crash course in mdadm, LVM, Btrfs, and choosing the right hardware path for data recovery.
K3s Stability Tuning and Reboot Loop Diagnosis
Diagnosed a continuous reboot loop caused by rapid pod restarts, CNI churn, and Ubuntu's default panic reboot behavior, then stabilized the host for deeper inspection.
Cloudflare Workers SSR Compatibility Debugging
A Cloudflare Pages compatibility change caused Astro SSR pages to serialize incorrectly. The fix was isolating the response-body mismatch and bridging Astro's output to a Workers-compatible stream.
Build Swarm Stabilization Sprint
A 12.5-hour stabilization sprint resolved 25+ build-swarm issues across 4 components and deployed the stabilization build across the fleet before sunrise.
The Reboot Loop That Blamed the Wrong Code
When Alpha-Centauri started rebooting every 90 seconds, I was convinced my build swarm code had achieved sentience and was trying to escape. Spoiler: it was innocent.
Build Swarm Validation and Container Runtime Cleanup
A build-swarm validation pass tightened binary handoff semantics, corrected thread-state handling, and updated a container runtime path that was still carrying assumptions from an older architecture.
Rsync Guardrails for the Build Swarm
A 7-hour build-swarm debugging session added transfer timeouts, corrected SSH transport options, narrowed uploads to the intended package artifact, and produced a CLI tool for safer fleet operations.
Unraid Audiobook Library Path Consolidation
A systematic audit and reorganization of the AudioBooks share flattened a legacy nested directory structure, verified 3.5TB of data, and restored a clean Audiobookshelf container path.
The GRUB That Forgot Everything
Deleted a corrupted GRUB. Now /etc/grub.d/ was empty. os-prober couldn't see Windows or CachyOS. NVIDIA parameters were wrong. Found the working config in a backup file I didn't know existed.
These are the highlights. The full journal has the rest of the trail.
Browse All Journal Entries