Networking

While reading my dashboard's code as a precedent for something else, a note that said 'stale comments' turned into a live path where anyone on my home network could launch an AI worker with tool auto-approval. Fixed at two layers, then the fix locked me out, which was its own lesson.

My storage box was showing offline, so I went looking at the storage box. The fault was a network port on a different machine that happened to be the storage box's only route out.

I assumed every machine could reach every other. Checking every direction properly found a quarter of the paths broken, and one repair job reported success while having quietly undone itself.

One machine showed permanently critical on the fleet dashboard while every other check against it was green. The probe was judging health on a ping command's exit code, and a ping that succeeds through a relay returns a real answer, a real latency, and a non-zero exit code.

Every cheap health check told me my storage link was perfectly healthy while it was actually running at a tenth of its speed, and the real work was proving which piece was lying.

Turned the fleet into a full mesh where every machine can reach every other one directly, passwordless, without hand-managed keys, laying the groundwork for running work anywhere in the fleet.

Adding one wireless card to the GPU box renumbered the PCI bus and renamed two interfaces, not one, which took down the isolated storage network's gateway through a chain nothing had advertised. The rebuild made the two radios a primary and hot spare, with a measured failover gap and an idempotent service.

Moved the public tunnel entrance off the busy compute box onto its own dedicated Raspberry Pi with its own uplink, so a reboot of any other machine can no longer take the whole public site down.