Monitoring

Turned the fleet dashboard into an installable phone app with alerts tiered by urgency, a critical channel that survives the app itself failing, and a dead-man's switch that can announce its own death.

My busiest compute machine was cooking itself under real load, so instead of just turning the work off, I built layers of protection that let it keep running without frying.

I caused an outage by running a heavy job on the one machine that holds everything together, then spent the night proving my failover plan actually works.