Guardrails

I have been building an agent platform on my own hardware one post at a time. This is the whole thing in one place: how a job gets dispatched, which model runs it, and what it has to prove before I believe it.

Three live API keys had made it into the searchable index of my own working history, straight through a redaction step that ran on every byte. The fix was small; deciding how to measure it before trusting it was the real work.

A routine tidy-up of my project notes was handed to an automated worker. It went past its brief, so it was stopped, its own success report was thrown out, and the recovery was done by hand against an approved list with every deletion checked twice. The good suggestions it made were kept.

The free model on my own hardware kept failing halfway through jobs. The cause was not on my side, and the fix was to stop asking the model to write its own proof of work.

A standing read-only collector that checks contract parity, machine-to-machine reachability, required services and timers, and required paths across the fleet. Three design choices carry it: it keeps last-known-good evidence, it separates drift from probe failure, and it never repairs anything.

The harness that runs my local model was reporting success unconditionally, no matter what happened. Hardening it went in four phases, each one proving the defect on the unmodified code before patching, and ended with a single terminal contract that maps every way a run can end to an honest status and exit code.

The free model on my GPU box was unreliable in a way I could not name. Before swapping it, I found the reason by reading its transcripts, then ran a controlled A/B of six scored tests in an isolated copy of the real harness. The candidate won 5 to 2, a longer soak passed 21 of 22, and the swap was one line.

Prospect got meaning-based search fused with its keyword search, and a semantic check for reposted listings. Both had to be designed so they could fail without taking anything down, and the duplicate threshold was calibrated on a copy before it touched the real database.