The thing that clicked today: two sibling photo-review experiments stopped being two things and became one, Loupe, running as a real managed service with its own brand, its own cloud backup, and an enrichment layer pulled straight out of Apple Photos. It opened with a recovery I didn't ask for and ended with a lot of threads tied off. Here's the whole day.
Built / shipped
- Two apps merged into one. I had two sibling review tools running side by side. I folded them into a single app, Loupe, built from the broader library tool as the base, with the candidate-review view (rule badges, per-rule chips, false-positive rescue) layered back in, all backed by one unified decisions database. I migrated the six real cuts from the old tool, cut over to the new app, gave it a public hostname, and deleted the two old server programs and their separate decision databases, but only after verifying the cutover, and only after archiving the old code and backing up the database first.
- Made it survive a reboot. The whole stack had been hand-started terminal sessions with no auto-start (which is exactly why the morning went the way it did; more below). I turned Loupe into a managed system service that restarts on crash and comes back after a reboot. Verified by killing it and watching it return in about three seconds.
- Brand and media. The app got a real identity: a wordmark, an icon, a favicon, an animated loading spinner, an italic display-serif header. On the media side it gained on-demand video conversion (capped at 720p, cached) and a 2048px preview-JPEG path so Apple's HEIC photos actually render in a browser, plus portrait contact-sheet cards with cycling backdrops sampled only from the local thumbnail cache.
- Full-library thumbnail backfill. A one-time job to pre-generate thumbnails for the whole library, about 91,000 of them. Ran roughly five and a half hours, finished all but five (corrupt videos), then I disabled it.
- Cloud backup of the originals. An upload-and-verify backup of ~1.65 TB of original files to cloud storage. Deletes nothing, with a resume loop around the provider's daily upload cap.
- AI period summaries. A three-layer feature: plain computed facts first (counts, time-of-day, cameras, location clustering, the usual home area, busiest day), then a venue name per cluster, then short prose grounded in those facts. Day, week, and overview views, cached forever after first generation.
- An enrichment layer from Apple. The biggest architectural win of the day: a read-only database that joins Apple's own scene labels, recognized text, people, and aesthetic scores back onto my library.
Problems & fixes
The box had silently rebooted. I came back to a dropped connection and a dead stack and assumed the worst: a crash, a storage fault, a power event. It was none of those. The reboot log plus a bumped kernel version told the real story: an unattended security upgrade had cleanly rebooted the machine overnight, and because the whole stack was hand-started with no auto-start, everything died with it. Checking reboot history before assuming a fault saved me an hour of chasing a phantom. Everything on disk was intact; I just had to restart it. This is the exact pain that motivated making Loupe a managed service later the same day.
The thumbnail generator was failing 100%, and it was running out of memory. It wasn't a code bug in the obvious sense. It was trying to decode up to 200 large HEIC images at once on a machine with only about 8 GB of RAM, and the box ran out of memory. The fix was to cap how many run at once (twelve image workers) and split the work into an image phase and a video phase. That cap now matters more than almost anything else in the pipeline. Every path that touches decoding respects it, and I've written down: never raise it.
Two silent write failures. Both the image library and the video tool decide the output format from the file extension. Writing to a temp name ending in .jpg.tmp makes both of them silently fail: no error, no file. The fix: encode to a JPEG in memory and write the bytes, or write to a …tmp.jpg name and rename afterward.
The public site was returning 502. The fix was making the app listen on all network interfaces and pointing the tunnel at the right address. The tunnel runs on a different host than the app, so pointing it at the app's own machine address could never have worked.
Summaries kept naming home after a nearby business. The venue lookup was labeling clusters by whatever shop sat a few meters away, so a quiet day at home came out labeled after some random nearby café or restaurant. And an earlier "discount far outliers" rule had erased a real trip: a genuine drive out to a ranch got filtered as noise. Two lessons collided here: an erased real place is invisible and unrecoverable, while a wrong label is at least catchable. So the outlier discount is gone entirely and only a minimum-cluster-size filter remains: under-filter, never over-filter. For home specifically, the label is the suburb name plus a distance check, rather than fighting it with radius tuning that the geometry didn't support anyway.
The location provider rejected the call. The older lookup API returned a flat denial for a new project. The newer nearby-search API worked, and skipping minor points of interest (bathrooms, parking lots) let the real venue resolve.
Decisions
- One app, one decisions database, with a hard gate: all six existing cuts verified present before any old database got deleted. It passed.
- Never send full-resolution originals over the public tunnel. Only derived previews and converted video cross it; the raw full-resolution path stays reachable only on my home network.
- The backup deletes nothing, ever: upload-and-verify only.
- Anchor enrichment to the Apple UUID, not fuzzy filename matching. My pipeline database has no Photos UUID, so the only join available was fuzzy filename matching at ~79% with ambiguity. The better call was to fix the root cause: do a one-time export from Photos keyed on the stable UUID, so labels, people, and scores all join on one key, and it's reusable for any future Apple pull.
- Accept the ~80% coverage ceiling honestly. The UUID bridge landed at 73,157 of 91,537 assets, about where the fuzzy join already was. But the structural reason is sound: ~10k of my rows are derivatives and duplicates that were never distinct Photos assets, plus ~8.6k with recycled names and junk timestamps. What the bridge bought wasn't more coverage. It was exactness: confidence-tiered matches, and a refusal to mis-assign the ambiguous rows rather than inflate the number by guessing.
Learned
- Check reboot history before assuming a crash. A dead stack and dropped connection look identical whether the cause was a fault or a clean upgrade reboot. The logs tell you which in seconds.
- Too many decodes at once will take down your own machine. Two hundred large-image decodes on a small box is an out-of-memory event, not a throughput win.
- Image and video writers pick their format from the file extension: a
.jpg.tmptemp name makes them fail silently. - Apple's timestamps are a mix, not a drift. Some capture timestamps are true UTC and some are local wall-clock stored as if it were a UTC value. That's not a fixable offset. It forced a wide matching window and it biases any time-of-day bucketing. This is now the most consequential open data-quality issue I have.
- Two independent signals agreeing is a real signal. Apple's text/document labels and its aesthetic score separately agree that screenshots are the lowest-value content in my library. That's the first category to attack in any bulk cull.
Still open / next
- The cloud backup is still uploading (~1.65 TB, multi-day), needs a final verification pass when it finishes.
- Five unreadable videos need a re-pull from the source library; I wrote a list for it.
- Summaries generate on first view. Only one day has been produced live so far.
- The enrichment work spawned a set of design prompts (wiring the labels and scores into the actual cull workflow under a "labels nominate, score orders, people protect" principle, a per-frame signal panel showing scores as library-relative percentile bars, and a Live Photo pairing treatment) that were issued but not all confirmed built. That's the next session's thread.