Selected work

The index is 264,626 frames turned into vectors by an image model, kept in Postgres with pgvector and written a second time as a Lance dataset on object storage. The same route answers out of either store, within two rows of a hundred on recall, so the store is a latency and cost choice and the eval says which.

A sampling thesis was tested and turned down. Scene cuts plus the frame where a narrator points took a tenth of the frames of one a second, and recall at ten fell from 0.39 to 0.20. The default search runs over every sampled frame and the write-up says why.

Built through Claude Code in a four-step loop, one implementer subagent per issue and an adversarial review before every PR, with every step and its model on the project log. A $25 monthly ceiling is enforced by an hourly Worker because neither provider sells a hard cap.

Two rules carry the project. A hypothesis is reportable only with a query that actually ran and a negation query that ran too, and the report has to list what was in scope and never checked. Both are validated in code against the run’s own tool log, not by a model.

The other half is the grader. A wrong top hypothesis at high confidence costs more than a wrong one hedged low, and a right answer hedged low costs a little too, so the ordering holds: confident-wrong scores below hedged-wrong, which scores below hedged-right, which scores below confident-right. A unit test pins that order.

Thirty of thirty Sonnet runs named the right cause or, on a control, said there was no incident, mean total 0.92. A second model, Nemotron, passes Honeycomb’s own process evaluator on all 30 runs and scores under 0.50 here on 25 of them, which is the case for grading outcomes instead of process.

The agent lives in Slack, Discord, and WhatsApp with thread memory, a SQLite fact store, and ChromaDB retrieval over each org’s own documents. It runs on a local model through Ollama, so the documents stay on their infrastructure.

A Cloudflare Workers API drafts briefings, reports, and replies with Claude. A grounded prompt refuses to invent, then a Haiku claim extractor reads back every generated statement and marks it grounded, partial, or unsupported. The document is held until a person clears the flagged ones.

That second pass is the piece I care about. A generated briefing reads exactly as well when it is wrong, so the check cannot be a person reading it. The same idea is what I would bring to an agent that troubleshoots production: make it cite the rows behind the claim, and make it say what it did not look at.

About 1,400 commits since January 2026 and 142 test files. The Python core traces into Honeycomb over OpenTelemetry.

Built to answer a question I kept hitting at work: can you get real transcription quality without shipping a user’s voice to somebody’s cloud. You can.

The hard part was memory. Whisper models plus an audio ring buffer plus the app itself will get you killed by the Android low-memory killer on mid-tier hardware. Rework of the buffering and model loading path took peak memory from 367MB to 230MB.

Accuracy is measured, not claimed: 8.38% word error rate against 14.97% for the platform recognizer on the same device and the same audio.

Public ballot scans, a local model, and no third party in the loop. Every stage of the funnel is inspectable, which is the whole point when the output is an election number.

The result landed within 1.4% of the official county totals. The gap is the interesting part, and the pipeline reports where it came from rather than hiding it in an aggregate.

It runs on one workstation over a few days. The cost of that is throughput, and for a job that has to be auditable afterward it is the right trade.

Transcribe, embed, retrieve, answer. Video and speech become one searchable index, so a plain question returns the passage that answers it instead of a filename.

Three models in one process on a phone, which makes the retrieval design the whole problem: what can stay resident, what has to be paged, what can be precomputed while charging. Getting good answers out of a small model means putting the right context in front of it, and there is no room to be sloppy about which rows those are.

Traces, metrics, and logs shaped for a phone rather than a dashboard shrunk to fit one. Different information hierarchy, different defaults, same data.