Project ยท July 2026

nameframe

A brand-naming platform with a calibrated scoring engine: a deterministic Go scorer across 12 axes, plus an LLM judge that stays advisory until it proves agreement with human taste.

Ai Evals Go

Name generators promise the perfect name in seconds and deliver 500 unsorted junk words. Nameframe automates the whole naming project instead: brief, generation, scoring, availability, voting. The current milestone exists because I caught the scorer lying. One evening the top blended results were zmetli, shlailwify, veehhle, laisfoo, all clustered around 0.76. The ruler itself was broken, and fixing the ruler became the product.

Mission Control, the operator console: run inspector with the junk-cluster run selected, per-axis fitness vector on the right

That screenshot is the actual evidence run. Three axes saturated at 1.0, aggregate 0.92, for names no human would ever ship. The fix: distinctiveness got de-saturated against a 135,000-word phonetic index with a legibility floor so unpronounceable strings can’t earn novelty credit, and trademark plus cross-cultural risk moved out of the weighted mean into hard veto gates.

Under the hood

There are two scorers with a strict authority split. The deterministic Go engine scores 12 axes: biphone-probability pronounceability, sound-symbolism tone fit, per-language hazardous-onset checks. The LLM judge scores five subjective axes against a rubric grounded in published naming methodologies, with anti-familiarity-bias anchor pairs so it stops rewarding legible compounds over ownable coinages.

The judge is advisory by construction. Its weight is server-clamped to zero until a calibration gate passes: Spearman rho against blind human labels, computed by hand-rolled, unit-tested agreement stats, gated on a holdout split so tuning can’t grade itself. It cleared the 0.40 floor at rho 0.4555 against 30 blind labels.

The calibration baseline: rho, weighted kappa, MAE, the human ceiling, and the judge pin that re-keys the cache on any change

The generator head-to-head was pre-registered: the new Go engine had to beat the old TypeScript one at a 0.50 decisive win rate, judged only after the judge itself cleared agreement with humans. Go lost, 0.33. The rewrite was deferred by data, and the prereg file forbids editing the bar after seeing the result.

The pre-registration file, anti-p-hacking clause included

Work quantified

Six shipped milestones and a seventh in flight, 500 plus commits, roughly 549 TypeScript tests and 170 Go tests, golden fixtures pinning exact generator output per seed. Next.js front, Go microservice, Supabase, Vercel.

The public landing page

← All projects