Evaluator-backed benchmarking: your benchmark is only as good as your evaluators Data Geoff RJune 9, 2026June 9, 202613 mins0 Evaluator-backed benchmarking is the structural counter to benchmark gaming. When the underlying evaluators carry verifiable identity, longitudinal…continue reading..
Reward models need reward-model QA Data Geoff RJune 8, 2026June 8, 202612 mins0 Reward model QA is the missing layer that turns step-level preference data into trustable training signal. When…continue reading..
The evaluator uniqueness primitive: from sybil resistance to agent evaluation Data Geoff RJune 5, 2026June 5, 202616 mins0 Evaluator uniqueness, the property that one person can prove they are one unique evaluator without disclosing identity,…continue reading..
Continuous training needs continuous evaluators Data Geoff RJune 4, 2026June 4, 202614 mins0 Longitudinal evaluation is the human-judgement layer that scales alongside continual model adaptation. A continually retrained model paired…continue reading..
Your reward model is only as good as your preference data Data Geoff RJune 3, 2026June 3, 202613 mins0 Preference data integrity is the upstream gate that determines what every distilled, fine-tuned, or RLHF-aligned model is…continue reading..
When benchmarks break: the case for traceable evaluator provenance Data Geoff RJune 1, 2026June 1, 202611 mins0 Evaluator provenance is the layer that turns benchmark results from “trust the publisher” claims into independently verifiable…continue reading..
Signed content for a world where platforms are AI Data Geoff RMay 23, 2026May 23, 202612 mins0 AI-mediated communication systems measurably shift the opinions of the groups they serve, and the question “did this…continue reading..
Reputation as public infrastructure Data Geoff RMay 22, 2026May 22, 202611 mins0 The supply of trusted AI evaluators is bottlenecked not by a shortage of humans but by platform-bound…continue reading..
Why persistent identity is the missing layer under AI evaluation Data Geoff RMay 19, 2026May 19, 202611 mins0 Model drift in flagship AI systems is often misattributed to changes in the model when it is,…continue reading..
From ownership to consent, audit and revocation Data Geoff RMay 5, 2026May 5, 202615 mins0 On the recent Ontology Privacy Hour, buried inside Nick Ris’s response to a question about why people want…continue reading..
