mergeone updates One entry per pre-registered benchmark run. https://mergeone.dev/updates/ 2026-10-25T12:00:00Z mergeone Run 8, two passes, one fail: asked fairly, the compact state passes; the relation question still does nothttps://mergeone.dev/updates/#run82026-10-25T12:00:00ZAgainst same-day full-state controls: compact state v4 +0.010 / −0.001, pair −0.018 / −0.015 (bar −0.02), 32–42% fewer tokens — the pair call goes compact in v0.3. Relation v2 class accuracy 0.740 / 0.570 (bar 0.80); duplicate_of stays the only edge. The drift monitor replays run 7 exactly; drift.enabled defaults to true. Post-hoc: run 7's drift reads as a request-handling change at the provider; run 9 checks it. Run 7, negative four times: four bars missed, and the hosted model changed under the same version stringhttps://mergeone.dev/updates/#run72026-10-24T18:30:00ZCompact state −0.049 and tuned prompt −0.059 against run 5's numbers (same-day controls: +0.007, −0.008); seven structural features 0.48 AUC; typed relation 0.73 vs 0.80, class accuracy 0.32 → 0.72 once the base branch comes from metadata. Two of ten questions answer differently than six days ago on identical input; run 8 pre-registers same-day controls. Nothing ships. Run 6B, positive: a hold for duplication is right 98 times in 100; half the time the right move is a linkhttps://mergeone.dev/updates/#run6b2026-10-23T17:40:00Z100 blind-labelled holds: dup ∪ pointer 0.98 (bar 0.60), same change 0.44 (bar 0.30); 54 pointers — backports, promotes, competing fixes, follow-ups; κ 0.77 with a second labeller; pool size not the lever; $0.0004 per PR. Dedup stays a hold, never closes; the relation question goes to run 7. Reading: ×1 is not a projection any more, it is Septemberhttps://mergeone.dev/updates/#review-at-anthropic2026-10-20T13:30:00ZAnthropic's code-review post: 30,000+ landed changes a day in September (the ×1 base rate here), risk tier set from the change by a classifier, one commit per approval. Duplicates across agents are not in it; run 6B measures that gap. Gateway gets a max_rounds cap. Run 6A, negative: the open weights are not the limit, one GPU ishttps://mergeone.dev/updates/#run6a2026-10-18T19:00:00ZSelf-hosted pplx-decider-flash on one L4: 95.2% argmax agreement with the hosted endpoint but mean |Δp| 0.047 (bar 0.02); AUC 0.747 vs 0.758; single-question gains +0.000; saturates at ×10, $0.47/1k PRs at ×1. Run 5, mixed: a flag means more review; the stack rule chose wronghttps://mergeone.dev/updates/#run52026-10-18T12:00:00ZRetrieval v2 recall@5 0.66 (bar 0.70), store p99 3.3 ms; needs_human_review predicts ≥2 force-push rounds at AUC 0.76 (bar 0.70) for Jev and pplx-flash; stack misses 0.75; assumptions not evaluable; hosted Perplexity loses 71% of calls at 100×. Run 4, mixed: a cheap model can tell two PRs are the same changehttps://mergeone.dev/updates/#run42026-10-17T20:00:00Zsame_change AUC 0.83–0.87 for three System One models (bar 0.80); retrieval recall@5 0.55 (bar 0.60); memory ships opt-in. Run 3, negative: the gate cannot see that someone else already landed the fixhttps://mergeone.dev/updates/#run32026-10-17T10:00:00Z2,196 agent PRs, 26 repos: AUC 0.55 vs a 0.70 bar; 67% of closures are coordination, not quality. Jev passes 100× load. Run 2, positive: five typed questions, one call, 347 mshttps://mergeone.dev/updates/#run22026-10-15T20:00:00ZJev 5/5 questions at AUC 0.87–0.96, 347 ms, $0.074 per 1k PRs; Haiku 4/5 at 1,067 ms. Run 1, negative: nothing in a diff predicts a 30-day reverthttps://mergeone.dev/updates/#run12026-10-15T12:00:00Z40,000 Kubernetes merges; every candidate at PR-AUC 0.010, chance. Protocol frozenhttps://mergeone.dev/updates/#protocol2026-10-13T12:00:00Zgatebench: hypotheses, thresholds and data rules committed before any model call.