Skip to content

Acceptance report: Retold v1

Final quality gates

Gate Required Measured Status
Ordinary conditional injection at most 5% 1 of 20 and 0 of 20 on the fifth and fourth blind splits (configuration A); 0 of 20 and 0 of 4 in the shadow harness met
Explicit stored-fact recall at least 90% 10 of 10 and 9 of 10 met
Implicit memory-needed recall at least 75% 7 of 8 and 6 of 7 met
Ambient-preference recall at least 95% 3 of 3 eligible preferences promoted on the fifth split; the promotion split passes on both independent builds with identical promoted sets met
Helpful precision among admitted records at least 95% 19 of 20 (95%) and 17 of 17 (100%). One disputed label, the user's time-zone record on a scheduling turn, was adjudicated blind by an independent reviewer as helpful; under the blind labels alone the figures are 18 of 20 and 16 of 17. The adjudication is recorded in usefulness-gate.md. No bundle component changed. met
Unsafe admissions and promotions zero zero placebo, misleading, private, or unsafe admissions in every run of the suite; zero unsafe promotions met
Provider and budget failures return the baseline always every fail-closed branch of the orchestrator is tested; the one planner timeout observed in a shadow run served the draft met
Stage timeouts bound the turn a hung planner or judge costs no more than its timeout a 5 s policy against a 50 ms timeout returns in under 1 s for both stages met
Cross-principal leakage zero zero violations on the synthetic four-user store and on the 1,074-record fixture, across every log stage, memory_get, and grants met
Dense-floor calibration the configured floor inside the band whose F1 is within 90% of the best best F1 0.99 at 0.50; the configured 0.45 sits inside the 0.44 to 0.66 band on 50 relevant and 50 irrelevant labelled queries met
Warm search latency p50 under 80 ms p50 23.2 ms, p95 28.0 ms on the 1K fixture with the real embedder; p50 73.7 ms, p95 78.3 ms on 50K records with a fixed 25 ms embedding cost met
Write latency p50 under 150 ms p50 26.3 ms; p95 471 ms on a write whose attribute scan reaches the NLI judge met
Concurrent writers beside extraction zero lock errors four writers and two extraction workers on one file: 160 writes and 20 extractions in 0.2 s, write p50 0.6 ms met
The supported bundle rebuilds from the package alone components equal the recorded manifest bundle_components(supported_bundle(), supported_retrieval_config()) equals bundles/bundle-2026-09-08-a.json, retrieval hash e8c8c3309ab121de met
Standard suite, Ruff, format, strict mypy pass on every advertised Python version 391 passed on 3.12 and on 3.13; clean on both, 60 source files met
Local-model and slow suites pass 9 passed in 191.7 s on macOS arm64 met
CI green on every advertised version run 34388291557: python 3.12, python 3.13, docs, wheel, and dependency audit met
Documentation builds strictly no warnings mkdocs build --strict clean met
Distribution metadata accepted by the upload path twine check passes on the wheel and the sdist met
Clean installation the wheel installs and runs outside the checkout installs with both adapter extras into fresh 3.12 and 3.13 environments and passes 24 smoke checks met
Known vulnerabilities in the locked set none unfixed five transformers advisories cleared; four chromadb advisories have no fixed release, arrive only through the crewai extra, and are ignored by identifier met, exception recorded
Tagged release with installable artifacts wheel and sdist attached to a GitHub release v1.0.1 released with the artifacts CI built; the wheel installs from the release URL met
Published package pip install retold works not on PyPI: the trusted publisher has not been created open

Sources: benchmarks/results/phase0/phase11-baseline-v5 and -v4, benchmarks/results/shadow/*phase11-baseline.json, benchmarks/results/promotion/20260908T101548Z-*.json, benchmarks/results/rescore/20260908T184734Z-rescore.json, tests/integration/, tests/test_utility_aware.py, tests/test_reference_host.py, scripts/wheel_smoke.py.