Code needs proof. Integration needs review.
Inspect the local harness validation, then explore the historical context benchmarks. Each result has its own scope, protocol and artifact availability.
Harness validation added: October 10, 2026
# same task, same repo base
$ run baseline_agent
measure: searches, files, missed impact
$ run snipara_assisted_agent
measure: same signals, same answer key
raw replay trace: not publicly available
The purpose is narrow: show whether shared Project Intelligence changes the agent's starting point before implementation begins.
The agent can change. The work continues.
Claude Code created a plan and completed the first two phases. A fresh Codex session resumed the saved Companion checkpoint and finished the next phase. The plan, decisions, completed tasks, attempt counters and original evidence stayed intact.
21 independent validation cases passed on October 10, 2026. One local fixture, using Companion 3.7.0 and a shared workspace with operator-assisted command dispatch.
Resume requires access to the project folder, including saved .snipara state and workflow artifacts, and the project’s Snipara MCP. This demonstration does not qualify live-process migration, MCP-only restoration or transfer of an Orchestrator controller between coding clients.
Independent checks ran outside the producing Codex session. This is an operator-reported demonstration, not a reliability benchmark or authenticated human acceptance.
One paired continuation, with and without Snipara
Both fresh Codex sessions received the same Claude-produced source, plan, decisions and ASCII evidence. The Snipara session also received the saved checkpoint and Hosted MCP access. Both passed; this fixture showed no speed or token saving.
| Observation | Ordinary files | Files + Snipara |
|---|---|---|
| Independent validation | 21/21 | 21/21 |
| Full agent session | 30.94 s | 55.63 s |
| Successful resume command | Not applicable | 2.38 s |
| Separate successful MCP probe | Not applicable | 1.06 s |
| Actual billing | Not measured | Not measured |
The Snipara session includes one network-permission retry. Successful command timings exclude the failed probe; full session time includes it. Session time includes implementation and is not resume latency. Avoided task rework, time to first useful progress and total harness overhead were not measured. One small fixture is insufficient for a general performance claim.
Reproduce the scenario: scripts, frozen inputs and both candidates · Download the paired result and token counts
Download the public checkpoint receiptThe harness was tested on both outcomes.
On October 10, 2026, local development runs exercised Companion 3.7.0 and Orchestrator 1.9.0 from published packages. One journey reached acceptance. Another passed external validation, then stopped after independent review refused the code.
Download the public validation summary. This is an operator-reported summary; raw native traces are not public.
- Context
- Plan
- Code
- Validate
- Review
- Integrate
- Code
- Qwen3 Coder Next · LM Studio
- External validation
- Passed · tree unchanged
- Independent review
- GPT-6.1 · refused
- Integration
- Blocked
A child’s cancellation exception was replaced by the wrong error. The review caught it before integration.
Recorded local tests · October 10, 2026. One development fixture. Scope and evidence
What this demonstrates
Scoped dispatch, external proof, separate review and an integration gate. The accepted Claude journey also exercised a bounded retry after an intentionally injected proof failure, then reversible integration and final acceptance in a temporary repository.
How to read these results
- One previously exposed async Python development fixture. These runs do not establish general coding reliability or fresh private benchmark performance.
- The Claude invocation used a test-local launcher to transmit permission, tool and cost controls. The first malformed review was blocked; a separate review with a corrected format accepted the candidate.
- Qwen3 Coder Next through LM Studio passed proof, but review found that a child’s CancelledError was replaced by ValueError. The defect was reproduced separately; no integration followed.
- Review was performed by a separate model session with operator-asserted reviewer identity. It was not authenticated human review.
- All integration happened in an isolated temporary repository. No production deployment was part of these tests.
A controlled replay with an answer key
The final implementation of a real Snipara task becomes the answer key. The replay measures how much work an agent does before it reaches that surface.
Same base
Both runs start from the commit before the selected Snipara task moved.
Same brief
The task request, model family, shell access, and time budget stay fixed.
Different start
The baseline agent starts cold. The assisted agent receives Snipara Project Intelligence first: decisions, impact, workflow state, and verification evidence.
Same answer key
Both runs are scored against the final merged workflow evidence, not against a marketing claim.
Safe Parallel Coding is wide enough to expose the real failure mode.
A narrow UI copy change would be too easy. This replay uses work that touched code, docs, package surfaces, agent tools, and verification gates.
Complete Safe Parallel Coding MVP5/MVP6
It crosses repository code, hosted MCP contracts, package surfaces, docs, checks, and deploy notes.
499d63a3, before the MVP5/MVP6 consolidation started
9785471a, after the public Safe Parallel Coding surface shipped
14 commits, 76 changed files, 5,475 insertions, 270 deletions, and 31 test/support files.
Shared Project Intelligence reduces rediscovery and improves planning before the agent writes code.
The difference was visible before implementation
The cold run could find relevant files, but the signal was buried in broad repository search results. The Snipara-assisted run started from the project-owned surfaces and verification gates.
files in the answer key
Git diff from 499d63a3 to 9785471a
unique local search hits
16 cold-start searches against the base commit
actual files opened cold
Top-five-files-per-search replay rule
surface categories surfaced
Project Intelligence brief, workflow plan, and impact context
16 local searches
3 Snipara artifacts
Project Intelligence, managed workflow plan, and code impact replaced broad repo rediscovery.
47 opened, 5 were final changed files
5 anchor files plus phase-level surfaces
The baseline found signal, but with much more noise and fewer implementation anchors.
3 of 7 categories in first opened files
7 of 7 categories in start context
Scored against web API, data/service, dashboard UX, CLI, hosted MCP mirrors, docs, and release/deploy config.
Likely local tests after file discovery
Risk, impacted routes, config facts, release/deploy checks
The code-impact artifact classified the guard surface as critical risk with routes and config evidence.
Not claimed
Not claimed
This replay did not preserve comparable raw model timestamps, so the page does not invent a duration.
The same proof shape now runs against local coding models
The replay above measures discovery and planning on a real Snipara task. The local-model benchmark measures the next layer: whether retrieved Project Intelligence changes code quality and task success under realistic handoffs. Together, the two benchmarks measure both project understanding and code execution.
GPT-5.3 Codex Spark solved the most continuity-dependent scenarios cold. That imperfect baseline is the useful point: strong models can solve some tasks without Snipara, but still miss project-history-dependent work.
These are not generic coding tasks. They are scenarios where the correct answer depends on prior decisions, handoffs, active work, and project-specific constraints.
The page should include a control class where cold agents already succeed and Snipara should show little or no lift. Until that run exists, this page does not claim broad coding uplift.
What was measured
Ten paired repetitions, six coding scenarios, three conditions: cold baseline, hosted Snipara retrieval, and an oracle continuity pack. Scoring combines hidden tests, static validity, maintainability, scenario review checks, and local project-intelligence checks.
Readout: in continuity-dependent scenarios, Snipara-hosted retrieval moved all three local models from 0/60 baseline passes to 60/60, 57/60, and 53/60 respectively.
Read the benchmark protocolLimits kept in the claim
The same proof shape also runs through Codex CLI
This table uses the same continuity-heavy coding scenarios as the local-model benchmark, but swaps the runtime from LM Studio to Codex CLI. It is reported separately so local-model claims and Codex cloud claims stay clean.
Readout
Across three Codex CLI models and ten paired repetitions, Snipara-hosted retrieval moved aggregate continuity-dependent passes from 25/180 to 179/180. The quality and continuity scores are the same deterministic 0-100 scoring dimensions used in the local-model table.
GPT-5.3 Codex Spark had the strongest cold baseline on this suite. GPT-5.4 scored slightly above GPT-5.5 in the Snipara condition. Those are benchmark observations, not a global model-ranking claim.
Read the benchmark protocolLimits kept in the claim
The same proof shape also runs through Claude
This table uses the same continuity-heavy coding scenarios as the local-model and Codex CLI benchmarks, but reports Anthropic Claude runs separately so provider and runtime claims stay clean.
Readout
Across Claude Sonnet and Claude Opus, Snipara-hosted retrieval moved aggregate continuity-dependent passes from 7/120 to 120/120. Code quality reached 98.4 for Sonnet and 98.3 for Opus; continuity reached 98.4 and 98.3 respectively.
Opus had a stronger cold baseline than Sonnet on this suite, but the main signal is consistent: both premium Claude models reached full pass coverage with hosted project continuity.
Read the benchmark protocolLimits kept in the claim
Only the starting context changes
The baseline is not sabotaged. It can use normal developer tooling. The assisted run receives Snipara context first, then still has to verify it against the repository.
Run A: cold agent
cold startOriginal engineering brief only
Git, local files, shell search, test runner
Architecture, active decisions, changed surfaces, package mirrors, docs, checks
16 searches returned 1,722 unique files; opening the first five results per search surfaced 5 of the 76 final changed files.
Run B: Snipara-assisted agent
with SniparaSame brief plus Snipara start-work context
A Project Intelligence brief, the managed workflow plan, prior phase handoff context, and a code-impact response.
Snipara named the guard route, data query, safety service, companion command, docs guide, four phases, and release/deploy gates.
All seven final surface categories were present in the assisted start package before the first edit.
This is how shared reality becomes bounded work
The Lead Plan is the proof boundary between Project Brain and execution. It lists recommended workers and evidence, but it stays fail-closed: no worker is launched, approved, or completed without explicit handoff and receipts.
Advisory contract, not autonomous execution
Update public docs and AI-readable discovery surfaces.
Required proof: String probes, docs route render, and discovery tests pass.
Check that multi-agent copy stays fail-closed.
Required proof: No autonomous worker launch, approval, or completion is claimed.
Hold release until proof and neighboring hardening are reviewed together.
Required proof: No production deploy until the combined release gate is explicit.
Proof gates
Only public proof, docs, metadata, and AI discovery files change.
Run focused discovery tests, web type-check, lint, and production build.
Push and deploy only after the coordinated release decision.
Receipts and outcomes
Worker scope, touched files, caveats, and next action are captured instead of left in chat.
Checks run, failures, skipped gates, and evidence links travel with the work package.
Accepted or rejected work updates the Project Brain so the next plan starts from evidence.
Candidate Project Brain updates
Candidate updates are not durable truth until a handoff, proof receipt, and outcome link exist.
How the reported scorecard was calculated
These rules describe the reported replay. The raw trace is not currently public, so the results should be read as a documented study summary rather than an independently reproducible package.
Count repo searches, Git inspection, and local file discovery before the first plan.
Count unique files read before the plan names the implementation surface.
Compare named routes, services, packages, docs, and tests against the answer key.
Compare proposed checks with the final verification list and missing gates.
Measure time until the plan includes impact, risk, verification, and next action.
Count issues found before commit, including wrong assumptions, failed checks, and stale context.
The final workflow defines what the agent needed to find.
We score both agents against the surfaces that actually mattered in the completed Snipara work. That keeps the comparison grounded in repository evidence.
Derived from final workflow evidence
What you can inspect today
The protocol, reported results, scoring rules, and limits are public here. Cards below state when the underlying artifact is unavailable instead of implying a download exists.
Start Work Brief
The page describes the fields supplied before the assisted agent opens files. The underlying brief is not published.
Impact Chain
The compared surface categories are reported on this page. The raw impact response is not published.
Verification Plan
The measurement rules are available below. The original verification-plan payload is not published.
PR Answer Pack
No sanitized PR Answer Pack is currently available for download.
Raw Agent Trace
The raw agent trace is not currently published. This page does not present a downloadable trace.
Protocol and limits
The public protocol, reported results, and interpretation limits are available for review.
This does not prove that Snipara writes better code.
It tests a narrower and more defensible claim: Snipara gives an AI coding agent a better project starting point before it writes code. The model still reasons, edits, runs tests, and owns the final implementation quality.
How to read this proof
The short answers below describe the scope and limits of the reported replay.
What does the Snipara proof page measure?
It measures whether shared Project Intelligence changes an AI coding agent's starting point before implementation, using the same task, repository base, answer key, and scoring rules.
Does this benchmark prove that Snipara writes better code?
No. It tests the narrower claim that Snipara gives an AI coding agent a better project starting point. The model still reasons, edits, runs tests, and owns the final implementation quality.
What stays fixed between the baseline and assisted runs?
Both runs use the same base, task request, model family, shell access, time budget, and answer key. The starting context is the changing condition.