Files
Agent-Workbench/docs/delivery-reliability-g-evidence/development/frozen-regression.tap
T

171 lines
18 KiB
Plaintext
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
✔ input that fits the engine window but exceeds remaining credit diagnoses the credit limit (0.4781ms)
✔ window overflow and historical pauses retain distinct and honest diagnoses (0.0846ms)
✔ a budget pause persists concrete text/tool/history/image breakdown and never dispatches an oversized request (2254.359ms)
✔ explicit frozen preview resumes the same Task in a new Run, preserves completed tools and does not change global config or selected model (3163.6716ms)
✔ insufficient budgets, silently clamped capacity and arbitrary config fields reject without a physical request or mutating task budget (2515.9725ms)
✔ changed session, changed registry, expiry and altered frozen preview all require a new explicit preview (2396.7525ms)
✔ preview preserves unknown invocation status and increasing Task allowance is committed only with confirmed new Run (3280.2137ms)
✔ batch APIs enforce ownership and review fingerprints, apply once, and persist the completed batch across restart (6288.6768ms)
✔ delivery reports bind actual batch identity and reject corrupted snapshot evidence without changing project files (4014.9419ms)
✔ a compatible model can seal proposals through the real HTTP loop while application stays behind user review (6595.7798ms)
✔ a sealed batch at the step limit hands off and continues the same Task into actual checks without replaying saved tools (15211.4275ms)
✔ applied batches cannot turn cancellation, other budgets, restart or unknown commands into a review continuation (2469.6939ms)
▶ complete sealed batch hands off after all tools settle with no next model request
✔ complete (4214.6965ms)
✔ later-error (4835.0104ms)
✔ complete sealed batch hands off after all tools settle with no next model request (9050.3886ms)
▶ partial, changed, failed or cancelled batches cannot claim complete review handoff
✔ partial (4499.5806ms)
✔ new-proposal (4157.7788ms)
✔ failed (4456.5093ms)
✔ conflict (4586.3256ms)
✔ cancelled (3893.4829ms)
✔ unknown (4707.8593ms)
✔ tampered (5051.9704ms)
✔ partial, changed, failed or cancelled batches cannot claim complete review handoff (31354.9596ms)
✔ actual process exit at file-batch prepared preserves durable identity and never replays writes (1810.9369ms)
✔ actual process exit at file-batch application_started preserves durable identity and never replays writes (2144.4908ms)
✔ actual process exit at file-batch write_intent preserves durable identity and never replays writes (1985.2734ms)
✔ actual process exit at file-batch staged preserves durable identity and never replays writes (2348.0761ms)
✔ actual process exit at file-batch replaced preserves durable identity and never replays writes (2281.0135ms)
✔ actual process exit at file-batch file_recorded preserves durable identity and never replays writes (2542.0479ms)
✔ actual process exit at file-batch finished preserves durable identity and never replays writes (2645.7507ms)
✔ actual process exit at file-batch rollback_started preserves durable identity and never replays writes (3307.3518ms)
✔ actual process exit at file-batch restore_intent preserves durable identity and never replays writes (3203.6028ms)
✔ actual process exit at file-batch restore_staged preserves durable identity and never replays writes (2872.6014ms)
✔ actual process exit at file-batch restored preserves durable identity and never replays writes (3207.1629ms)
✔ actual process exit at file-batch restore_recorded preserves durable identity and never replays writes (3456.625ms)
✔ actual process exit at file-batch rollback_finished preserves durable identity and never replays writes (3291.3962ms)
warning: in the working copy of 'README.md', LF will be replaced by CRLF the next time Git touches it
warning: in the working copy of 'src/two.mjs', LF will be replaced by CRLF the next time Git touches it
✔ sealed batches preserve bytes until review, then apply and explicitly restore BOM/CRLF/new files (2892.13ms)
Switched to a new branch 'owner-branch'
✔ a conflict in the last file or Git branch prevents writing every file in the batch (2881.8143ms)
✔ duplicate canonical paths, ancestor collisions, links, dependencies and Windows aliases never form a batch (1391.4823ms)
✔ fingerprints, changed proposals and corrupt snapshot artifacts cannot write or restore any target (2466.4145ms)
✔ a failure after replacement restores only batch bytes and never retries an old application (1727.9885ms)
✔ rollback preserves owner edits, blocks other project writers, and explicit acknowledgement releases recovery (2384.1335ms)
✔ an unattempted owner-created file identical to a suggestion remains external and is never removed (1809.7168ms)
✔ active batch requests are serialized and cancellation leaves a verifiable ended batch (1452.2638ms)
✔ batch application and rollback preserve unrelated Git index and worktree bytes (3214.0272ms)
✔ recovery reuses exact staged bytes but retains a conflicting temporary file for owner inspection (2623.7291ms)
✔ one Task can seal pending proposals from earlier Runs while snapshots retain each source Run (1615.13ms)
✔ controlled installation pins the native kernel and bridge independently of the existing connection-probe identity (959.0218ms)
✔ controlled profiles disable every native project/terminal provider and native model/retry route using the actual include patch contract (0.5566ms)
✔ owned tool catalogues seal full schemas canonically and reject duplicate or malformed declarations (5.3393ms)
✔ bridge authentication, current Run, process, SDK session and browser-origin checks reject before dispatch (735.109ms)
✔ bridge duplicates, unknown tools, malformed arguments and source changes are refused before a second side effect (929.2965ms)
✔ bridge model requests bind the exact catalogue, route and output bound and preserve raw request/result identity (637.5648ms)
✔ stopped and oversized bridge requests never reach a local model or tool callback (764.412ms)
✔ sealed r1-off failure fits its original 16K config with authentic latest status and readable source (14.1493ms)
✔ sealed r1-on failure fits its original 16K config with authentic latest status and readable source (10.7966ms)
✔ sealed r2-on failure fits its original 16K config with authentic latest status and readable source (8.5914ms)
✔ sealed r2-off failure fits its original 16K config with authentic latest status and readable source (8.7137ms)
✔ archived native call pairs require exact saved arguments/results rather than model-written summaries (8.3391ms)
✔ full system/user requirements and mandatory current failure can refuse rather than be trimmed (13.0742ms)
✔ small native input is byte-preserving; developer instructions remain mandatory after compaction (0.2426ms)
✔ older failing checks and pending proposals remain visible after a later large file read (14.7652ms)
✔ bounded result keeps every check status and cuts Unicode only on whole characters (0.4404ms)
✔ controlled engine HTTP freezes one child, pages original owned input/report, rejects foreign reads and prevents changed-source reuse (15113.5555ms)
✔ native text and call IDs preserve original result pairing while unsupported content fails closed (1.3291ms)
✔ one engine child freezes real installation identity before creating tasks and rejects a changed source (8460.0535ms)
✔ owned native tools return saved results, use real approvals/checks and account each actual request once across new engine Runs (26594.9272ms)
✔ forged native scope cannot change a parent file and a final completion claim cannot replace worker proof (13479.2898ms)
✔ duplicate native IDs are rejected without replaying a saved tool (11901.6727ms)
✔ native input exceeding the context budget is retained and sends no model request (10305.5649ms)
✔ bounded native sends verify original tool provenance and reject forged views or missing versions (13687.7538ms)
✔ stop retains a late reported model usage once and executes no tool from that output (10848.4664ms)
✔ a source change after the original model intent stops before native tools execute (10222.0307ms)
✔ an engine source change while approval is pending invalidates the old permission before any check starts (14590.1925ms)
✔ missing or noncompleted turn end cannot be adopted even with an owned receipt and assistant text (0.5232ms)
✔ closed root with an original live descendant remains unknown and retains its process tree (0.2726ms)
✔ host recovery reserves original pending attempts exactly once and explicit review preserves unknown outcomes (15021.7273ms)
✔ corrupted model reservation cannot reduce recovered budget or permit a new engine request (12877.3045ms)
✔ saved engine budgets verify all Chat/Responses route variants and reject a mismatched output allowance (695.6396ms)
✔ owned CC Switch SQLite imports the selected TOML Responses wire_api and leaves unknown clients untouched (813.027ms)
✔ Responses executes one original tool through Runtime, carries encrypted Items, reports raw usage and seals exact wire bytes (1908.0632ms)
✔ mixed-protocol pool keeps both actual variants and changes route on 429 without re-executing completed tools (2258.6802ms)
▶ Responses incomplete, failed, missing call identity and unsupported hosted outputs never dispatch a tool
✔ Reject {"body":{"id":"resp_owned","status":"incomplete","output":[{"type":"function_call","id":"fc_partial" (1290.5845ms)
✔ Reject {"body":{"id":"resp_owned","status":"failed","output":[{"type":"function_call","id":"fc_failed","cal (1700.8777ms)
✔ Reject {"body":{"id":"resp_owned","status":"completed","output":[{"type":"function_call","id":"fc_bad","cal (1602.5777ms)
✔ Reject {"body":{"id":"resp_owned","status":"completed","output":[{"type":"computer_call","id":"not-supporte (1630.9884ms)
✔ Responses incomplete, failed, missing call identity and unsupported hosted outputs never dispatch a tool (6225.922ms)
▶ invalid JSON in a nonstream Chat or Responses tool group prevents every tool and pool replay
✔ chat-completions (1812.3813ms)
✔ responses (1808.9892ms)
✔ invalid JSON in a nonstream Chat or Responses tool group prevents every tool and pool replay (3621.9089ms)
✔ wire adapter preserves tool IDs but never passes Responses-private Items to Chat or another endpoint (0.2591ms)
✔ connection JSON import accepts explicit Responses protocol and continues to reject unknown protocols without reading client sessions (685.0721ms)
✔ reservation uses actual known usage from the same route, never lowers estimates or rewrites original receipts (1.7281ms)
✔ reservation verification binds original Task/Run consumption and rejects changed usage or foreign provenance (0.48ms)
✔ known high input calibrates the next request and stops before dispatch while preserving the completed read (1447.5822ms)
✔ one reported overrun stops legacy requests and keeps actual use and excess (1214.2904ms)
✔ one reported overrun stops pool requests and keeps actual use and excess (1782.9067ms)
✔ one reported overrun stops connection requests and keeps actual use and excess (1641.352ms)
✔ reported usage below the limit prevents an unaffordable legacy retry before another dispatch (1083.9795ms)
✔ reported usage below the limit prevents an unaffordable pool retry before another dispatch (1931.2061ms)
✔ native bridge seals calibrated refusal with original consumption and preserves versioned request verification (12627.0073ms)
✔ continuation reads advance through historical pages without a false pause while repeated current-Run pages still pause (1.8553ms)
✔ a new tool page replaces old latest-feedback space and still obeys its escaped protocol budget (0.6019ms)
✔ budgeted artifact pages count escaped protocol bytes and reconstruct original multilingual JSON without losing characters (9.6355ms)
✔ known artifact coverage detects semantic repeats despite changing limits, and advancing pages or direct evidence restore progress (0.3638ms)
✔ actual large artifact reads are present in the next bounded HTTP request and next_offset progresses without redoing tools (3561.2758ms)
✔ a real repeated artifact loop pauses the same Task after recorded results, survives reload and only resumes explicitly (3629.9503ms)
✔ actual HTTP continuation after reload sends distinct historically covered pages without false no_progress or replay (4839.0984ms)
✔ context APIs retain exact request identity, paginate archived groups and reject foreign sessions/runs (3409.9772ms)
✔ delivery reports verify request body, sealed plan and original archived bytes, and show corruption as needing verification (1692.5361ms)
✔ native seed framing reserves capacity before optional history without changing the selected window (664.3132ms)
✔ direct code evidence remains selected while recent historical-index browsing is archived within the complete request budget (1424.1889ms)
✔ one long user turn becomes bounded complete groups with exact UTF-8 history and recent JSON arguments (3589.1491ms)
✔ tool definitions, exact system bytes, output reserve and remaining Run/Task budget are accounted before requests (866.638ms)
✔ oversized required user instructions pause intact and can be explicitly resumed after increasing the window (1282.7167ms)
✔ all task instructions, constraints and acceptance remain in the runtime system instead of an eight-item or byte prefix (817.1144ms)
✔ a newest manageable multi-tool group stays exact even beyond the working target; oversized earlier groups are archived (698.5481ms)
✔ invalid orphan result is rejected before archive or network preparation (492.6358ms)
✔ invalid missing result is rejected before archive or network preparation (537.6192ms)
✔ invalid wrong result is rejected before archive or network preparation (484.0046ms)
✔ invalid duplicate call is rejected before archive or network preparation (570.5568ms)
✔ a group larger than the artifact limit uses verified Unicode-safe chunks without replacing tool JSON (1414.5826ms)
✔ changed or missing archived evidence stops reuse and is never silently recreated (960.8616ms)
✔ transaction failure prevents publishing a plan and any corresponding model request (797.6377ms)
✔ the Provider rejects body mutation after a sealed plan before calling HTTP (690.4846ms)
✔ HTTP failure and explicit same-Task resume reuse exact completed results and request hashes without repeating a command (7196.0068ms)
✔ a real process exit after plan persistence marks interruption and never issues an automatic model request (1120.1729ms)
✔ a stop from the event loop interrupts immediate model replies before later tools (1106.1643ms)
✔ a stop after the first tool preserves a complete interrupted multi-call protocol group (1398.4979ms)
✔ the compatible HTTP loop preserves call IDs, validates errors and waits before executing commands (4600.2004ms)
✔ denial prevents command execution and produces a paired denied result (1364.1836ms)
✔ stopping during approval interrupts pending calls without executing or replaying them (1018.3062ms)
✔ stopping a running fixture command terminates its own process tree and retains uncertain state (1522.848ms)
✔ command output decodes split UTF-8 and excludes the workbench key from child environment (1016.0014ms)
✔ model-list testing uses a tentative key without changing the saved connection and redacts service errors (6.8813ms)
✔ truncated or repeated model calls fail before execution; tool input validation rejects unknown capabilities (2093.9969ms)
✔ request projection keeps exact tool pairs and complete recent user instructions while preserving disk history (1535.6741ms)
✔ the latest oversized session result reaches the next request together with already read source modules (2051.5081ms)
✔ every call in a large latest group keeps its actual outcome, including denied and unknown results (1080.2312ms)
✔ a small complete latest tool group is sent once and needs no duplicate feedback view (865.6272ms)
✔ real HTTP repeated state inspection pauses after three identical engineering outcomes and preserves tool pairing (2529.9692ms)
✔ small code and actual failed command stay visible when their original multi-tool group cannot fit (1481.1618ms)
✔ mixed group retains six distinct small file observations with exact multilingual text and bounded request bytes (1718.8272ms)
✔ working observations are bound to the current Task and stop treating a pre-application file revision as current evidence (2250.6173ms)
✔ a sealed observation rejects changed or missing original result artifacts instead of recreating evidence (1370.3846ms)
✔ a real passed-check observation is excluded after a source edit invalidates the check, while its historical sent evidence remains valid (5238.7316ms)
✔ budgeted file pages reconstruct UTF-8 text including long single lines and surrogate boundaries (4.237ms)
✔ actual HTTP multi-tool reads survive an oversized runtime snapshot and the sealed evidence survives reopening (9468.6392ms)
✔ actual HTTP file pagination advances over a long file with each returned page visible in the next request (3985.3817ms)
✔ later mixed history pages cannot displace the actual module sources and complete requirements (5271.0972ms)
✔ a later narrow page of the same file revision cannot evict the wider actual observation (4200.7842ms)
✔ six small source files stay complete across metadata turns even when mandatory evidence exceeds the soft target (3110.5548ms)
✔ historical read-artifact navigation points to the original artifact and next page, never its result wrapper (1155.7987ms)
✔ a 1.6.1 full-provenance sent view remains verifiable after introducing the compact view (1261.4608ms)
✔ actual HTTP requests keep four observed modules after a later large history read and preserve all source bytes (9257.7856ms)
ℹ tests 155
ℹ suites 0
ℹ pass 155
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 241938.7563