Files
Agent-Workbench/docs/delivery-integrity-i-evidence/development/final-related.tap
T

196 lines
21 KiB
Plaintext
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
✔ a budget pause persists concrete text/tool/history/image breakdown and never dispatches an oversized request (3001.0286ms)
✔ explicit frozen preview resumes the same Task in a new Run, preserves completed tools and does not change global config or selected model (4072.4143ms)
✔ insufficient budgets, silently clamped capacity and arbitrary config fields reject without a physical request or mutating task budget (2659.9932ms)
✔ changed session, changed registry, expiry and altered frozen preview all require a new explicit preview (3275.6356ms)
✔ preview preserves unknown invocation status and increasing Task allowance is committed only with confirmed new Run (4488.2766ms)
▶ complete sealed batch hands off after all tools settle with no next model request
✔ complete (4667.7431ms)
✔ later-error (4690.8716ms)
✔ complete sealed batch hands off after all tools settle with no next model request (9359.3613ms)
▶ partial, changed, failed or cancelled batches cannot claim complete review handoff
✔ partial (4352.2104ms)
✔ new-proposal (4852.8286ms)
✔ failed (3125.9427ms)
✔ conflict (3448.6452ms)
✔ cancelled (3613.6131ms)
✔ unknown (3912.7689ms)
✔ tampered (3973.3072ms)
✔ partial, changed, failed or cancelled batches cannot claim complete review handoff (27280.5099ms)
✔ reviewed application invalidates old checks, names current SHA and advances through a new approved check to Git preview (8854.8468ms)
✔ skip-tests, foreign task, unknown operations, diagnostic and parent review retain their boundaries (3432.4847ms)
✔ actual process exit at Git queued never replays a commit and can reconcile the durable result (12340.2951ms)
✔ actual process exit at Git preparing never replays a commit and can reconcile the durable result (13252.0315ms)
✔ actual process exit at Git index_locked never replays a commit and can reconcile the durable result (15194.94ms)
✔ actual process exit at Git prepared never replays a commit and can reconcile the durable result (16850.3298ms)
✔ actual process exit at Git ref_prepared never replays a commit and can reconcile the durable result (15436.6024ms)
✔ actual process exit at Git publishing never replays a commit and can reconcile the durable result (15794.2493ms)
✔ actual process exit at Git committed never replays a commit and can reconcile the durable result (12604.0619ms)
✔ actual process exit at Git index_published never replays a commit and can reconcile the durable result (10236.2772ms)
✔ actual process exit at Git finished never replays a commit and can reconcile the durable result (9771.1846ms)
✔ an unknown Git result cannot overwrite an externally changed index or delete an unrelated lock (8156.435ms)
✔ known publication with a later source edit retains the commit and marks the delivery for inspection (14070.3587ms)
✔ CRLF normalization, multiple applied revisions and unrelated special index flags preserve exact bytes and ownership (10463.456ms)
✔ adding new durable record kinds to an existing hot Store preserves all existing Maps and records (9695.9554ms)
✔ an unrelated owner deletion is fingerprinted and preserved while linked task paths are rejected (15113.4876ms)
✔ actual failed tests become a reviewed local commit and evidence report while preserving unrelated index and files (14304.8722ms)
✔ selected preexisting unstaged, staged and untracked owner edits cannot enter a task commit (17483.0517ms)
Switched to a new branch 'owner-branch'
✔ source, unrelated stage and branch changes after review invalidate a commit plan without changing HEAD (23298.2751ms)
Switched to a new branch 'delivery-branch'
✔ a later Git metadata change keeps matching source evidence valid but a real source edit requires checks (11192.2448ms)
✔ a corrupt passed-check log and a pending proposal block Git delivery instead of inventing evidence (7081.4325ms)
HEAD is now at 98156f8 Sealed delivery fixture baseline
✔ Git filters, merge state, detached HEAD and old task ownership are explicitly rejected (23950.1283ms)
✔ local snapshot commits disable repository hooks and signing and isolate the Git child environment (8330.7539ms)
✔ stopping before publication cancels safely; stopping after publication preserves the actual completed commit (16715.9498ms)
✔ delivery API confirms exactly one owned plan, serves verified reports and preserves results across restart (16662.5867ms)
✔ the model can prepare an owned Git review and evidence report while publishing remains a separate user action (10488.5549ms)
✔ HTTP Git review pairs deferred calls, survives reload and never requests more work until the saved preview is resolved (12340.657ms)
✔ a reviewed Git preview can be revised or withdrawn without publishing or losing the current Task (10162.1815ms)
✔ sealed r1-off failure fits its original 16K config with authentic latest status and readable source (15.1515ms)
✔ sealed r1-on failure fits its original 16K config with authentic latest status and readable source (11.8943ms)
✔ sealed r2-on failure fits its original 16K config with authentic latest status and readable source (9.5296ms)
✔ sealed r2-off failure fits its original 16K config with authentic latest status and readable source (9.5289ms)
✔ archived native call pairs require exact saved arguments/results rather than model-written summaries (8.7536ms)
✔ full system/user requirements and mandatory current failure can refuse rather than be trimmed (12.9348ms)
✔ small native input is byte-preserving; developer instructions remain mandatory after compaction (0.1827ms)
✔ older failing checks and pending proposals remain visible after a later large file read (14.9322ms)
✔ bounded result keeps every check status and cuts Unicode only on whole characters (0.4835ms)
✔ controlled engine HTTP freezes one child, pages original owned input/report, rejects foreign reads and prevents changed-source reuse (18006.7623ms)
✔ a sealed whole-batch review pauses the owned native session without certifying engine completion or losing the saved host result (17453.7748ms)
✔ a changed review seal still closes the owned native process and cannot claim a verified review handoff (15537.5997ms)
✔ native text and call IDs preserve original result pairing while unsupported content fails closed (0.4987ms)
✔ one engine child freezes real installation identity before creating tasks and rejects a changed source (14536.3288ms)
✔ owned native tools return saved results, use real approvals/checks and account each actual request once across new engine Runs (29634.2257ms)
✔ forged native scope cannot change a parent file and a final completion claim cannot replace worker proof (15734.0349ms)
✔ duplicate native IDs are rejected without replaying a saved tool (18235.2364ms)
✔ native input exceeding the context budget is retained and sends no model request (19144.3169ms)
✔ bounded native sends verify original tool provenance and reject forged views or missing versions (22219.0718ms)
✔ stop retains a late reported model usage once and executes no tool from that output (17657.4062ms)
✔ a source change after the original model intent stops before native tools execute (14045.6272ms)
✔ an engine source change while approval is pending invalidates the old permission before any check starts (17119.9651ms)
✔ missing or noncompleted turn end cannot be adopted even with an owned receipt and assistant text (0.5355ms)
✔ closed root with an original live descendant remains unknown and retains its process tree (0.2802ms)
✔ host recovery reserves original pending attempts exactly once and explicit review preserves unknown outcomes (17678.2513ms)
✔ corrupted model reservation cannot reduce recovered budget or permit a new engine request (13397.7164ms)
✔ model registry keeps original credentials/endpoint intact, rejects stale edits, and forgets only volatile keys on restart (983.4723ms)
✔ config import previews exact revisions, keeps secrets volatile, syncs only when enabled and preserves manual edits on conflicts (627.8694ms)
✔ real HTTP pool fails over 429 to another model and never repeats completed tools; attempts seal actual request bytes and Run identity (3398.8165ms)
✔ pool rotation and weights select members on new Runs; disabled and missing credentials are excluded without requests (3771.3019ms)
✔ 401 blocks the credential revision; updating the key restores availability without leaking error echoes (1555.8345ms)
✔ queued request cancellation and in-flight disable release only owned leases and do not dispatch another HTTP request (888.4885ms)
✔ compiled model-center HTTP API persists public profiles, blocks forged origins, and local operations never invoke a model (2758.2535ms)
✔ model registry disk failures restore routing, pool profiles and credential references without partial updates (620.6265ms)
✔ model selection and saved configuration commit together and failures preserve the original mode and endpoint (893.0357ms)
✔ configuration imports roll back the entire batch and retain one usable preview after validation or commit failure (592.1125ms)
✔ HTTP quota retains native code/type/param/request id, bounded original and allowlisted headers, never repeats that connection (1542.1167ms)
✔ Retry-After above 90 seconds remains exact and a short local budget cannot trigger an early retry (3.1421ms)
✔ Retry-After date, expired date, huge value and invalid header retain explicit semantics (1.3998ms)
✔ unclassified 429 remains unspecified and every bounded actual retry retains its own original error (1020.4978ms)
✔ oversized HTTP error saves exactly the bounded prefix and labels its incomplete range (1253.7188ms)
✔ Chat SSE error preserves upstream fields, original failed event, partial tool and reported usage without executing or retrying (1239.0601ms)
✔ Responses failed SSE retains original event/response identity and usage (1404.2791ms)
✔ Responses incomplete SSE retains original event/response identity and usage (1505.4573ms)
✔ unsupported successful Responses output retains original JSON and usage with a local protocol origin (1407.2557ms)
✔ HTTP 200 native error is still an upstream failure rather than a fabricated successful answer (0.8289ms)
✔ JSON-escaped credential echoes are redacted in the original preview and capture as well as parsed fields (0.6968ms)
✔ quota failover uses another independent pool member and never replays a failed stream (1678.212ms)
✔ native parameters are isolated by pool member, preserve Responses state and bind the original output reservation (0.2808ms)
✔ unsupported native fields and invalid values reject before dispatch instead of silently dropping or overriding constraints (0.284ms)
✔ native settings for legacy and registered profiles persist without changing credentials or old records and reject stale edits (808.2526ms)
✔ compiled native settings API saves/restores exact parameters and rejects reserved fields without mutating the profile (2610.0683ms)
✔ legacy connection test sends its saved native fields and selected output limit field without changing the configuration (2682.7254ms)
✔ identity distinguishes missing, invalid, mixed, exact and explicitly declared aliases without guessing (1.4244ms)
✔ chat-completions JSON rejects a different model before tool dispatch, retaining source and reported usage (1447.2741ms)
✔ chat-completions SSE rejects a different model before tool dispatch, retaining source and reported usage (1779.4688ms)
✔ responses JSON rejects a different model before tool dispatch, retaining source and reported usage (2010.1696ms)
✔ responses SSE rejects a different model before tool dispatch, retaining source and reported usage (1215.29ms)
✔ record policy accepts missing identity but keeps it explicit, and strict declared mapping is frozen in the consumption (1.1463ms)
✔ mixed streamed identities cannot pass strict mode, and interrupted streams retain observed identifiers without authenticating them (1.2475ms)
✔ identity refusal never fails over to another pool member and raw response model cannot leak known credentials (1145.6244ms)
✔ strict refusal remains visible even when the same original response exhausts the budget (1307.9103ms)
✔ strict identity crosses the controlled engine bridge with its original source and zero tool dispatch (12712.8417ms)
✔ frozen direct connection policy and delivery reports preserve response identity without filling historical gaps (2667.3305ms)
✔ saved engine budgets verify all Chat/Responses route variants and reject a mismatched output allowance (910.3778ms)
✔ owned CC Switch SQLite imports the selected TOML Responses wire_api and leaves unknown clients untouched (1313.7849ms)
✔ Responses executes one original tool through Runtime, carries encrypted Items, reports raw usage and seals exact wire bytes (2658.3653ms)
✔ mixed-protocol pool keeps both actual variants and changes route on 429 without re-executing completed tools (2639.4948ms)
▶ Responses incomplete, failed, missing call identity and unsupported hosted outputs never dispatch a tool
✔ Reject {"body":{"id":"resp_owned","status":"incomplete","output":[{"type":"function_call","id":"fc_partial" (1439.6259ms)
✔ Reject {"body":{"id":"resp_owned","status":"failed","output":[{"type":"function_call","id":"fc_failed","cal (1947.6016ms)
✔ Reject {"body":{"id":"resp_owned","status":"completed","output":[{"type":"function_call","id":"fc_bad","cal (2095.986ms)
✔ Reject {"body":{"id":"resp_owned","status":"completed","output":[{"type":"computer_call","id":"not-supporte (2156.9239ms)
✔ Responses incomplete, failed, missing call identity and unsupported hosted outputs never dispatch a tool (7641.1151ms)
▶ invalid JSON in a nonstream Chat or Responses tool group prevents every tool and pool replay
✔ chat-completions (1934.2424ms)
✔ responses (2392.7998ms)
✔ invalid JSON in a nonstream Chat or Responses tool group prevents every tool and pool replay (4327.4206ms)
✔ wire adapter preserves tool IDs but never passes Responses-private Items to Chat or another endpoint (0.1859ms)
✔ connection JSON import accepts explicit Responses protocol and continues to reject unknown protocols without reading client sessions (842.4279ms)
✔ reservation uses actual known usage from the same route, never lowers estimates or rewrites original receipts (2.1389ms)
✔ reservation verification binds original Task/Run consumption and rejects changed usage or foreign provenance (0.4803ms)
✔ known high input calibrates the next request and stops before dispatch while preserving the completed read (1935.5015ms)
✔ one reported overrun stops legacy requests and keeps actual use and excess (1663.5722ms)
✔ one reported overrun stops pool requests and keeps actual use and excess (1408.1744ms)
✔ one reported overrun stops connection requests and keeps actual use and excess (1443.6653ms)
✔ reported usage below the limit prevents an unaffordable legacy retry before another dispatch (1065.855ms)
✔ reported usage below the limit prevents an unaffordable pool retry before another dispatch (1602.5959ms)
✔ native bridge seals calibrated refusal with original consumption and preserves versioned request verification (19980.0413ms)
✔ capability routing skips text-only members before acquiring a lease and records exact refusal in the original Run (1442.9189ms)
✔ all incompatible members reject with no network request, no lease, preserved Task and explicit non-dispatched ledger (1217.8025ms)
✔ explicit synthetic tool probe verifies only actual matching call, never executes it and loses verification on revision changes (1020.4743ms)
✔ plain text probe cannot verify tools; wrong tool nonce and client supplied evidence cannot claim verified support (1209.52ms)
✔ a disabled connection invalidates an in-flight probe even when toggle mutates its original registry object (773.0656ms)
✔ HTTP download preserves original malformed SSE bytes and artifact SHA instead of decoding invalid UTF8 (2225.1124ms)
✔ terminated SSE adapts empty optional tool arguments, retains the exact wire bytes and executes the validated batch once (2383.9ms)
✔ reported SSE usage survives rejected required arguments "" without dispatching any tool (1343.4389ms)
✔ reported SSE usage survives rejected required arguments "{\"path\":" without dispatching any tool (1274.4042ms)
✔ reported SSE usage survives rejected required arguments "null" without dispatching any tool (1271.7899ms)
✔ a stream without DONE retains reported usage but cannot adapt or execute even an optional tool (1301.6774ms)
✔ a timeout after reasoning-only fragments creates one unknown receipt and never invents reported tokens (1614.1801ms)
✔ real Chat SSE carries UTF8 and split tool JSON; Runtime executes only after DONE, stores partial progress and original usage (2302.2097ms)
✔ Responses typed SSE validates original item identity, terminal data, raw usage and tool result pair (2711.1435ms)
✔ incomplete network stream retains unknown attempt and never automatically switches pool or dispatches fragment tools (1764.2068ms)
✔ stopping an active HTTP stream closes only original request and keeps partial text without executing tools (1290.246ms)
✔ explicit streaming probe uses actual SSE despite unknown declaration and binds its own original attempt (723.2799ms)
✔ coalesced initial SSE events publish the latest partial state even when the upstream stays idle (2031.7889ms)
✔ stream timeout closes the original HTTP response, preserves redacted capture and never retries (932.0622ms)
✔ stream idle timeout preserves incomplete tool arguments, records unknown use and never fails over (1082.8753ms)
✔ SSE accepts CR-only final event boundaries and retains exact invalid UTF8 bytes without dispatch (0.8259ms)
▶ stream parser rejects truncation, finish length, malformed JSON, duplicate sequence, changed final item and oversized data
✔ reject 0 (0.2273ms)
✔ reject 1 (0.1735ms)
✔ reject 2 (0.2001ms)
✔ reject 3 (4.9227ms)
✔ reject 4 (0.1995ms)
✔ reject 5 (0.1851ms)
✔ stream parser rejects truncation, finish length, malformed JSON, duplicate sequence, changed final item and oversized data (6.4788ms)
✔ unchanged proposals cannot hide repeated original-log reads or identical session inspections (4587.472ms)
✔ reusing a verified proof never resets identical-state detection or creates a new proof (4212.8365ms)
✔ coverage merges adjacent pages, separates changed sources and never claims old Run coverage in a continuation (0.8891ms)
✔ continuation reads advance through historical pages without a false pause while repeated current-Run pages still pause (1.78ms)
✔ a new tool page replaces old latest-feedback space and still obeys its escaped protocol budget (0.445ms)
✔ budgeted artifact pages count escaped protocol bytes and reconstruct original multilingual JSON without losing characters (11.3555ms)
✔ known artifact coverage detects semantic repeats despite changing limits, and advancing pages or direct evidence restore progress (0.4221ms)
✔ actual large artifact reads are present in the next bounded HTTP request and next_offset progresses without redoing tools (4129.1261ms)
✔ a real repeated artifact loop pauses the same Task after recorded results, survives reload and only resumes explicitly (4449.2262ms)
✔ actual HTTP continuation after reload sends distinct historically covered pages without false no_progress or replay (3932.1187ms)
✔ a stop from the event loop interrupts immediate model replies before later tools (944.7483ms)
✔ a stop after the first tool preserves a complete interrupted multi-call protocol group (1461.5199ms)
✔ the compatible HTTP loop preserves call IDs, validates errors and waits before executing commands (4793.6285ms)
✔ denial prevents command execution and produces a paired denied result (1124.9174ms)
✔ stopping during approval interrupts pending calls without executing or replaying them (1078.4253ms)
✔ stopping a running fixture command terminates its own process tree and retains uncertain state (1580.7365ms)
✔ command output decodes split UTF-8 and excludes the workbench key from child environment (1208.0752ms)
✔ model-list testing uses a tentative key without changing the saved connection and redacts service errors (6.5434ms)
✔ truncated or repeated model calls fail before execution; tool input validation rejects unknown capabilities (1826.3279ms)
✔ request projection keeps exact tool pairs and complete recent user instructions while preserving disk history (927.6593ms)
ℹ tests 179
ℹ suites 0
ℹ pass 179
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 441832.9689