Files
Agent-Workbench/docs/controlled-engine-final-related.tap

491 lines
22 KiB
Plaintext

TAP version 13
# Subtest: controlled installation pins the native kernel and bridge independently of the existing connection-probe identity
ok 1 - controlled installation pins the native kernel and bridge independently of the existing connection-probe identity
---
duration_ms: 1151.9707
type: 'test'
...
# Subtest: controlled profiles disable every native project/terminal provider and native model/retry route using the actual include patch contract
ok 2 - controlled profiles disable every native project/terminal provider and native model/retry route using the actual include patch contract
---
duration_ms: 3.3788
type: 'test'
...
# Subtest: owned tool catalogues seal full schemas canonically and reject duplicate or malformed declarations
ok 3 - owned tool catalogues seal full schemas canonically and reject duplicate or malformed declarations
---
duration_ms: 9.0229
type: 'test'
...
# Subtest: bridge authentication, current Run, process, SDK session and browser-origin checks reject before dispatch
ok 4 - bridge authentication, current Run, process, SDK session and browser-origin checks reject before dispatch
---
duration_ms: 721.6352
type: 'test'
...
# Subtest: bridge duplicates, unknown tools, malformed arguments and source changes are refused before a second side effect
ok 5 - bridge duplicates, unknown tools, malformed arguments and source changes are refused before a second side effect
---
duration_ms: 724.1132
type: 'test'
...
# Subtest: bridge model requests bind the exact catalogue, route and output bound and preserve raw request/result identity
ok 6 - bridge model requests bind the exact catalogue, route and output bound and preserve raw request/result identity
---
duration_ms: 639.9443
type: 'test'
...
# Subtest: stopped and oversized bridge requests never reach a local model or tool callback
ok 7 - stopped and oversized bridge requests never reach a local model or tool callback
---
duration_ms: 619.7513
type: 'test'
...
# Subtest: controlled engine HTTP freezes one child, pages original owned input/report, rejects foreign reads and prevents changed-source reuse
ok 8 - controlled engine HTTP freezes one child, pages original owned input/report, rejects foreign reads and prevents changed-source reuse
---
duration_ms: 12684.1
type: 'test'
...
# Subtest: native text and call IDs preserve original result pairing while unsupported content fails closed
ok 9 - native text and call IDs preserve original result pairing while unsupported content fails closed
---
duration_ms: 0.7982
type: 'test'
...
# Subtest: one engine child freezes real installation identity before creating tasks and rejects a changed source
ok 10 - one engine child freezes real installation identity before creating tasks and rejects a changed source
---
duration_ms: 6166.9441
type: 'test'
...
# Subtest: owned native tools return saved results, use real approvals/checks and account each actual request once across new engine Runs
ok 11 - owned native tools return saved results, use real approvals/checks and account each actual request once across new engine Runs
---
duration_ms: 20342.7705
type: 'test'
...
# Subtest: forged native scope cannot change a parent file and a final completion claim cannot replace worker proof
ok 12 - forged native scope cannot change a parent file and a final completion claim cannot replace worker proof
---
duration_ms: 10503.1743
type: 'test'
...
# Subtest: duplicate native IDs are rejected without replaying a saved tool
ok 13 - duplicate native IDs are rejected without replaying a saved tool
---
duration_ms: 9769.3062
type: 'test'
...
# Subtest: native input exceeding the context budget is retained and sends no model request
ok 14 - native input exceeding the context budget is retained and sends no model request
---
duration_ms: 8368.3851
type: 'test'
...
# Subtest: stop retains a late reported model usage once and executes no tool from that output
ok 15 - stop retains a late reported model usage once and executes no tool from that output
---
duration_ms: 8832.5885
type: 'test'
...
# Subtest: a source change after the original model intent stops before native tools execute
ok 16 - a source change after the original model intent stops before native tools execute
---
duration_ms: 8770.6273
type: 'test'
...
# Subtest: an engine source change while approval is pending invalidates the old permission before any check starts
ok 17 - an engine source change while approval is pending invalidates the old permission before any check starts
---
duration_ms: 12517.1317
type: 'test'
...
# Subtest: missing or noncompleted turn end cannot be adopted even with an owned receipt and assistant text
ok 18 - missing or noncompleted turn end cannot be adopted even with an owned receipt and assistant text
---
duration_ms: 0.6144
type: 'test'
...
# Subtest: closed root with an original live descendant remains unknown and retains its process tree
ok 19 - closed root with an original live descendant remains unknown and retains its process tree
---
duration_ms: 0.265
type: 'test'
...
# Subtest: host recovery reserves original pending attempts exactly once and explicit review preserves unknown outcomes
ok 20 - host recovery reserves original pending attempts exactly once and explicit review preserves unknown outcomes
---
duration_ms: 11895.6241
type: 'test'
...
# Subtest: corrupted model reservation cannot reduce recovered budget or permit a new engine request
ok 21 - corrupted model reservation cannot reduce recovered budget or permit a new engine request
---
duration_ms: 11084.0353
type: 'test'
...
# Subtest: two Purge source versions lock nine original inputs without executing JavaScript or postinstall and remain disabled by default
ok 22 - two Purge source versions lock nine original inputs without executing JavaScript or postinstall and remain disabled by default
---
duration_ms: 590.0825
type: 'test'
...
# Subtest: incomplete, unsupported, relative and oversized sources cannot enable the experiment
ok 23 - incomplete, unsupported, relative and oversized sources cannot enable the experiment
---
duration_ms: 566.0959
type: 'test'
...
# Subtest: invalid source fingerprints and invented privilege or retry fields fail before any parent Task is created
ok 24 - invalid source fingerprints and invented privilege or retry fields fail before any parent Task is created
---
duration_ms: 1797.8737
type: 'test'
...
# Subtest: source edits between detection and plan are rejected without creating a parent Task
ok 25 - source edits between detection and plan are rejected without creating a parent Task
---
duration_ms: 1664.9114
type: 'test'
...
# Subtest: source edits after sealing cannot launch a child and do not reset its cumulative budget
ok 26 - source edits after sealing cannot launch a child and do not reset its cumulative budget
---
duration_ms: 6790.4646
type: 'test'
...
# Subtest: literal experiment text is sealed into the parent, worker and each new Run while actual host approvals and saved results remain required
ok 27 - literal experiment text is sealed into the parent, worker and each new Run while actual host approvals and saved results remain required
---
duration_ms: 20563.1615
type: 'test'
...
# Subtest: source changes after a model response stop before any native tool is accepted, retaining the actual usage once
ok 28 - source changes after a model response stop before any native tool is accepted, retaining the actual usage once
---
duration_ms: 8754.3529
type: 'test'
...
# Subtest: source changes while approval is pending invalidate the old permission before an engineering check starts
ok 29 - source changes while approval is pending invalidate the old permission before an engineering check starts
---
duration_ms: 11209.6943
type: 'test'
...
# Subtest: altered text, interpolation, section order and policy flags cannot reuse a sealed experiment
ok 30 - altered text, interpolation, section order and policy flags cannot reuse a sealed experiment
---
duration_ms: 1936.8173
type: 'test'
...
# Subtest: the observed SDK system-message layout and the explicit system field both require exactly one effective section, never a user or tool copy
ok 31 - the observed SDK system-message layout and the explicit system field both require exactly one effective section, never a user or tool copy
---
duration_ms: 2074.857
type: 'test'
...
# Subtest: a missing or duplicated experimental section in the actual input is rejected before any model dispatch
ok 32 - a missing or duplicated experimental section in the actual input is rejected before any model dispatch
---
duration_ms: 17380.2444
type: 'test'
...
# Subtest: experimental prompt overhead still participates in the original complete-input budget guard
ok 33 - experimental prompt overhead still participates in the original complete-input budget guard
---
duration_ms: 7880.6256
type: 'test'
...
# Subtest: source detection HTTP remains local-authenticated and read-only with no configured model requests
ok 34 - source detection HTTP remains local-authenticated and read-only with no configured model requests
---
duration_ms: 1674.4081
type: 'test'
...
# Subtest: a live SQLite host with the actual 1.9 artifact object upgrades, keeps its original blobs and verifies real browser PNG bytes
ok 35 - a live SQLite host with the actual 1.9 artifact object upgrades, keeps its original blobs and verifies real browser PNG bytes
---
duration_ms: 3990.0537
type: 'test'
...
# Subtest: failed upgrade restores the original artifact object, provider and request handler
ok 36 - failed upgrade restores the original artifact object, provider and request handler
---
duration_ms: 650.2843
type: 'test'
...
# Subtest: claiming the same runtime owner twice preserves its original metadata and every database row
ok 37 - claiming the same runtime owner twice preserves its original metadata and every database row
---
duration_ms: 786.1772
type: 'test'
...
# Subtest: sessions without controlled-engine history retain their prior export shape and never expose another session record
ok 38 - sessions without controlled-engine history retain their prior export shape and never expose another session record
---
duration_ms: 885.3583
type: 'test'
...
# Subtest: live workers, controlled drivers, or MCP operations refuse an upgrade before changing handlers, managers or database rows
ok 39 - live workers, controlled drivers, or MCP operations refuse an upgrade before changing handlers, managers or database rows
---
duration_ms: 719.9001
type: 'test'
...
# Subtest: an idle upgrade loads the newly introduced engine record map, resets obsolete managers and preserves original rows, sessions and provider objects
ok 40 - an idle upgrade loads the newly introduced engine record map, resets obsolete managers and preserves original rows, sessions and provider objects
---
duration_ms: 715.3545
type: 'test'
...
# Subtest: an installation failure restores old managers, original record maps, handlers and all existing database rows
ok 41 - an installation failure restores old managers, original record maps, handlers and all existing database rows
---
duration_ms: 608.5167
type: 'test'
...
# Subtest: the owned controlled driver remains an active session and project even after outer task activity has ended
ok 42 - the owned controlled driver remains an active session and project even after outer task activity has ended
---
duration_ms: 718.2548
type: 'test'
...
# Subtest: release and upgrade help or invalid arguments have no file or network setup side effects in an independent directory
ok 43 - release and upgrade help or invalid arguments have no file or network setup side effects in an independent directory
---
duration_ms: 778.1488
type: 'test'
...
# Subtest: hot upgrade refreshes retained model and importer methods while preserving credential and routing objects
ok 44 - hot upgrade refreshes retained model and importer methods while preserving credential and routing objects
---
duration_ms: 720.2096
type: 'test'
...
# Subtest: hot upgrade refuses active probes and restores retained model methods if installation fails
ok 45 - hot upgrade refuses active probes and restores retained model methods if installation fails
---
duration_ms: 607.1572
type: 'test'
...
# Subtest: a directly started compiled host upgrades twice with the same PID, sessions, applied bytes and volatile provider
ok 46 - a directly started compiled host upgrades twice with the same PID, sessions, applied bytes and volatile provider
---
duration_ms: 2297.5297
type: 'test'
...
# Subtest: the running 0.1.0 process upgrades in place and retains its project, session, files and volatile model connection
ok 47 - the running 0.1.0 process upgrades in place and retains its project, session, files and volatile model connection
---
duration_ms: 7872.0945
type: 'test'
...
# Subtest: HTTP worker identities require a current sealed plan, reject overrides and close unused plans without touching source
ok 48 - HTTP worker identities require a current sealed plan, reject overrides and close unused plans without touching source
---
duration_ms: 12890.5654
type: 'test'
...
# Subtest: HTTP unknown recovery only accepts the original evidence SHA and preserves changed source for explicit retention
ok 49 - HTTP unknown recovery only accepts the original evidence SHA and preserves changed source for explicit retention
---
duration_ms: 14937.169
type: 'test'
...
# Subtest: verified isolated outputs form a new parent batch, rerun real checks and commit only reviewed source
ok 50 - verified isolated outputs form a new parent batch, rerun real checks and commit only reviewed source
---
duration_ms: 39646.6209
type: 'test'
...
# Subtest: overlapping worker paths are explicit conflicts even if each independent task checks successfully
ok 51 - overlapping worker paths are explicit conflicts even if each independent task checks successfully
---
duration_ms: 25921.6378
type: 'test'
...
# Subtest: source edits after isolated work preserve the owner file and invalidate collection without proposing a merge
ok 52 - source edits after isolated work preserve the owner file and invalidate collection without proposing a merge
---
duration_ms: 24966.3418
type: 'test'
...
# Subtest: changed child bytes after parent preview block the entire batch before any parent write
ok 53 - changed child bytes after parent preview block the entire batch before any parent write
---
duration_ms: 32278.4325
type: 'test'
...
# Subtest: corrupt original check logs cannot become collected outputs or a parent review
ok 54 - corrupt original check logs cannot become collected outputs or a parent review
---
duration_ms: 23980.1217
type: 'test'
...
# Subtest: stopping a parent accounts late responses and executes none of their proposed tools
ok 55 - stopping a parent accounts late responses and executes none of their proposed tools
---
duration_ms: 11578.974
type: 'test'
...
# Subtest: unreported interrupted requests reserve estimated usage and the total deadline cannot reset on resume
ok 56 - unreported interrupted requests reserve estimated usage and the total deadline cannot reset on resume
---
duration_ms: 11701.1139
type: 'test'
...
# Subtest: a response exceeding the cumulative token allocation stops before registering or executing its tools
ok 57 - a response exceeding the cumulative token allocation stops before registering or executing its tools
---
duration_ms: 11891.4924
type: 'test'
...
# Subtest: cumulative tool allocation rejects the entire returned batch before any tool side effect
ok 58 - cumulative tool allocation rejects the entire returned batch before any tool side effect
---
duration_ms: 11005.762
type: 'test'
...
# Subtest: single response 16 tool ceiling rejects the entire returned batch before any tool side effect
ok 59 - single response 16 tool ceiling rejects the entire returned batch before any tool side effect
---
duration_ms: 11839.3924
type: 'test'
...
# Subtest: consumed request steps remain consumed across explicit worker resumes
ok 60 - consumed request steps remain consumed across explicit worker resumes
---
duration_ms: 13171.7007
type: 'test'
...
# Subtest: a saved completed response with missing usage is conservatively accounted exactly once during recovery
ok 61 - a saved completed response with missing usage is conservatively accounted exactly once during recovery
---
duration_ms: 9790.602
type: 'test'
...
# Subtest: an unexecuted second workspace plan resumes its original identity without recreating the first checkout
ok 62 - an unexecuted second workspace plan resumes its original identity without recreating the first checkout
---
duration_ms: 11411.6379
type: 'test'
...
# Subtest: a verified interrupted parent proposal resumes sealing without duplicating or replacing its original change
ok 63 - a verified interrupted parent proposal resumes sealing without duplicating or replacing its original change
---
duration_ms: 35390.0937
type: 'test'
...
# Subtest: recovery attaches a sealed parent batch saved before its group reference without creating another batch
ok 64 - recovery attaches a sealed parent batch saved before its group reference without creating another batch
---
duration_ms: 31619.1511
type: 'test'
...
# Subtest: explicitly retaining an unexecuted parent merge cancels its batch and frees the next parent goal
ok 65 - explicitly retaining an unexecuted parent merge cancels its batch and frees the next parent goal
---
duration_ms: 34596.5256
type: 'test'
...
# Subtest: unknown collaboration blocks ordinary proposals but permits only original batch recovery and rollback
ok 66 - unknown collaboration blocks ordinary proposals but permits only original batch recovery and rollback
---
duration_ms: 31390.4102
type: 'test'
...
# Subtest: after a restart the batch application lazily reinstates the worker result guard before any parent write
ok 67 - after a restart the batch application lazily reinstates the worker result guard before any parent write
---
duration_ms: 27310.6374
type: 'test'
...
# Subtest: unknown original tool inputs must match before acknowledgement, and reviewed checkpoint IDs preserve the unknown fact
ok 68 - unknown original tool inputs must match before acknowledgement, and reviewed checkpoint IDs preserve the unknown fact
---
duration_ms: 11003.5207
type: 'test'
...
# Subtest: a parent stop observes and waits for a manually approved child check and its real descendant process
ok 69 - a parent stop observes and waits for a manually approved child check and its real descendant process
---
duration_ms: 10319.0931
type: 'test'
...
# Subtest: a real host crash preserves unknown checks, refuses a still living owned PID and never replays a model or tool
ok 70 - a real host crash preserves unknown checks, refuses a still living owned PID and never replays a model or tool
---
duration_ms: 18487.4985
type: 'test'
...
# Subtest: a persisted child initialization resumes the original control run after a failure before its worker record
ok 71 - a persisted child initialization resumes the original control run after a failure before its worker record
---
duration_ms: 9501.9025
type: 'test'
...
# Subtest: failed request intent persistence never dispatches a worker request or changes source
ok 72 - failed request intent persistence never dispatches a worker request or changes source
---
duration_ms: 10081.7626
type: 'test'
...
# Subtest: failed usage persistence rolls back the uncommitted usage and conservatively reserves one unknown request without tools
ok 73 - failed usage persistence rolls back the uncommitted usage and conservatively reserves one unknown request without tools
---
duration_ms: 10433.9083
type: 'test'
...
# Subtest: a child file change after observation prevents acknowledgement even when the tool inputs did not change
ok 74 - a child file change after observation prevents acknowledgement even when the tool inputs did not change
---
duration_ms: 9608.3476
type: 'test'
...
# Subtest: worker scopes and allocated budgets reject aliases, extra permissions and aggregate overflow
ok 75 - worker scopes and allocated budgets reject aliases, extra permissions and aggregate overflow
---
duration_ms: 484.9588
type: 'test'
...
# Subtest: two sequential Git workspaces freeze a new parent goal, separate owned tasks and preserve all source bytes
ok 76 - two sequential Git workspaces freeze a new parent goal, separate owned tasks and preserve all source bytes
---
duration_ms: 9834.5953
type: 'test'
...
# Subtest: real isolated changes, approvals and npm checks produce original owned outputs, not model completion claims
ok 77 - real isolated changes, approvals and npm checks produce original owned outputs, not model completion claims
---
duration_ms: 25746.6165
type: 'test'
...
# Subtest: forged worker tools cannot execute terminal or Git even when absent from the advertised tool list
ok 78 - forged worker tools cannot execute terminal or Git even when absent from the advertised tool list
---
duration_ms: 10457.6447
type: 'test'
...
# Subtest: a completion-only model pauses each worker without fabricated changes or checks
ok 79 - a completion-only model pauses each worker without fabricated changes or checks
---
duration_ms: 12938.9405
type: 'test'
...
# Subtest: restart retains frozen scope and marks interrupted parent and workers unknown without sending requests
ok 80 - restart retains frozen scope and marks interrupted parent and workers unknown without sending requests
---
duration_ms: 10242.9614
type: 'test'
...
1..80
# tests 80
# suites 0
# pass 80
# fail 0
# cancelled 0
# skipped 0
# todo 0
# duration_ms 472534.7525