Files
Agent-Workbench/docs/embedded-browser-b-rejected-candidate-evidence/source/tests-04.tap
T

503 lines
21 KiB
Plaintext

TAP version 13
# Subtest: controlled engine HTTP freezes one child, pages original owned input/report, rejects foreign reads and prevents changed-source reuse
ok 1 - controlled engine HTTP freezes one child, pages original owned input/report, rejects foreign reads and prevents changed-source reuse
---
duration_ms: 17895.7345
type: 'test'
...
# Subtest: native text and call IDs preserve original result pairing while unsupported content fails closed
ok 2 - native text and call IDs preserve original result pairing while unsupported content fails closed
---
duration_ms: 0.7635
type: 'test'
...
# Subtest: one engine child freezes real installation identity before creating tasks and rejects a changed source
ok 3 - one engine child freezes real installation identity before creating tasks and rejects a changed source
---
duration_ms: 12587.6709
type: 'test'
...
# Subtest: owned native tools return saved results, use real approvals/checks and account each actual request once across new engine Runs
ok 4 - owned native tools return saved results, use real approvals/checks and account each actual request once across new engine Runs
---
duration_ms: 27725.6432
type: 'test'
...
# Subtest: forged native scope cannot change a parent file and a final completion claim cannot replace worker proof
ok 5 - forged native scope cannot change a parent file and a final completion claim cannot replace worker proof
---
duration_ms: 17185.0896
type: 'test'
...
# Subtest: duplicate native IDs are rejected without replaying a saved tool
ok 6 - duplicate native IDs are rejected without replaying a saved tool
---
duration_ms: 15911.7216
type: 'test'
...
# Subtest: native input exceeding the context budget is retained and sends no model request
ok 7 - native input exceeding the context budget is retained and sends no model request
---
duration_ms: 14338.0909
type: 'test'
...
# Subtest: stop retains a late reported model usage once and executes no tool from that output
ok 8 - stop retains a late reported model usage once and executes no tool from that output
---
duration_ms: 14530.0368
type: 'test'
...
# Subtest: a source change after the original model intent stops before native tools execute
ok 9 - a source change after the original model intent stops before native tools execute
---
duration_ms: 14248.3302
type: 'test'
...
# Subtest: an engine source change while approval is pending invalidates the old permission before any check starts
ok 10 - an engine source change while approval is pending invalidates the old permission before any check starts
---
duration_ms: 17717.8992
type: 'test'
...
# Subtest: missing or noncompleted turn end cannot be adopted even with an owned receipt and assistant text
ok 11 - missing or noncompleted turn end cannot be adopted even with an owned receipt and assistant text
---
duration_ms: 0.5168
type: 'test'
...
# Subtest: closed root with an original live descendant remains unknown and retains its process tree
ok 12 - closed root with an original live descendant remains unknown and retains its process tree
---
duration_ms: 0.2526
type: 'test'
...
# Subtest: host recovery reserves original pending attempts exactly once and explicit review preserves unknown outcomes
ok 13 - host recovery reserves original pending attempts exactly once and explicit review preserves unknown outcomes
---
duration_ms: 18570.7899
type: 'test'
...
# Subtest: corrupted model reservation cannot reduce recovered budget or permit a new engine request
ok 14 - corrupted model reservation cannot reduce recovered budget or permit a new engine request
---
duration_ms: 14504.5152
type: 'test'
...
# Subtest: engine detection seals the actual CLI, SDK, profile and Node and rejects mixed versions without launch
ok 15 - engine detection seals the actual CLI, SDK, profile and Node and rejects mixed versions without launch
---
duration_ms: 691.4474
type: 'test'
...
# Subtest: owned engine probe waits for real ready identity, verifies six native responses and saves immutable task evidence
ok 16 - owned engine probe waits for real ready identity, verifies six native responses and saves immutable task evidence
---
duration_ms: 1365.5348
type: 'test'
...
# Subtest: an invalid startup nonce is rejected before initialize and preserves the process close and partial report
ok 17 - an invalid startup nonce is rejected before initialize and preserves the process close and partial report
---
duration_ms: 1474.5263
type: 'test'
...
# Subtest: SDK identity and session directory mismatches retain raw responses and never become verified
ok 18 - SDK identity and session directory mismatches retain raw responses and never become verified
---
duration_ms: 3976.8659
type: 'test'
...
# Subtest: cancel during a pending initialize closes only the owned process without sending an unsupported MCP cancellation
ok 19 - cancel during a pending initialize closes only the owned process without sending an unsupported MCP cancellation
---
duration_ms: 1546.7165
type: 'test'
...
# Subtest: cancel before readiness sends no protocol request and a user retry creates a different process and record
ok 20 - cancel before readiness sends no protocol request and a user retry creates a different process and record
---
duration_ms: 1877.5257
type: 'test'
...
# Subtest: source or scratch changes and unexpected model traffic fail after preserving the actual observed result
ok 21 - source or scratch changes and unexpected model traffic fail after preserving the actual observed result
---
duration_ms: 4286.0265
type: 'test'
...
# Subtest: unowned idle notifications stay quarantined and server requests receive a rejection without granting operations
ok 22 - unowned idle notifications stay quarantined and server requests receive a rejection without granting operations
---
duration_ms: 2147.3696
type: 'test'
...
# Subtest: engine probe environment excludes parent secrets and NODE_OPTIONS while preserving operating system paths
ok 23 - engine probe environment excludes parent secrets and NODE_OPTIONS while preserving operating system paths
---
duration_ms: 1433.9806
type: 'test'
...
# Subtest: foreign sessions and altered reports or task bindings cannot reuse verified engine evidence
ok 24 - foreign sessions and altered reports or task bindings cannot reuse verified engine evidence
---
duration_ms: 1284.3574
type: 'test'
...
# Subtest: restart never replays an active engine check or stops a historical PID, and keeps prior task state
ok 25 - restart never replays an active engine check or stops a historical PID, and keeps prior task state
---
duration_ms: 1215.2126
type: 'test'
...
# Subtest: model argument validation and runtime activity do not grant external engine prompt or project execution
ok 26 - model argument validation and runtime activity do not grant external engine prompt or project execution
---
duration_ms: 544.6424
type: 'test'
...
# Subtest: a follow-up question receives the actual failed engine check from its original Task and keeps its report unchanged
ok 27 - a follow-up question receives the actual failed engine check from its original Task and keeps its report unchanged
---
duration_ms: 2218.6958
type: 'test'
...
# Subtest: unfixed dependency URLs and invalid integrity are reported without exposing URL credentials or allowing a plan
ok 28 - unfixed dependency URLs and invalid integrity are reported without exposing URL credentials or allowing a plan
---
duration_ms: 570.0069
type: 'test'
...
# Subtest: hot upgrade refuses an active dependency operation or evaluation before changing runtime objects
ok 29 - hot upgrade refuses an active dependency operation or evaluation before changing runtime objects
---
duration_ms: 0.5217
type: 'test'
...
# Subtest: read-only environment diagnosis identifies actual runtime, missing dependencies and sealed inputs without installing or running scripts
ok 30 - read-only environment diagnosis identifies actual runtime, missing dependencies and sealed inputs without installing or running scripts
---
duration_ms: 8845.5782
type: 'test'
...
# Subtest: diagnosis rejects unsupported managers, lock integrity and credential configuration without reading config content
ok 31 - diagnosis rejects unsupported managers, lock integrity and credential configuration without reading config content
---
duration_ms: 6599.2291
type: 'test'
...
# Subtest: approved real npm ci runs only in owned worktree, skips scripts, retains logs and binds later actual tests
ok 32 - approved real npm ci runs only in owned worktree, skips scripts, retains logs and binds later actual tests
---
duration_ms: 14692.5886
type: 'test'
...
# Subtest: explicitly allowing lifecycle scripts executes sealed root script and preserves the separate preparation result
ok 33 - explicitly allowing lifecycle scripts executes sealed root script and preserves the separate preparation result
---
duration_ms: 12092.2564
type: 'test'
...
# Subtest: stale lockfile, tampered plan/config, cross-session identities and mismatched download sources never dispatch npm
ok 34 - stale lockfile, tampered plan/config, cross-session identities and mismatched download sources never dispatch npm
---
duration_ms: 9886.3817
type: 'test'
...
# Subtest: registry failure keeps original npm exit/log and performs no automatic fallback or replay
ok 35 - registry failure keeps original npm exit/log and performs no automatic fallback or replay
---
duration_ms: 10303.0288
type: 'test'
...
# Subtest: stopping a hanging dependency download records only its owned process and preserves stopped logs
ok 36 - stopping a hanging dependency download records only its owned process and preserves stopped logs
---
duration_ms: 12201.907
type: 'test'
...
# Subtest: restart retains unknown preparation, blocks writes and checks, and requires explicit read-only reconciliation
ok 37 - restart retains unknown preparation, blocks writes and checks, and requires explicit read-only reconciliation
---
duration_ms: 9518.1219
type: 'test'
...
# Subtest: HTTP environment flow survives refresh, requires explicit approval and preserves task and artifact ownership
ok 38 - HTTP environment flow survives refresh, requires explicit approval and preserves task and artifact ownership
---
duration_ms: 14735.0257
type: 'test'
...
# Subtest: Python environment diagnosis reads declarations and runtime identity without executing project code or installing packages
ok 39 - Python environment diagnosis reads declarations and runtime identity without executing project code or installing packages
---
duration_ms: 872.6118
type: 'test'
...
# Subtest: empty build cache directories keep dependency identity while added executable files invalidate it
ok 40 - empty build cache directories keep dependency identity while added executable files invalidate it
---
duration_ms: 917.1903
type: 'test'
...
# Subtest: dependency snapshots reject external links and incomplete budgets instead of claiming environment equivalence
ok 41 - dependency snapshots reject external links and incomplete budgets instead of claiming environment equivalence
---
duration_ms: 534.7586
type: 'test'
...
# Subtest: model-requested environment preparation uses the original approval boundary and never treats preparation as passed engineering checks
ok 42 - model-requested environment preparation uses the original approval boundary and never treats preparation as passed engineering checks
---
duration_ms: 13090.0432
type: 'test'
...
# Subtest: timeout and offline cache miss retain actual failures and new attempts require newly sealed plans
ok 43 - timeout and offline cache miss retain actual failures and new attempts require newly sealed plans
---
duration_ms: 15299.3515
type: 'test'
...
# Subtest: both evaluation worktrees can approve fixed dependency preparation and different actual dependencies reject candidate execution
ok 44 - both evaluation worktrees can approve fixed dependency preparation and different actual dependencies reject candidate execution
---
duration_ms: 36235.2608
type: 'test'
...
# Subtest: evaluation denominator distinguishes attempted unsealed candidates from unexecuted samples
ok 45 - evaluation denominator distinguishes attempted unsealed candidates from unexecuted samples
---
duration_ms: 0.3066
type: 'test'
...
# Subtest: evaluation summary retains sealed outcomes and stopped or failed original attempts
ok 46 - evaluation summary retains sealed outcomes and stopped or failed original attempts
---
duration_ms: 0.4276
type: 'test'
...
# Subtest: evaluation contract fixes bounded samples, source paths, checks and aggregate budget
ok 47 - evaluation contract fixes bounded samples, source paths, checks and aggregate budget
---
duration_ms: 554.6807
type: 'test'
...
# Subtest: same-source workspaces seal real commands before execution and preserve original staged and unstaged bytes
ok 48 - same-source workspaces seal real commands before execution and preserve original staged and unstaged bytes
---
duration_ms: 16813.8217
type: 'test'
...
# Subtest: actual original failures and candidate success are both preserved, then adoption requires parent review, checks and Git
ok 49 - actual original failures and candidate success are both preserved, then adoption requires parent review, checks and Git
---
duration_ms: 63061.515
type: 'test'
...
# Subtest: candidate cannot shrink tests and changing original test bytes rejects execution before any model request
ok 50 - candidate cannot shrink tests and changing original test bytes rejects execution before any model request
---
duration_ms: 22874.7829
type: 'test'
...
# Subtest: missing upstream usage remains unknown and does not become a free run or invented cost
ok 51 - missing upstream usage remains unknown and does not become a free run or invented cost
---
duration_ms: 39257.0579
type: 'test'
...
# Subtest: missing raw check logs preserve unknown result and original sample denominator
ok 52 - missing raw check logs preserve unknown result and original sample denominator
---
duration_ms: 38870.3273
type: 'test'
...
# Subtest: source and model changes never silently replace an existing batch input
ok 53 - source and model changes never silently replace an existing batch input
---
duration_ms: 2930.4719
type: 'test'
...
# Subtest: model revision change before preparing a fixed sample cannot silently use the replacement connection
ok 54 - model revision change before preparing a fixed sample cannot silently use the replacement connection
---
duration_ms: 2865.9911
type: 'test'
...
# Subtest: HTTP evaluations require host token, original fingerprint and session ownership; worker API cannot bypass evaluation flow
ok 55 - HTTP evaluations require host token, original fingerprint and session ownership; worker API cannot bypass evaluation flow
---
duration_ms: 17979.2792
type: 'test'
...
# Subtest: sample queue cannot skip or reset attempted samples, and reports freeze further writes
ok 56 - sample queue cannot skip or reset attempted samples, and reports freeze further writes
---
duration_ms: 31777.5762
type: 'test'
...
# Subtest: stop cancels only current candidate and preserves partial results instead of replaying a request
ok 57 - stop cancels only current candidate and preserves partial results instead of replaying a request
---
duration_ms: 27586.6407
type: 'test'
...
# Subtest: restart recovers interrupted evaluations as unknown and keeps completed checks and plan identity
ok 58 - restart recovers interrupted evaluations as unknown and keeps completed checks and plan identity
---
duration_ms: 21499.7638
type: 'test'
...
# Subtest: JSON migration survives process exit before commit and retains exact legacy backups
ok 59 - JSON migration survives process exit before commit and retains exact legacy backups
---
duration_ms: 2099.3264
type: 'test'
...
# Subtest: process exit after_state rolls back state, event and outbox together
ok 60 - process exit after_state rolls back state, event and outbox together
---
duration_ms: 1512.5225
type: 'test'
...
# Subtest: process exit after_event rolls back state, event and outbox together
ok 61 - process exit after_event rolls back state, event and outbox together
---
duration_ms: 1606.8829
type: 'test'
...
# Subtest: crash recovery at accepted retains execution identity and never replays side effects
ok 62 - crash recovery at accepted retains execution identity and never replays side effects
---
duration_ms: 1463.881
type: 'test'
...
# Subtest: crash recovery at queued retains execution identity and never replays side effects
ok 63 - crash recovery at queued retains execution identity and never replays side effects
---
duration_ms: 1798.1463
type: 'test'
...
# Subtest: crash recovery at approval_wait retains execution identity and never replays side effects
ok 64 - crash recovery at approval_wait retains execution identity and never replays side effects
---
duration_ms: 2044.6788
type: 'test'
...
# Subtest: crash recovery at approved retains execution identity and never replays side effects
ok 65 - crash recovery at approved retains execution identity and never replays side effects
---
duration_ms: 2419.8106
type: 'test'
...
# Subtest: crash recovery at effect_unrecorded retains execution identity and never replays side effects
ok 66 - crash recovery at effect_unrecorded retains execution identity and never replays side effects
---
duration_ms: 2452.318
type: 'test'
...
# Subtest: crash recovery at result_committed retains execution identity and never replays side effects
ok 67 - crash recovery at result_committed retains execution identity and never replays side effects
---
duration_ms: 2346.6013
type: 'test'
...
# Subtest: crash recovery at before_next_model retains execution identity and never replays side effects
ok 68 - crash recovery at before_next_model retains execution identity and never replays side effects
---
duration_ms: 2606.4393
type: 'test'
...
# Subtest: state and event failures publish no success and preserve the last committed memory snapshot
ok 69 - state and event failures publish no success and preserve the last committed memory snapshot
---
duration_ms: 568.905
type: 'test'
...
# Subtest: tool result artifacts and event cursors retain project ownership across restart
ok 70 - tool result artifacts and event cursors retain project ownership across restart
---
duration_ms: 1209.0904
type: 'test'
...
# Subtest: token budgets pause before tools execute and continuation keeps original task and accumulated usage
ok 71 - token budgets pause before tools execute and continuation keeps original task and accumulated usage
---
duration_ms: 1330.6393
type: 'test'
...
# Subtest: a time budget interrupts an unresponsive model and persists a resumable run
ok 72 - a time budget interrupts an unresponsive model and persists a resumable run
---
duration_ms: 888.5986
type: 'test'
...
# Subtest: a corrupted large result artifact cannot become a recovered success
ok 73 - a corrupted large result artifact cannot become a recovered success
---
duration_ms: 748.7943
type: 'test'
...
# Subtest: hard-link aliases cannot expose protected bytes through reads, proposals, inventory or automatic project rules
ok 74 - hard-link aliases cannot expose protected bytes through reads, proposals, inventory or automatic project rules
---
duration_ms: 563.1552
type: 'test'
...
# Subtest: ambiguous project hard links are explicit and ordinary independent file copies remain readable
ok 75 - ambiguous project hard links are explicit and ordinary independent file copies remain readable
---
duration_ms: 551.4092
type: 'test'
...
# Subtest: proposals preserve project bytes until review; apply backs up the exact original bytes
ok 76 - proposals preserve project bytes until review; apply backs up the exact original bytes
---
duration_ms: 588.5107
type: 'test'
...
# Subtest: new nested files are created only on apply and unchanged content creates no proposal
ok 77 - new nested files are created only on apply and unchanged content creates no proposal
---
duration_ms: 601.913
type: 'test'
...
# Subtest: stale reads and changed baselines are rejected without overwriting external edits
ok 78 - stale reads and changed baselines are rejected without overwriting external edits
---
duration_ms: 584.3901
type: 'test'
...
# Subtest: file paths reject traversal, absolute paths, credentials and outside junctions
ok 79 - file paths reject traversal, absolute paths, credentials and outside junctions
---
duration_ms: 591.8071
type: 'test'
...
# Subtest: read and search expose bounded text scopes and reject non UTF-8 / oversized files
ok 80 - read and search expose bounded text scopes and reject non UTF-8 / oversized files
---
duration_ms: 458.4432
type: 'test'
...
# Subtest: project-local junctions cannot expose or create protected Git and credential files
ok 81 - project-local junctions cannot expose or create protected Git and credential files
---
duration_ms: 490.2613
type: 'test'
...
# Subtest: Windows trailing-dot and trailing-space path aliases cannot bypass protected paths
ok 82 - Windows trailing-dot and trailing-space path aliases cannot bypass protected paths
---
duration_ms: 406.9311
type: 'test'
...
1..82
# tests 82
# suites 0
# pass 82
# fail 0
# cancelled 0
# skipped 0
# todo 0
# duration_ms 490016.6222