509 lines
22 KiB
Plaintext
509 lines
22 KiB
Plaintext
TAP version 13
|
|
# Subtest: native text and call IDs preserve original result pairing while unsupported content fails closed
|
|
ok 1 - native text and call IDs preserve original result pairing while unsupported content fails closed
|
|
---
|
|
duration_ms: 0.8417
|
|
type: 'test'
|
|
...
|
|
# Subtest: one engine child freezes real installation identity before creating tasks and rejects a changed source
|
|
ok 2 - one engine child freezes real installation identity before creating tasks and rejects a changed source
|
|
---
|
|
duration_ms: 15609.1906
|
|
type: 'test'
|
|
...
|
|
# Subtest: owned native tools return saved results, use real approvals/checks and account each actual request once across new engine Runs
|
|
ok 3 - owned native tools return saved results, use real approvals/checks and account each actual request once across new engine Runs
|
|
---
|
|
duration_ms: 45928.8612
|
|
type: 'test'
|
|
...
|
|
# Subtest: forged native scope cannot change a parent file and a final completion claim cannot replace worker proof
|
|
ok 4 - forged native scope cannot change a parent file and a final completion claim cannot replace worker proof
|
|
---
|
|
duration_ms: 22915.781
|
|
type: 'test'
|
|
...
|
|
# Subtest: duplicate native IDs are rejected without replaying a saved tool
|
|
ok 5 - duplicate native IDs are rejected without replaying a saved tool
|
|
---
|
|
duration_ms: 18751.1202
|
|
type: 'test'
|
|
...
|
|
# Subtest: native input exceeding the context budget is retained and sends no model request
|
|
ok 6 - native input exceeding the context budget is retained and sends no model request
|
|
---
|
|
duration_ms: 19254.2542
|
|
type: 'test'
|
|
...
|
|
# Subtest: stop retains a late reported model usage once and executes no tool from that output
|
|
ok 7 - stop retains a late reported model usage once and executes no tool from that output
|
|
---
|
|
duration_ms: 15389.4059
|
|
type: 'test'
|
|
...
|
|
# Subtest: a source change after the original model intent stops before native tools execute
|
|
ok 8 - a source change after the original model intent stops before native tools execute
|
|
---
|
|
duration_ms: 15856.2769
|
|
type: 'test'
|
|
...
|
|
# Subtest: an engine source change while approval is pending invalidates the old permission before any check starts
|
|
ok 9 - an engine source change while approval is pending invalidates the old permission before any check starts
|
|
---
|
|
duration_ms: 20388.1789
|
|
type: 'test'
|
|
...
|
|
# Subtest: missing or noncompleted turn end cannot be adopted even with an owned receipt and assistant text
|
|
ok 10 - missing or noncompleted turn end cannot be adopted even with an owned receipt and assistant text
|
|
---
|
|
duration_ms: 0.5181
|
|
type: 'test'
|
|
...
|
|
# Subtest: closed root with an original live descendant remains unknown and retains its process tree
|
|
ok 11 - closed root with an original live descendant remains unknown and retains its process tree
|
|
---
|
|
duration_ms: 0.2523
|
|
type: 'test'
|
|
...
|
|
# Subtest: host recovery reserves original pending attempts exactly once and explicit review preserves unknown outcomes
|
|
ok 12 - host recovery reserves original pending attempts exactly once and explicit review preserves unknown outcomes
|
|
---
|
|
duration_ms: 23705.5842
|
|
type: 'test'
|
|
...
|
|
# Subtest: corrupted model reservation cannot reduce recovered budget or permit a new engine request
|
|
ok 13 - corrupted model reservation cannot reduce recovered budget or permit a new engine request
|
|
---
|
|
duration_ms: 16626.6754
|
|
type: 'test'
|
|
...
|
|
# Subtest: engine detection seals the actual CLI, SDK, profile and Node and rejects mixed versions without launch
|
|
ok 14 - engine detection seals the actual CLI, SDK, profile and Node and rejects mixed versions without launch
|
|
---
|
|
duration_ms: 2382.5231
|
|
type: 'test'
|
|
...
|
|
# Subtest: owned engine probe waits for real ready identity, verifies six native responses and saves immutable task evidence
|
|
ok 15 - owned engine probe waits for real ready identity, verifies six native responses and saves immutable task evidence
|
|
---
|
|
duration_ms: 4581.737
|
|
type: 'test'
|
|
...
|
|
# Subtest: an invalid startup nonce is rejected before initialize and preserves the process close and partial report
|
|
ok 16 - an invalid startup nonce is rejected before initialize and preserves the process close and partial report
|
|
---
|
|
duration_ms: 3261.5512
|
|
type: 'test'
|
|
...
|
|
# Subtest: SDK identity and session directory mismatches retain raw responses and never become verified
|
|
ok 17 - SDK identity and session directory mismatches retain raw responses and never become verified
|
|
---
|
|
duration_ms: 7007.113
|
|
type: 'test'
|
|
...
|
|
# Subtest: cancel during a pending initialize closes only the owned process without sending an unsupported MCP cancellation
|
|
ok 18 - cancel during a pending initialize closes only the owned process without sending an unsupported MCP cancellation
|
|
---
|
|
duration_ms: 2667.2594
|
|
type: 'test'
|
|
...
|
|
# Subtest: cancel before readiness sends no protocol request and a user retry creates a different process and record
|
|
ok 19 - cancel before readiness sends no protocol request and a user retry creates a different process and record
|
|
---
|
|
duration_ms: 3397.7636
|
|
type: 'test'
|
|
...
|
|
# Subtest: source or scratch changes and unexpected model traffic fail after preserving the actual observed result
|
|
ok 20 - source or scratch changes and unexpected model traffic fail after preserving the actual observed result
|
|
---
|
|
duration_ms: 10931.1076
|
|
type: 'test'
|
|
...
|
|
# Subtest: unowned idle notifications stay quarantined and server requests receive a rejection without granting operations
|
|
ok 21 - unowned idle notifications stay quarantined and server requests receive a rejection without granting operations
|
|
---
|
|
duration_ms: 6881.2404
|
|
type: 'test'
|
|
...
|
|
# Subtest: engine probe environment excludes parent secrets and NODE_OPTIONS while preserving operating system paths
|
|
ok 22 - engine probe environment excludes parent secrets and NODE_OPTIONS while preserving operating system paths
|
|
---
|
|
duration_ms: 3611.1364
|
|
type: 'test'
|
|
...
|
|
# Subtest: foreign sessions and altered reports or task bindings cannot reuse verified engine evidence
|
|
ok 23 - foreign sessions and altered reports or task bindings cannot reuse verified engine evidence
|
|
---
|
|
duration_ms: 3796.8687
|
|
type: 'test'
|
|
...
|
|
# Subtest: restart never replays an active engine check or stops a historical PID, and keeps prior task state
|
|
ok 24 - restart never replays an active engine check or stops a historical PID, and keeps prior task state
|
|
---
|
|
duration_ms: 3378.7785
|
|
type: 'test'
|
|
...
|
|
# Subtest: model argument validation and runtime activity do not grant external engine prompt or project execution
|
|
ok 25 - model argument validation and runtime activity do not grant external engine prompt or project execution
|
|
---
|
|
duration_ms: 1597.0333
|
|
type: 'test'
|
|
...
|
|
# Subtest: a follow-up question receives the actual failed engine check from its original Task and keeps its report unchanged
|
|
ok 26 - a follow-up question receives the actual failed engine check from its original Task and keeps its report unchanged
|
|
---
|
|
duration_ms: 4256.6161
|
|
type: 'test'
|
|
...
|
|
# Subtest: unfixed dependency URLs and invalid integrity are reported without exposing URL credentials or allowing a plan
|
|
ok 27 - unfixed dependency URLs and invalid integrity are reported without exposing URL credentials or allowing a plan
|
|
---
|
|
duration_ms: 1244.7712
|
|
type: 'test'
|
|
...
|
|
# Subtest: hot upgrade refuses an active dependency operation or evaluation before changing runtime objects
|
|
ok 28 - hot upgrade refuses an active dependency operation or evaluation before changing runtime objects
|
|
---
|
|
duration_ms: 0.4882
|
|
type: 'test'
|
|
...
|
|
# Subtest: read-only environment diagnosis identifies actual runtime, missing dependencies and sealed inputs without installing or running scripts
|
|
ok 29 - read-only environment diagnosis identifies actual runtime, missing dependencies and sealed inputs without installing or running scripts
|
|
---
|
|
duration_ms: 8038.7124
|
|
type: 'test'
|
|
...
|
|
# Subtest: diagnosis rejects unsupported managers, lock integrity and credential configuration without reading config content
|
|
ok 30 - diagnosis rejects unsupported managers, lock integrity and credential configuration without reading config content
|
|
---
|
|
duration_ms: 10098.6573
|
|
type: 'test'
|
|
...
|
|
# Subtest: approved real npm ci runs only in owned worktree, skips scripts, retains logs and binds later actual tests
|
|
ok 31 - approved real npm ci runs only in owned worktree, skips scripts, retains logs and binds later actual tests
|
|
---
|
|
duration_ms: 17827.49
|
|
type: 'test'
|
|
...
|
|
# Subtest: explicitly allowing lifecycle scripts executes sealed root script and preserves the separate preparation result
|
|
ok 32 - explicitly allowing lifecycle scripts executes sealed root script and preserves the separate preparation result
|
|
---
|
|
duration_ms: 12082.2757
|
|
type: 'test'
|
|
...
|
|
# Subtest: stale lockfile, tampered plan/config, cross-session identities and mismatched download sources never dispatch npm
|
|
ok 33 - stale lockfile, tampered plan/config, cross-session identities and mismatched download sources never dispatch npm
|
|
---
|
|
duration_ms: 14872.4814
|
|
type: 'test'
|
|
...
|
|
# Subtest: registry failure keeps original npm exit/log and performs no automatic fallback or replay
|
|
ok 34 - registry failure keeps original npm exit/log and performs no automatic fallback or replay
|
|
---
|
|
duration_ms: 11688.8052
|
|
type: 'test'
|
|
...
|
|
# Subtest: stopping a hanging dependency download records only its owned process and preserves stopped logs
|
|
ok 35 - stopping a hanging dependency download records only its owned process and preserves stopped logs
|
|
---
|
|
duration_ms: 10634.0737
|
|
type: 'test'
|
|
...
|
|
# Subtest: restart retains unknown preparation, blocks writes and checks, and requires explicit read-only reconciliation
|
|
ok 36 - restart retains unknown preparation, blocks writes and checks, and requires explicit read-only reconciliation
|
|
---
|
|
duration_ms: 9732.5355
|
|
type: 'test'
|
|
...
|
|
# Subtest: HTTP environment flow survives refresh, requires explicit approval and preserves task and artifact ownership
|
|
ok 37 - HTTP environment flow survives refresh, requires explicit approval and preserves task and artifact ownership
|
|
---
|
|
duration_ms: 17390.229
|
|
type: 'test'
|
|
...
|
|
# Subtest: Python environment diagnosis reads declarations and runtime identity without executing project code or installing packages
|
|
ok 38 - Python environment diagnosis reads declarations and runtime identity without executing project code or installing packages
|
|
---
|
|
duration_ms: 1138.1942
|
|
type: 'test'
|
|
...
|
|
# Subtest: empty build cache directories keep dependency identity while added executable files invalidate it
|
|
ok 39 - empty build cache directories keep dependency identity while added executable files invalidate it
|
|
---
|
|
duration_ms: 1207.8461
|
|
type: 'test'
|
|
...
|
|
# Subtest: dependency snapshots reject external links and incomplete budgets instead of claiming environment equivalence
|
|
ok 40 - dependency snapshots reject external links and incomplete budgets instead of claiming environment equivalence
|
|
---
|
|
duration_ms: 1057.502
|
|
type: 'test'
|
|
...
|
|
# Subtest: model-requested environment preparation uses the original approval boundary and never treats preparation as passed engineering checks
|
|
ok 41 - model-requested environment preparation uses the original approval boundary and never treats preparation as passed engineering checks
|
|
---
|
|
duration_ms: 16019.1771
|
|
type: 'test'
|
|
...
|
|
# Subtest: timeout and offline cache miss retain actual failures and new attempts require newly sealed plans
|
|
ok 42 - timeout and offline cache miss retain actual failures and new attempts require newly sealed plans
|
|
---
|
|
duration_ms: 18029.8733
|
|
type: 'test'
|
|
...
|
|
# Subtest: both evaluation worktrees can approve fixed dependency preparation and different actual dependencies reject candidate execution
|
|
ok 43 - both evaluation worktrees can approve fixed dependency preparation and different actual dependencies reject candidate execution
|
|
---
|
|
duration_ms: 37556.4495
|
|
type: 'test'
|
|
...
|
|
# Subtest: evaluation denominator distinguishes attempted unsealed candidates from unexecuted samples
|
|
ok 44 - evaluation denominator distinguishes attempted unsealed candidates from unexecuted samples
|
|
---
|
|
duration_ms: 0.3112
|
|
type: 'test'
|
|
...
|
|
# Subtest: evaluation summary retains sealed outcomes and stopped or failed original attempts
|
|
ok 45 - evaluation summary retains sealed outcomes and stopped or failed original attempts
|
|
---
|
|
duration_ms: 0.4247
|
|
type: 'test'
|
|
...
|
|
# Subtest: evaluation contract fixes bounded samples, source paths, checks and aggregate budget
|
|
ok 46 - evaluation contract fixes bounded samples, source paths, checks and aggregate budget
|
|
---
|
|
duration_ms: 892.8838
|
|
type: 'test'
|
|
...
|
|
# Subtest: same-source workspaces seal real commands before execution and preserve original staged and unstaged bytes
|
|
ok 47 - same-source workspaces seal real commands before execution and preserve original staged and unstaged bytes
|
|
---
|
|
duration_ms: 20243.194
|
|
type: 'test'
|
|
...
|
|
# Subtest: actual original failures and candidate success are both preserved, then adoption requires parent review, checks and Git
|
|
ok 48 - actual original failures and candidate success are both preserved, then adoption requires parent review, checks and Git
|
|
---
|
|
duration_ms: 78233.9299
|
|
type: 'test'
|
|
...
|
|
# Subtest: candidate cannot shrink tests and changing original test bytes rejects execution before any model request
|
|
ok 49 - candidate cannot shrink tests and changing original test bytes rejects execution before any model request
|
|
---
|
|
duration_ms: 28091.6476
|
|
type: 'test'
|
|
...
|
|
# Subtest: missing upstream usage remains unknown and does not become a free run or invented cost
|
|
ok 50 - missing upstream usage remains unknown and does not become a free run or invented cost
|
|
---
|
|
duration_ms: 60112.4446
|
|
type: 'test'
|
|
...
|
|
# Subtest: missing raw check logs preserve unknown result and original sample denominator
|
|
ok 51 - missing raw check logs preserve unknown result and original sample denominator
|
|
---
|
|
duration_ms: 50831.5537
|
|
type: 'test'
|
|
...
|
|
# Subtest: source and model changes never silently replace an existing batch input
|
|
ok 52 - source and model changes never silently replace an existing batch input
|
|
---
|
|
duration_ms: 14581.2659
|
|
type: 'test'
|
|
...
|
|
# Subtest: model revision change before preparing a fixed sample cannot silently use the replacement connection
|
|
ok 53 - model revision change before preparing a fixed sample cannot silently use the replacement connection
|
|
---
|
|
duration_ms: 10946.8637
|
|
type: 'test'
|
|
...
|
|
# Subtest: HTTP evaluations require host token, original fingerprint and session ownership; worker API cannot bypass evaluation flow
|
|
ok 54 - HTTP evaluations require host token, original fingerprint and session ownership; worker API cannot bypass evaluation flow
|
|
---
|
|
duration_ms: 28529.0188
|
|
type: 'test'
|
|
...
|
|
# Subtest: sample queue cannot skip or reset attempted samples, and reports freeze further writes
|
|
ok 55 - sample queue cannot skip or reset attempted samples, and reports freeze further writes
|
|
---
|
|
duration_ms: 39653.1077
|
|
type: 'test'
|
|
...
|
|
# Subtest: stop cancels only current candidate and preserves partial results instead of replaying a request
|
|
ok 56 - stop cancels only current candidate and preserves partial results instead of replaying a request
|
|
---
|
|
duration_ms: 27434.5931
|
|
type: 'test'
|
|
...
|
|
# Subtest: restart recovers interrupted evaluations as unknown and keeps completed checks and plan identity
|
|
ok 57 - restart recovers interrupted evaluations as unknown and keeps completed checks and plan identity
|
|
---
|
|
duration_ms: 26220.1701
|
|
type: 'test'
|
|
...
|
|
# Subtest: JSON migration survives process exit before commit and retains exact legacy backups
|
|
ok 58 - JSON migration survives process exit before commit and retains exact legacy backups
|
|
---
|
|
duration_ms: 2133.9924
|
|
type: 'test'
|
|
...
|
|
# Subtest: process exit after_state rolls back state, event and outbox together
|
|
ok 59 - process exit after_state rolls back state, event and outbox together
|
|
---
|
|
duration_ms: 1707.4334
|
|
type: 'test'
|
|
...
|
|
# Subtest: process exit after_event rolls back state, event and outbox together
|
|
ok 60 - process exit after_event rolls back state, event and outbox together
|
|
---
|
|
duration_ms: 1798.9375
|
|
type: 'test'
|
|
...
|
|
# Subtest: crash recovery at accepted retains execution identity and never replays side effects
|
|
ok 61 - crash recovery at accepted retains execution identity and never replays side effects
|
|
---
|
|
duration_ms: 2111.6784
|
|
type: 'test'
|
|
...
|
|
# Subtest: crash recovery at queued retains execution identity and never replays side effects
|
|
ok 62 - crash recovery at queued retains execution identity and never replays side effects
|
|
---
|
|
duration_ms: 2608.3353
|
|
type: 'test'
|
|
...
|
|
# Subtest: crash recovery at approval_wait retains execution identity and never replays side effects
|
|
ok 63 - crash recovery at approval_wait retains execution identity and never replays side effects
|
|
---
|
|
duration_ms: 3222.8478
|
|
type: 'test'
|
|
...
|
|
# Subtest: crash recovery at approved retains execution identity and never replays side effects
|
|
ok 64 - crash recovery at approved retains execution identity and never replays side effects
|
|
---
|
|
duration_ms: 3591.2984
|
|
type: 'test'
|
|
...
|
|
# Subtest: crash recovery at effect_unrecorded retains execution identity and never replays side effects
|
|
ok 65 - crash recovery at effect_unrecorded retains execution identity and never replays side effects
|
|
---
|
|
duration_ms: 3190.0154
|
|
type: 'test'
|
|
...
|
|
# Subtest: crash recovery at result_committed retains execution identity and never replays side effects
|
|
ok 66 - crash recovery at result_committed retains execution identity and never replays side effects
|
|
---
|
|
duration_ms: 3739.5844
|
|
type: 'test'
|
|
...
|
|
# Subtest: crash recovery at before_next_model retains execution identity and never replays side effects
|
|
ok 67 - crash recovery at before_next_model retains execution identity and never replays side effects
|
|
---
|
|
duration_ms: 3207.8017
|
|
type: 'test'
|
|
...
|
|
# Subtest: state and event failures publish no success and preserve the last committed memory snapshot
|
|
ok 68 - state and event failures publish no success and preserve the last committed memory snapshot
|
|
---
|
|
duration_ms: 680.6512
|
|
type: 'test'
|
|
...
|
|
# Subtest: tool result artifacts and event cursors retain project ownership across restart
|
|
ok 69 - tool result artifacts and event cursors retain project ownership across restart
|
|
---
|
|
duration_ms: 1604.0525
|
|
type: 'test'
|
|
...
|
|
# Subtest: token budgets pause before tools execute and continuation keeps original task and accumulated usage
|
|
ok 70 - token budgets pause before tools execute and continuation keeps original task and accumulated usage
|
|
---
|
|
duration_ms: 1647.253
|
|
type: 'test'
|
|
...
|
|
# Subtest: a time budget interrupts an unresponsive model and persists a resumable run
|
|
ok 71 - a time budget interrupts an unresponsive model and persists a resumable run
|
|
---
|
|
duration_ms: 989.7703
|
|
type: 'test'
|
|
...
|
|
# Subtest: a corrupted large result artifact cannot become a recovered success
|
|
ok 72 - a corrupted large result artifact cannot become a recovered success
|
|
---
|
|
duration_ms: 1525.1506
|
|
type: 'test'
|
|
...
|
|
# Subtest: hard-link aliases cannot expose protected bytes through reads, proposals, inventory or automatic project rules
|
|
ok 73 - hard-link aliases cannot expose protected bytes through reads, proposals, inventory or automatic project rules
|
|
---
|
|
duration_ms: 1048.9867
|
|
type: 'test'
|
|
...
|
|
# Subtest: ambiguous project hard links are explicit and ordinary independent file copies remain readable
|
|
ok 74 - ambiguous project hard links are explicit and ordinary independent file copies remain readable
|
|
---
|
|
duration_ms: 690.5394
|
|
type: 'test'
|
|
...
|
|
# Subtest: proposals preserve project bytes until review; apply backs up the exact original bytes
|
|
ok 75 - proposals preserve project bytes until review; apply backs up the exact original bytes
|
|
---
|
|
duration_ms: 798.9705
|
|
type: 'test'
|
|
...
|
|
# Subtest: new nested files are created only on apply and unchanged content creates no proposal
|
|
ok 76 - new nested files are created only on apply and unchanged content creates no proposal
|
|
---
|
|
duration_ms: 656.1803
|
|
type: 'test'
|
|
...
|
|
# Subtest: stale reads and changed baselines are rejected without overwriting external edits
|
|
ok 77 - stale reads and changed baselines are rejected without overwriting external edits
|
|
---
|
|
duration_ms: 695.1139
|
|
type: 'test'
|
|
...
|
|
# Subtest: file paths reject traversal, absolute paths, credentials and outside junctions
|
|
ok 78 - file paths reject traversal, absolute paths, credentials and outside junctions
|
|
---
|
|
duration_ms: 550.9841
|
|
type: 'test'
|
|
...
|
|
# Subtest: read and search expose bounded text scopes and reject non UTF-8 / oversized files
|
|
ok 79 - read and search expose bounded text scopes and reject non UTF-8 / oversized files
|
|
---
|
|
duration_ms: 574.171
|
|
type: 'test'
|
|
...
|
|
# Subtest: project-local junctions cannot expose or create protected Git and credential files
|
|
ok 80 - project-local junctions cannot expose or create protected Git and credential files
|
|
---
|
|
duration_ms: 560.8074
|
|
type: 'test'
|
|
...
|
|
# Subtest: Windows trailing-dot and trailing-space path aliases cannot bypass protected paths
|
|
ok 81 - Windows trailing-dot and trailing-space path aliases cannot bypass protected paths
|
|
---
|
|
duration_ms: 866.8621
|
|
type: 'test'
|
|
...
|
|
# Subtest: a starter is copied into user data once; restarting from another program version preserves existing projects and editable bytes
|
|
ok 82 - a starter is copied into user data once; restarting from another program version preserves existing projects and editable bytes
|
|
---
|
|
duration_ms: 1470.7973
|
|
type: 'test'
|
|
...
|
|
# Subtest: startup does not insert a demo or rewrite/reorder an existing user project registry
|
|
ok 83 - startup does not insert a demo or rewrite/reorder an existing user project registry
|
|
---
|
|
duration_ms: 830.9528
|
|
type: 'test'
|
|
...
|
|
1..83
|
|
# tests 83
|
|
# suites 0
|
|
# pass 83
|
|
# fail 0
|
|
# cancelled 0
|
|
# skipped 0
|
|
# todo 0
|
|
# duration_ms 632767.4879
|