437 lines
20 KiB
Plaintext
437 lines
20 KiB
Plaintext
TAP version 13
|
|
# Subtest: a starter is copied into user data once; restarting from another program version preserves existing projects and editable bytes
|
|
ok 1 - a starter is copied into user data once; restarting from another program version preserves existing projects and editable bytes
|
|
---
|
|
duration_ms: 1470.7753
|
|
type: 'test'
|
|
...
|
|
# Subtest: startup does not insert a demo or rewrite/reorder an existing user project registry
|
|
ok 2 - startup does not insert a demo or rewrite/reorder an existing user project registry
|
|
---
|
|
duration_ms: 819.6316
|
|
type: 'test'
|
|
...
|
|
# Subtest: PNG bytes are fully decoded, metadata-free and bounded; JPEG EXIF orientation is applied to the sent PNG
|
|
ok 3 - PNG bytes are fully decoded, metadata-free and bounded; JPEG EXIF orientation is applied to the sent PNG
|
|
---
|
|
duration_ms: 104.6542
|
|
type: 'test'
|
|
...
|
|
# Subtest: forged type, short headers, CRC damage, incomplete chunks, trailing payloads, animation and dimension bombs reject
|
|
ok 4 - forged type, short headers, CRC damage, incomplete chunks, trailing payloads, animation and dimension bombs reject
|
|
---
|
|
duration_ms: 24.1861
|
|
type: 'test'
|
|
...
|
|
# Subtest: draft image ownership, withdrawal, reload and atomic Task binding retain original SHA and original bytes
|
|
ok 5 - draft image ownership, withdrawal, reload and atomic Task binding retain original SHA and original bytes
|
|
---
|
|
duration_ms: 1688.2396
|
|
type: 'test'
|
|
...
|
|
# Subtest: a failed session transaction restores draft status and cannot leave a Run consuming the attachment
|
|
ok 6 - a failed session transaction restores draft status and cannot leave a Run consuming the attachment
|
|
---
|
|
duration_ms: 1004.2999
|
|
type: 'test'
|
|
...
|
|
# Subtest: draft limits, duplicate selection, source changes and cross-Task association reject before dispatch
|
|
ok 7 - draft limits, duplicate selection, source changes and cross-Task association reject before dispatch
|
|
---
|
|
duration_ms: 1884.55
|
|
type: 'test'
|
|
...
|
|
# Subtest: demo and legacy paths refuse image dispatch without creating a Task or consuming the draft
|
|
ok 8 - demo and legacy paths refuse image dispatch without creating a Task or consuming the draft
|
|
---
|
|
duration_ms: 873.4509
|
|
type: 'test'
|
|
...
|
|
# Subtest: chat-completions sends one owned image per actual request, pairs original tools and verifies exact sealed bytes
|
|
ok 9 - chat-completions sends one owned image per actual request, pairs original tools and verifies exact sealed bytes
|
|
---
|
|
duration_ms: 2363.9082
|
|
type: 'test'
|
|
...
|
|
# Subtest: responses sends one owned image per actual request, pairs original tools and verifies exact sealed bytes
|
|
ok 10 - responses sends one owned image per actual request, pairs original tools and verifies exact sealed bytes
|
|
---
|
|
duration_ms: 2627.0488
|
|
type: 'test'
|
|
...
|
|
# Subtest: mixed pool excludes unknown vision before lease and sends zero HTTP to the incompatible member
|
|
ok 11 - mixed pool excludes unknown vision before lease and sends zero HTTP to the incompatible member
|
|
---
|
|
duration_ms: 1863.4975
|
|
type: 'test'
|
|
...
|
|
# Subtest: vision probe requires correct randomized pixel order and evidence expires when the connection revision changes
|
|
ok 12 - vision probe requires correct randomized pixel order and evidence expires when the connection revision changes
|
|
---
|
|
duration_ms: 1329.2678
|
|
type: 'test'
|
|
...
|
|
# Subtest: HTTP upload/read/withdraw uses session authorization, returns byte SHA and refuses unknown ownership
|
|
ok 13 - HTTP upload/read/withdraw uses session authorization, returns byte SHA and refuses unknown ownership
|
|
---
|
|
duration_ms: 985.8731
|
|
type: 'test'
|
|
...
|
|
# Subtest: image accounting excludes Base64 as text but seals every original request byte and reserves image input separately
|
|
ok 14 - image accounting excludes Base64 as text but seals every original request byte and reserves image input separately
|
|
---
|
|
duration_ms: 3.0513
|
|
type: 'test'
|
|
...
|
|
# Subtest: directory picker reads one level, preserves empty folders, skips credential and linked folders
|
|
ok 15 - directory picker reads one level, preserves empty folders, skips credential and linked folders
|
|
---
|
|
duration_ms: 752.5085
|
|
type: 'test'
|
|
...
|
|
# Subtest: HTML preview serves encoded names and relative modules on an isolated loopback origin; closing revokes it
|
|
ok 16 - HTML preview serves encoded names and relative modules on an isolated loopback origin; closing revokes it
|
|
---
|
|
duration_ms: 770.3354
|
|
type: 'test'
|
|
...
|
|
# Subtest: HTML preview refuses credentials, hidden inventories, traversal, external junctions, hardlinks and unsupported file types
|
|
ok 17 - HTML preview refuses credentials, hidden inventories, traversal, external junctions, hardlinks and unsupported file types
|
|
---
|
|
duration_ms: 634.9496
|
|
type: 'test'
|
|
...
|
|
# Subtest: runtime shutdown closes local preview ports and pending starts cannot escape shutdown
|
|
ok 18 - runtime shutdown closes local preview ports and pending starts cannot escape shutdown
|
|
---
|
|
duration_ms: 857.6374
|
|
type: 'test'
|
|
...
|
|
# Subtest: idle file previews expire and release their loopback servers
|
|
ok 19 - idle file previews expire and release their loopback servers
|
|
---
|
|
duration_ms: 878.0056
|
|
type: 'test'
|
|
...
|
|
# Subtest: directory and preview HTTP controls require workbench authentication; project pages cannot read bootstrap
|
|
ok 20 - directory and preview HTTP controls require workbench authentication; project pages cannot read bootstrap
|
|
---
|
|
duration_ms: 2217.5125
|
|
type: 'test'
|
|
...
|
|
# Subtest: MCP registration stays disabled until an explicit real stdio probe and schema-based tool selection, with actual process cleanup
|
|
ok 21 - MCP registration stays disabled until an explicit real stdio probe and schema-based tool selection, with actual process cleanup
|
|
---
|
|
duration_ms: 2010.7814
|
|
type: 'test'
|
|
...
|
|
# Subtest: MCP probe holds the project operation lock until its owned process and report are settled
|
|
ok 22 - MCP probe holds the project operation lock until its owned process and report are settled
|
|
---
|
|
duration_ms: 1907.666
|
|
type: 'test'
|
|
...
|
|
# Subtest: MCP validates the declared JSON schema subset and rejects unsupported schemas before external dispatch
|
|
ok 23 - MCP validates the declared JSON schema subset and rejects unsupported schemas before external dispatch
|
|
---
|
|
duration_ms: 0.8041
|
|
type: 'test'
|
|
...
|
|
# Subtest: actual model MCP call approves long schema-valid arguments, validates the sent catalog and stores one original result without leaking environment secrets
|
|
ok 24 - actual model MCP call approves long schema-valid arguments, validates the sent catalog and stores one original result without leaking environment secrets
|
|
---
|
|
duration_ms: 5010.0181
|
|
type: 'test'
|
|
...
|
|
# Subtest: denying an MCP call never starts another process or executes the external tool
|
|
ok 25 - denying an MCP call never starts another process or executes the external tool
|
|
---
|
|
duration_ms: 3746.7283
|
|
type: 'test'
|
|
...
|
|
# Subtest: actual handshake drift rejects tools/call even if program entry hash is unchanged
|
|
ok 26 - actual handshake drift rejects tools/call even if program entry hash is unchanged
|
|
---
|
|
duration_ms: 4120.8819
|
|
type: 'test'
|
|
...
|
|
# Subtest: cancelled dispatched external side effect stays unknown, closes its owned process and cannot replay on reopen
|
|
ok 27 - cancelled dispatched external side effect stays unknown, closes its owned process and cannot replay on reopen
|
|
---
|
|
duration_ms: 4996.6873
|
|
type: 'test'
|
|
...
|
|
# Subtest: changed executable input, foreign project IDs and corrupted original contracts block enable and dispatch
|
|
ok 28 - changed executable input, foreign project IDs and corrupted original contracts block enable and dispatch
|
|
---
|
|
duration_ms: 1786.9062
|
|
type: 'test'
|
|
...
|
|
# Subtest: registered MCP HTTP API uses fixed execution contracts and project-owned probe records with no model requests
|
|
ok 29 - registered MCP HTTP API uses fixed execution contracts and project-owned probe records with no model requests
|
|
---
|
|
duration_ms: 3855.4457
|
|
type: 'test'
|
|
...
|
|
# Subtest: unknown MCP pauses later tool calls and all project writers; fingerprinted acknowledgement clears only the review gates and explicit resume does not replay
|
|
ok 30 - unknown MCP pauses later tool calls and all project writers; fingerprinted acknowledgement clears only the review gates and explicit resume does not replay
|
|
---
|
|
duration_ms: 5656.5752
|
|
type: 'test'
|
|
...
|
|
# Subtest: MCP acknowledgement persistence failure restores both job and tool review gates; corrupt evidence cannot be acknowledged
|
|
ok 31 - MCP acknowledgement persistence failure restores both job and tool review gates; corrupt evidence cannot be acknowledged
|
|
---
|
|
duration_ms: 5456.4823
|
|
type: 'test'
|
|
...
|
|
# Subtest: actual host exit after an external effect restores unknown jobs without a fabricated report, allows source-bound review and never replays the call
|
|
ok 32 - actual host exit after an external effect restores unknown jobs without a fabricated report, allows source-bound review and never replays the call
|
|
---
|
|
duration_ms: 4030.7308
|
|
type: 'test'
|
|
...
|
|
# Subtest: changed enabled MCP programs can be explicitly disabled without launching or trusting their changed source
|
|
ok 33 - changed enabled MCP programs can be explicitly disabled without launching or trusting their changed source
|
|
---
|
|
duration_ms: 1879.6398
|
|
type: 'test'
|
|
...
|
|
# Subtest: stdio preserves split UTF-8 frames and final replies drained after process exit
|
|
ok 34 - stdio preserves split UTF-8 frames and final replies drained after process exit
|
|
---
|
|
duration_ms: 1821.9941
|
|
type: 'test'
|
|
...
|
|
# Subtest: server-initiated model requests are refused instead of inheriting provider privileges
|
|
ok 35 - server-initiated model requests are refused instead of inheriting provider privileges
|
|
---
|
|
duration_ms: 899.8835
|
|
type: 'test'
|
|
...
|
|
# Subtest: invalid or over-budget protocol invalid closes only its owned process and is not retried
|
|
ok 36 - invalid or over-budget protocol invalid closes only its owned process and is not retried
|
|
---
|
|
duration_ms: 973.8443
|
|
type: 'test'
|
|
...
|
|
# Subtest: invalid or over-budget protocol oversized closes only its owned process and is not retried
|
|
ok 37 - invalid or over-budget protocol oversized closes only its owned process and is not retried
|
|
---
|
|
duration_ms: 883.8389
|
|
type: 'test'
|
|
...
|
|
# Subtest: cancel and timeout do not resend a dispatched request and pre-cancel has no request trace
|
|
ok 38 - cancel and timeout do not resend a dispatched request and pre-cancel has no request trace
|
|
---
|
|
duration_ms: 988.7068
|
|
type: 'test'
|
|
...
|
|
# Subtest: startup failure remains undispatched and has no process PID to terminate
|
|
ok 39 - startup failure remains undispatched and has no process PID to terminate
|
|
---
|
|
duration_ms: 718.6511
|
|
type: 'test'
|
|
...
|
|
# Subtest: REA launch filters a fixture credential canary and rejects changed entry identity before a new connection
|
|
ok 40 - REA launch filters a fixture credential canary and rejects changed entry identity before a new connection
|
|
---
|
|
duration_ms: 1065.1002
|
|
type: 'test'
|
|
...
|
|
# Subtest: schema1 connections remain readable without inventing observed capabilities or migrating original state
|
|
ok 41 - schema1 connections remain readable without inventing observed capabilities or migrating original state
|
|
---
|
|
duration_ms: 750.6685
|
|
type: 'test'
|
|
...
|
|
# Subtest: protocol, unsupported capabilities and stream choice persist; client claims cannot forge verified evidence
|
|
ok 42 - protocol, unsupported capabilities and stream choice persist; client claims cannot forge verified evidence
|
|
---
|
|
duration_ms: 1007.2279
|
|
type: 'test'
|
|
...
|
|
# Subtest: ability evidence belongs to exact connection and credential revision; unknown and unsupported reject required routing
|
|
ok 43 - ability evidence belongs to exact connection and credential revision; unknown and unsupported reject required routing
|
|
---
|
|
duration_ms: 0.4034
|
|
type: 'test'
|
|
...
|
|
# Subtest: invalid declarations and inconsistent stream settings reject before registry, revision and credentials mutate
|
|
ok 44 - invalid declarations and inconsistent stream settings reject before registry, revision and credentials mutate
|
|
---
|
|
duration_ms: 718.0848
|
|
type: 'test'
|
|
...
|
|
# Subtest: model registry keeps original credentials/endpoint intact, rejects stale edits, and forgets only volatile keys on restart
|
|
ok 45 - model registry keeps original credentials/endpoint intact, rejects stale edits, and forgets only volatile keys on restart
|
|
---
|
|
duration_ms: 1083.432
|
|
type: 'test'
|
|
...
|
|
# Subtest: config import previews exact revisions, keeps secrets volatile, syncs only when enabled and preserves manual edits on conflicts
|
|
ok 46 - config import previews exact revisions, keeps secrets volatile, syncs only when enabled and preserves manual edits on conflicts
|
|
---
|
|
duration_ms: 815.2745
|
|
type: 'test'
|
|
...
|
|
# Subtest: real HTTP pool fails over 429 to another model and never repeats completed tools; attempts seal actual request bytes and Run identity
|
|
ok 47 - real HTTP pool fails over 429 to another model and never repeats completed tools; attempts seal actual request bytes and Run identity
|
|
---
|
|
duration_ms: 3526.169
|
|
type: 'test'
|
|
...
|
|
# Subtest: pool rotation and weights select members on new Runs; disabled and missing credentials are excluded without requests
|
|
ok 48 - pool rotation and weights select members on new Runs; disabled and missing credentials are excluded without requests
|
|
---
|
|
duration_ms: 4290.422
|
|
type: 'test'
|
|
...
|
|
# Subtest: 401 blocks the credential revision; updating the key restores availability without leaking error echoes
|
|
ok 49 - 401 blocks the credential revision; updating the key restores availability without leaking error echoes
|
|
---
|
|
duration_ms: 1797.7703
|
|
type: 'test'
|
|
...
|
|
# Subtest: queued request cancellation and in-flight disable release only owned leases and do not dispatch another HTTP request
|
|
ok 50 - queued request cancellation and in-flight disable release only owned leases and do not dispatch another HTTP request
|
|
---
|
|
duration_ms: 816.3339
|
|
type: 'test'
|
|
...
|
|
# Subtest: compiled model-center HTTP API persists public profiles, blocks forged origins, and local operations never invoke a model
|
|
ok 51 - compiled model-center HTTP API persists public profiles, blocks forged origins, and local operations never invoke a model
|
|
---
|
|
duration_ms: 2212.2004
|
|
type: 'test'
|
|
...
|
|
# Subtest: model registry disk failures restore routing, pool profiles and credential references without partial updates
|
|
ok 52 - model registry disk failures restore routing, pool profiles and credential references without partial updates
|
|
---
|
|
duration_ms: 1009.1705
|
|
type: 'test'
|
|
...
|
|
# Subtest: model selection and saved configuration commit together and failures preserve the original mode and endpoint
|
|
ok 53 - model selection and saved configuration commit together and failures preserve the original mode and endpoint
|
|
---
|
|
duration_ms: 727.3941
|
|
type: 'test'
|
|
...
|
|
# Subtest: configuration imports roll back the entire batch and retain one usable preview after validation or commit failure
|
|
ok 54 - configuration imports roll back the entire batch and retain one usable preview after validation or commit failure
|
|
---
|
|
duration_ms: 714.11
|
|
type: 'test'
|
|
...
|
|
# Subtest: HTTP quota retains native code/type/param/request id, bounded original and allowlisted headers, never repeats that connection
|
|
ok 55 - HTTP quota retains native code/type/param/request id, bounded original and allowlisted headers, never repeats that connection
|
|
---
|
|
duration_ms: 1921.7962
|
|
type: 'test'
|
|
...
|
|
# Subtest: Retry-After above 90 seconds remains exact and a short local budget cannot trigger an early retry
|
|
ok 56 - Retry-After above 90 seconds remains exact and a short local budget cannot trigger an early retry
|
|
---
|
|
duration_ms: 1.2223
|
|
type: 'test'
|
|
...
|
|
# Subtest: Retry-After date, expired date, huge value and invalid header retain explicit semantics
|
|
ok 57 - Retry-After date, expired date, huge value and invalid header retain explicit semantics
|
|
---
|
|
duration_ms: 0.8285
|
|
type: 'test'
|
|
...
|
|
# Subtest: unclassified 429 remains unspecified and every bounded actual retry retains its own original error
|
|
ok 58 - unclassified 429 remains unspecified and every bounded actual retry retains its own original error
|
|
---
|
|
duration_ms: 1602.0531
|
|
type: 'test'
|
|
...
|
|
# Subtest: oversized HTTP error saves exactly the bounded prefix and labels its incomplete range
|
|
ok 59 - oversized HTTP error saves exactly the bounded prefix and labels its incomplete range
|
|
---
|
|
duration_ms: 1753.9606
|
|
type: 'test'
|
|
...
|
|
# Subtest: Chat SSE error preserves upstream fields, original failed event, partial tool and reported usage without executing or retrying
|
|
ok 60 - Chat SSE error preserves upstream fields, original failed event, partial tool and reported usage without executing or retrying
|
|
---
|
|
duration_ms: 1725.8399
|
|
type: 'test'
|
|
...
|
|
# Subtest: Responses failed SSE retains original event/response identity and usage
|
|
ok 61 - Responses failed SSE retains original event/response identity and usage
|
|
---
|
|
duration_ms: 1965.2807
|
|
type: 'test'
|
|
...
|
|
# Subtest: Responses incomplete SSE retains original event/response identity and usage
|
|
ok 62 - Responses incomplete SSE retains original event/response identity and usage
|
|
---
|
|
duration_ms: 1424.3667
|
|
type: 'test'
|
|
...
|
|
# Subtest: unsupported successful Responses output retains original JSON and usage with a local protocol origin
|
|
ok 63 - unsupported successful Responses output retains original JSON and usage with a local protocol origin
|
|
---
|
|
duration_ms: 1311.3357
|
|
type: 'test'
|
|
...
|
|
# Subtest: HTTP 200 native error is still an upstream failure rather than a fabricated successful answer
|
|
ok 64 - HTTP 200 native error is still an upstream failure rather than a fabricated successful answer
|
|
---
|
|
duration_ms: 0.6124
|
|
type: 'test'
|
|
...
|
|
# Subtest: JSON-escaped credential echoes are redacted in the original preview and capture as well as parsed fields
|
|
ok 65 - JSON-escaped credential echoes are redacted in the original preview and capture as well as parsed fields
|
|
---
|
|
duration_ms: 0.4303
|
|
type: 'test'
|
|
...
|
|
# Subtest: quota failover uses another independent pool member and never replays a failed stream
|
|
ok 66 - quota failover uses another independent pool member and never replays a failed stream
|
|
---
|
|
duration_ms: 1724.5754
|
|
type: 'test'
|
|
...
|
|
# Subtest: native parameters are isolated by pool member, preserve Responses state and bind the original output reservation
|
|
ok 67 - native parameters are isolated by pool member, preserve Responses state and bind the original output reservation
|
|
---
|
|
duration_ms: 0.3412
|
|
type: 'test'
|
|
...
|
|
# Subtest: unsupported native fields and invalid values reject before dispatch instead of silently dropping or overriding constraints
|
|
ok 68 - unsupported native fields and invalid values reject before dispatch instead of silently dropping or overriding constraints
|
|
---
|
|
duration_ms: 0.3406
|
|
type: 'test'
|
|
...
|
|
# Subtest: native settings for legacy and registered profiles persist without changing credentials or old records and reject stale edits
|
|
ok 69 - native settings for legacy and registered profiles persist without changing credentials or old records and reject stale edits
|
|
---
|
|
duration_ms: 899.4035
|
|
type: 'test'
|
|
...
|
|
# Subtest: compiled native settings API saves/restores exact parameters and rejects reserved fields without mutating the profile
|
|
ok 70 - compiled native settings API saves/restores exact parameters and rejects reserved fields without mutating the profile
|
|
---
|
|
duration_ms: 2073.1159
|
|
type: 'test'
|
|
...
|
|
# Subtest: legacy connection test sends its saved native fields and selected output limit field without changing the configuration
|
|
ok 71 - legacy connection test sends its saved native fields and selected output limit field without changing the configuration
|
|
---
|
|
duration_ms: 2002.6654
|
|
type: 'test'
|
|
...
|
|
1..71
|
|
# tests 71
|
|
# suites 0
|
|
# pass 71
|
|
# fail 0
|
|
# cancelled 0
|
|
# skipped 0
|
|
# todo 0
|
|
# duration_ms 62629.5909
|