Files
Agent-Workbench/docs/engineering-evals-de-evidence/0698-evaluation-e-summary-guards-completed.tap
T

95 lines
3.8 KiB
Plaintext

TAP version 13
# Subtest: evaluation denominator distinguishes attempted unsealed candidates from unexecuted samples
ok 1 - evaluation denominator distinguishes attempted unsealed candidates from unexecuted samples
---
duration_ms: 0.7143
type: 'test'
...
# Subtest: evaluation summary retains sealed outcomes and stopped or failed original attempts
ok 2 - evaluation summary retains sealed outcomes and stopped or failed original attempts
---
duration_ms: 0.4023
type: 'test'
...
# Subtest: evaluation contract fixes bounded samples, source paths, checks and aggregate budget
ok 3 - evaluation contract fixes bounded samples, source paths, checks and aggregate budget
---
duration_ms: 535.9338
type: 'test'
...
# Subtest: same-source workspaces seal real commands before execution and preserve original staged and unstaged bytes
ok 4 - same-source workspaces seal real commands before execution and preserve original staged and unstaged bytes
---
duration_ms: 11954.8767
type: 'test'
...
# Subtest: actual original failures and candidate success are both preserved, then adoption requires parent review, checks and Git
ok 5 - actual original failures and candidate success are both preserved, then adoption requires parent review, checks and Git
---
duration_ms: 43840.5683
type: 'test'
...
# Subtest: candidate cannot shrink tests and changing original test bytes rejects execution before any model request
ok 6 - candidate cannot shrink tests and changing original test bytes rejects execution before any model request
---
duration_ms: 17413.8154
type: 'test'
...
# Subtest: missing upstream usage remains unknown and does not become a free run or invented cost
ok 7 - missing upstream usage remains unknown and does not become a free run or invented cost
---
duration_ms: 28071.3939
type: 'test'
...
# Subtest: missing raw check logs preserve unknown result and original sample denominator
ok 8 - missing raw check logs preserve unknown result and original sample denominator
---
duration_ms: 27832.2176
type: 'test'
...
# Subtest: source and model changes never silently replace an existing batch input
ok 9 - source and model changes never silently replace an existing batch input
---
duration_ms: 2356.603
type: 'test'
...
# Subtest: model revision change before preparing a fixed sample cannot silently use the replacement connection
ok 10 - model revision change before preparing a fixed sample cannot silently use the replacement connection
---
duration_ms: 2581.2669
type: 'test'
...
# Subtest: HTTP evaluations require host token, original fingerprint and session ownership; worker API cannot bypass evaluation flow
ok 11 - HTTP evaluations require host token, original fingerprint and session ownership; worker API cannot bypass evaluation flow
---
duration_ms: 13027.238
type: 'test'
...
# Subtest: sample queue cannot skip or reset attempted samples, and reports freeze further writes
ok 12 - sample queue cannot skip or reset attempted samples, and reports freeze further writes
---
duration_ms: 22968.5123
type: 'test'
...
# Subtest: stop cancels only current candidate and preserves partial results instead of replaying a request
ok 13 - stop cancels only current candidate and preserves partial results instead of replaying a request
---
duration_ms: 17775.9072
type: 'test'
...
# Subtest: restart recovers interrupted evaluations as unknown and keeps completed checks and plan identity
ok 14 - restart recovers interrupted evaluations as unknown and keeps completed checks and plan identity
---
duration_ms: 16900.1644
type: 'test'
...
1..14
# tests 14
# suites 0
# pass 14
# fail 0
# cancelled 0
# skipped 0
# todo 0
# duration_ms 205509.5395