Files
Agent-Workbench/docs/engineering-evals-abc-evidence/evaluation-abc-reviewed.tap
T
2026-11-01 09:56:25 +08:00

27 lines
2.2 KiB
Plaintext
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
✔ evaluation contract fixes bounded samples, source paths, checks and aggregate budget (879.7691ms)
✔ same-source workspaces seal real commands before execution and preserve original staged and unstaged bytes (14874.6757ms)
✔ actual original failures and candidate success are both preserved, then adoption requires parent review, checks and Git (49362.4978ms)
✔ candidate cannot shrink tests and changing original test bytes rejects execution before any model request (19490.2731ms)
✔ missing upstream usage remains unknown and does not become a free run or invented cost (28035.4922ms)
✔ missing raw check logs preserve unknown result and original sample denominator (27592.2318ms)
✔ source and model changes never silently replace an existing batch input (2375.6565ms)
✔ model revision change before preparing a fixed sample cannot silently use the replacement connection (2210.0987ms)
✔ HTTP evaluations require host token, original fingerprint and session ownership; worker API cannot bypass evaluation flow (12196.9645ms)
✔ sample queue cannot skip or reset attempted samples, and reports freeze further writes (21512.0204ms)
✔ stop cancels only current candidate and preserves partial results instead of replaying a request (17772.6846ms)
✔ restart recovers interrupted evaluations as unknown and keeps completed checks and plan identity (16171.5361ms)
✔ worker scopes and allocated budgets reject aliases, extra permissions and aggregate overflow (770.0996ms)
✔ two sequential Git workspaces freeze a new parent goal, separate owned tasks and preserve all source bytes (13643.8712ms)
✔ real isolated changes, approvals and npm checks produce original owned outputs, not model completion claims (28402.4151ms)
✔ forged worker tools cannot execute terminal or Git even when absent from the advertised tool list (12969.4039ms)
✔ a completion-only model pauses each worker without fabricated changes or checks (14462.927ms)
✔ restart retains frozen scope and marks interrupted parent and workers unknown without sending requests (12239.9837ms)
ℹ tests 18
ℹ suites 0
ℹ pass 18
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 212727.8273