fix(models): validate engine protocol budgets and freeze release acceptance inputs
This commit is contained in:
@@ -23,3 +23,11 @@ E界面已构建,正在操作验收;F独立发行与正式部署未执行。
|
||||
目视截图发现并修复:片段同批到达后上游停顿,节流需要定时补发最新状态;选池后主窗口可能仍显示离线演示,旧提交校验还错误要求未选中的原连接模型名。现在用公开选路身份同步主窗口,刷新仍保留实际池名,原连接配置不被改写;新UI流程明确等待选路并刷新后发送。下载直接读取工件字节,畸形UTF8也不再被文本解码改写。37项流式/请求计划/升级、20项旧模型中心/API兼容和26项原字节/连接专项通过(重叠范围不累计)。
|
||||
|
||||
原UI定位失败、外部API增加连接后未刷新界面的夹具失败、真实节流缺陷、错误mobile选择器和原首次完整UI运行都保留。首次历史UI全量运行的Purge来源传成打包目录,修正为原source/upstream;原日志与阶段结果单独保存,重跑使用新顶层日志。完整回归早于最后修复启动,属于开发中诊断;最终结论以冻结后的独立源码整轮为准。
|
||||
|
||||
## F:工程门槛和发行冻结前回顾
|
||||
|
||||
固定原Node/React/Python工程通过实际Responses SSE、ModelHub、受控Harness、原审批、原检查、父工程合并和本地Git,共13节点;父工程8/8、9/9、10/10,原测试和用户暂存/未暂存字节保持。原失败cf832704及成功c0121328目录均保留,真实模型请求0。失败发现引擎完成核验只支持body.max_tokens,已逐Chat/Responses实际变体验证原输出预留、原输入SHA及归属,未略过预算核对。新增预算失配回归和协议14/14、Runtime/引擎36/36分别通过,范围重叠不累加。
|
||||
|
||||
完整历史UI25类191组通过(model-protocols-all-ui-validation.json)。开发中整套回归553项552通过/1失败,原日志保留;旧测试把破损JSON预期为工具内错误,现按完整响应契约改为合法JSON的语义参数错误,并新增两个协议整组不执行/不重放回归。最终验收须来自冻结后的独立源码整轮,不把开发中结果当最终发行通过。
|
||||
|
||||
新原生观察已封存protocol-native-before,正式仍1.16;用户期间新增的项目/Task以正式部署前新快照为准,不沿用旧计数、不重放原任务。私有发行执行器只复制到新路径;正式公开模型中心的新默认能力字段将逐项校验,存储行、导出、配置和原工件必须严格保持。封存脚本初次替换顺序拒绝、构建误用未安装的本地npm路径失败均保留;改为限定顺序和固定Node附带npm后构建通过。
|
||||
|
||||
@@ -15,3 +15,9 @@
|
||||
单看控件存在会遗漏主任务问题。截图中主窗口还显示离线演示,导致发现选池/关闭窗口时的外部状态同步与旧legacy模型校验缺陷;现在选择和刷新都保留实际池名,再由同一后端选路执行。流式首批事件合并时曾只显示0字符,增加定时补发最终片段,停止时清除计时器,专门检查上游不再发送任何事件的情形。
|
||||
|
||||
桌面/390px和真实TLS配对观察窗口5组完整流程通过并目视回看。手机原生Safari、信任/PWA已有1.16实机记录;1.17新增流式尚无新实机结论。工程回归继续使用固定原测试和真实Node/React/Python执行,增加同一原工程算法通过Responses SSE与受控Harness的独立门槛,不能拿文本fixture结束当工程完成。
|
||||
|
||||
## F:执行成功还要经过交付证明
|
||||
|
||||
三个真实工程检查通过时,Responses子任务最初仍暂停。原因是旧证据核验把max_tokens当唯一输出预留字段;新请求变体本身和实际执行正确,但核验不兼容。修复后逐变体核对原native.maxTokens、完整请求指标和归属,原审批/失败/成果没有重放,重新独立工程完成。这样的暂停有价值,不能把它当成理由放松交付证明。
|
||||
|
||||
破损JSON整组拒绝与合法JSON参数校验是两个边界,旧Runtime测试现在保留后者,新增前者分别覆盖Chat和Responses及池不重放。完整历史UI191组和原工程通过不替代独立源码发行门槛;正式升级前重新捕获全部用户状态,允许公开能力字段的已知默认增量,禁止把其它差异统一忽略。
|
||||
|
||||
@@ -0,0 +1,209 @@
|
||||
{
|
||||
"success": true,
|
||||
"version": "1.17.0",
|
||||
"at": "2026-10-31T14:54:40.155Z",
|
||||
"groups": 191,
|
||||
"results": [
|
||||
{
|
||||
"script": "verify-model-protocols-ui.mjs",
|
||||
"success": true,
|
||||
"groups": 5,
|
||||
"durationMs": 22412,
|
||||
"evidence": "docs/model-protocols-model-protocols-ui-validation.json",
|
||||
"log": ".qa/model-protocols-verify-model-protocols-ui.log"
|
||||
},
|
||||
{
|
||||
"script": "verify-companion-ui.mjs",
|
||||
"success": true,
|
||||
"groups": 7,
|
||||
"durationMs": 24040,
|
||||
"evidence": "docs/model-protocols-companion-ui-validation.json",
|
||||
"log": ".qa/model-protocols-verify-companion-ui.log"
|
||||
},
|
||||
{
|
||||
"script": "verify-controlled-ui.mjs",
|
||||
"success": true,
|
||||
"groups": 9,
|
||||
"durationMs": 88501,
|
||||
"evidence": "docs/model-protocols-controlled-final-ui-validation.json",
|
||||
"log": ".qa/model-protocols-verify-controlled-ui.log"
|
||||
},
|
||||
{
|
||||
"script": "verify-purge-ui.mjs",
|
||||
"success": true,
|
||||
"groups": 9,
|
||||
"durationMs": 32355,
|
||||
"evidence": "docs/model-protocols-purge-final-ui-validation.json",
|
||||
"log": ".qa/model-protocols-verify-purge-ui.log"
|
||||
},
|
||||
{
|
||||
"script": "verify-workers-ui.mjs",
|
||||
"success": true,
|
||||
"groups": 10,
|
||||
"durationMs": 173264,
|
||||
"evidence": "docs/model-protocols-workers-ui-validation.json",
|
||||
"log": ".qa/model-protocols-verify-workers-ui.log"
|
||||
},
|
||||
{
|
||||
"script": "verify-task-plans-ui.mjs",
|
||||
"success": true,
|
||||
"groups": 8,
|
||||
"durationMs": 17386,
|
||||
"evidence": "docs/model-protocols-task-ui-validation.json",
|
||||
"log": ".qa/model-protocols-verify-task-plans-ui.log"
|
||||
},
|
||||
{
|
||||
"script": "verify-browser-ui.mjs",
|
||||
"success": true,
|
||||
"groups": 10,
|
||||
"durationMs": 27875,
|
||||
"evidence": "docs/model-protocols-browser-ui-validation.json",
|
||||
"log": ".qa/model-protocols-verify-browser-ui.log"
|
||||
},
|
||||
{
|
||||
"script": "verify-ui.mjs",
|
||||
"success": true,
|
||||
"groups": 10,
|
||||
"durationMs": 24543,
|
||||
"evidence": "docs/model-protocols-basic-ui-validation.json",
|
||||
"log": ".qa/model-protocols-verify-ui.log"
|
||||
},
|
||||
{
|
||||
"script": "verify-diagnostics-ui.mjs",
|
||||
"success": true,
|
||||
"groups": 9,
|
||||
"durationMs": 23892,
|
||||
"evidence": "docs/model-protocols-diagnostics-ui-validation.json",
|
||||
"log": ".qa/model-protocols-verify-diagnostics-ui.log"
|
||||
},
|
||||
{
|
||||
"script": "verify-execution-ui.mjs",
|
||||
"success": true,
|
||||
"groups": 7,
|
||||
"durationMs": 32631,
|
||||
"evidence": "docs/model-protocols-execution-ui-validation.json",
|
||||
"log": ".qa/model-protocols-verify-execution-ui.log"
|
||||
},
|
||||
{
|
||||
"script": "verify-delivery-ui.mjs",
|
||||
"success": true,
|
||||
"groups": 9,
|
||||
"durationMs": 56215,
|
||||
"evidence": "docs/model-protocols-delivery-ui-validation.json",
|
||||
"log": ".qa/model-protocols-verify-delivery-ui.log"
|
||||
},
|
||||
{
|
||||
"script": "verify-index-ui.mjs",
|
||||
"success": true,
|
||||
"groups": 8,
|
||||
"durationMs": 8665,
|
||||
"evidence": "docs/model-protocols-index-ui-validation.json",
|
||||
"log": ".qa/model-protocols-verify-index-ui.log"
|
||||
},
|
||||
{
|
||||
"script": "verify-change-sets-ui.mjs",
|
||||
"success": true,
|
||||
"groups": 11,
|
||||
"durationMs": 16725,
|
||||
"evidence": "docs/model-protocols-change-sets-ui-validation.json",
|
||||
"log": ".qa/model-protocols-verify-change-sets-ui.log"
|
||||
},
|
||||
{
|
||||
"script": "verify-context-ui.mjs",
|
||||
"success": true,
|
||||
"groups": 11,
|
||||
"durationMs": 25214,
|
||||
"evidence": "docs/model-protocols-context-ui-validation.json",
|
||||
"log": ".qa/model-protocols-verify-context-ui.log"
|
||||
},
|
||||
{
|
||||
"script": "verify-workspaces-ui.mjs",
|
||||
"success": true,
|
||||
"groups": 9,
|
||||
"durationMs": 30360,
|
||||
"evidence": "docs/model-protocols-workspaces-ui-validation.json",
|
||||
"log": ".qa/model-protocols-verify-workspaces-ui.log"
|
||||
},
|
||||
{
|
||||
"script": "verify-python-ui.mjs",
|
||||
"success": true,
|
||||
"groups": 7,
|
||||
"durationMs": 36071,
|
||||
"evidence": "docs/model-protocols-python-ui-validation.json",
|
||||
"log": ".qa/model-protocols-verify-python-ui.log"
|
||||
},
|
||||
{
|
||||
"script": "verify-react-delivery.mjs",
|
||||
"success": true,
|
||||
"groups": 6,
|
||||
"durationMs": 62073,
|
||||
"evidence": "docs/model-protocols-react-delivery-validation.json",
|
||||
"log": ".qa/model-protocols-verify-react-delivery.log"
|
||||
},
|
||||
{
|
||||
"script": "verify-research-ui.mjs",
|
||||
"success": true,
|
||||
"groups": 11,
|
||||
"durationMs": 39718,
|
||||
"evidence": "docs/model-protocols-research-ui-validation.json",
|
||||
"log": ".qa/model-protocols-verify-research-ui.log"
|
||||
},
|
||||
{
|
||||
"script": "verify-engines-ui.mjs",
|
||||
"success": true,
|
||||
"groups": 10,
|
||||
"durationMs": 18470,
|
||||
"evidence": "docs/model-protocols-engines-ui-validation.json",
|
||||
"log": ".qa/model-protocols-verify-engines-ui.log"
|
||||
},
|
||||
{
|
||||
"script": "verify-model-center-ui.mjs",
|
||||
"success": true,
|
||||
"groups": 6,
|
||||
"durationMs": 14851,
|
||||
"evidence": "docs/model-protocols-model-center-ui-validation.json",
|
||||
"log": ".qa/model-protocols-verify-model-center-ui.log"
|
||||
},
|
||||
{
|
||||
"script": "verify-project-memory-ui.mjs",
|
||||
"success": true,
|
||||
"groups": 4,
|
||||
"durationMs": 9629,
|
||||
"evidence": "docs/model-protocols-project-memory-ui-validation.json",
|
||||
"log": ".qa/model-protocols-verify-project-memory-ui.log"
|
||||
},
|
||||
{
|
||||
"script": "verify-task-checkpoints-ui.mjs",
|
||||
"success": true,
|
||||
"groups": 3,
|
||||
"durationMs": 10062,
|
||||
"evidence": "docs/model-protocols-task-checkpoints-ui-validation.json",
|
||||
"log": ".qa/model-protocols-verify-task-checkpoints-ui.log"
|
||||
},
|
||||
{
|
||||
"script": "verify-skills-ui.mjs",
|
||||
"success": true,
|
||||
"groups": 4,
|
||||
"durationMs": 9400,
|
||||
"evidence": "docs/model-protocols-skills-ui-validation.json",
|
||||
"log": ".qa/model-protocols-verify-skills-ui.log"
|
||||
},
|
||||
{
|
||||
"script": "verify-mcp-ui.mjs",
|
||||
"success": true,
|
||||
"groups": 6,
|
||||
"durationMs": 12386,
|
||||
"evidence": "docs/model-protocols-mcp-ui-validation.json",
|
||||
"log": ".qa/model-protocols-verify-mcp-ui.log"
|
||||
},
|
||||
{
|
||||
"script": "verify-mcp-recovery-ui.mjs",
|
||||
"success": true,
|
||||
"groups": 2,
|
||||
"durationMs": 7948,
|
||||
"evidence": "docs/model-protocols-mcp-recovery-ui-validation.json",
|
||||
"log": ".qa/model-protocols-verify-mcp-recovery-ui.log"
|
||||
}
|
||||
],
|
||||
"scope": "Every verifier owns an independent port and synthetic data directory. Synthetic provider fixtures verify protocols; configured-model acceptance is separate. No production service or real key read/copied."
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
{
|
||||
"success": true,
|
||||
"url": "http://127.0.0.1:62053",
|
||||
"browser": "Microsoft Edge 156.0.4314.50",
|
||||
"date": "2026-10-31T14:47:44.485Z",
|
||||
"projectRoot": "D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\ui-64a4aaa1-6579-409f-88da-70e034ec84fb\\ui-project",
|
||||
"viewports": [
|
||||
1440,
|
||||
780,
|
||||
390
|
||||
],
|
||||
"passed": [
|
||||
"project selection and real file preview",
|
||||
"real command consent, execution and stale snapshot rejection",
|
||||
"diff review and verified file application",
|
||||
"manual editor creates a reviewable proposal",
|
||||
"file responses stay with the current selection",
|
||||
"denial, session selection, reload, stale response rejection and export",
|
||||
"project inventory responses cannot replace another project",
|
||||
"780 and 390 px: project drawer, file, diff and trace controls, no page overflow",
|
||||
"unified and detailed research reports render",
|
||||
"model list, ephemeral key, compatible fixture tool loop and return to offline mode"
|
||||
],
|
||||
"pageErrors": [],
|
||||
"requestFailures": [],
|
||||
"modelValidation": "controlled local HTTP fixture; no real model"
|
||||
}
|
||||
@@ -0,0 +1,29 @@
|
||||
{
|
||||
"success": true,
|
||||
"version": "1.17.0",
|
||||
"testedAt": "2026-10-31T14:50:03.375Z",
|
||||
"cases": [
|
||||
"Desktop batch scope, per-file diffs, sealed single-write guard and pre-apply bytes",
|
||||
"Three-file reviewed application preserves exact BOM/CRLF/LF and creates the new file",
|
||||
"Explicit batch rollback restores exact originals and removes only the reviewed new file",
|
||||
"A last-file owner conflict blocks the whole batch and shows the real recovery direction",
|
||||
"Durable partial recovery reads actual hashes and preserves owner content during rollback",
|
||||
"Changed observation fingerprints reject stale acknowledgement without releasing the project",
|
||||
"Explicit owner-content acknowledgement ends recovery without claiming successful application",
|
||||
"Switching projects clears the previous task notification while retaining its durable error and recovery history",
|
||||
"390px step-limit review, same-task continuation entry, rollback confirmation and keyboard close fit the viewport",
|
||||
"A delayed preparation response never opens a different project or changes its selected session",
|
||||
"A failed batch response after its component unmounts cannot display an error in the newly selected project"
|
||||
],
|
||||
"pageErrors": [],
|
||||
"requestFailures": [],
|
||||
"screenshots": [
|
||||
".qa/screenshots/model-protocols-batch-desktop-review.png",
|
||||
".qa/screenshots/model-protocols-batch-desktop-conflict.png",
|
||||
".qa/screenshots/model-protocols-batch-desktop-recovery.png",
|
||||
".qa/screenshots/model-protocols-batch-mobile-review.png",
|
||||
".qa/screenshots/model-protocols-batch-mobile-rollback.png"
|
||||
],
|
||||
"projectRoot": "D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\batch-ui-40c92fcb-1295-4f58-80f8-fbd6efcaff0c",
|
||||
"scope": "Real independent demo host and durable batch states; UI fixtures include actual owner conflict/rollback results. No real model credentials or user project were used."
|
||||
}
|
||||
@@ -0,0 +1,175 @@
|
||||
{
|
||||
"success": true,
|
||||
"version": "1.17.0",
|
||||
"testedAt": "2026-10-31T14:47:19.835Z",
|
||||
"directory": "D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\browser-ui-358841cd-bd07-4cde-9eff-0622bd53b2f1",
|
||||
"passed": [
|
||||
"actual step editor seals the chosen build, viewport and three CSS interactions without opening the page",
|
||||
"review shows exact build identity and planned assertions; cancelling executes no browser work",
|
||||
"real inner Edge clicks the intended element and the two counted assertions display with original Task/Run/build identity",
|
||||
"original large JSON is paged in the UI and downloads the complete hash-identical result",
|
||||
"the original screenshot renders at its actual resolution and downloads identical PNG bytes",
|
||||
"a real failed assertion retains actual count one and requires a newly reviewed plan instead of replaying",
|
||||
"390px forms and result records remain usable and the inner browser captures its real 390px layout",
|
||||
"stopping an actual in-flight assertion closes its owned browser and preserves prior successful and failed jobs",
|
||||
"a delayed original-record failure stays with its source and does not appear in a newly selected task",
|
||||
"actual host restart preserves the four original browser jobs and PNGs without model requests or execution replay"
|
||||
],
|
||||
"pageErrors": [],
|
||||
"requestFailures": [],
|
||||
"screenshots": [
|
||||
"D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\browser-ui-358841cd-bd07-4cde-9eff-0622bd53b2f1\\browser-desktop-results.png",
|
||||
"D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\browser-ui-358841cd-bd07-4cde-9eff-0622bd53b2f1\\browser-desktop-original-image.png",
|
||||
"D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\browser-ui-358841cd-bd07-4cde-9eff-0622bd53b2f1\\browser-mobile-results.png",
|
||||
"D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\browser-ui-358841cd-bd07-4cde-9eff-0622bd53b2f1\\browser-mobile-original-image.png"
|
||||
],
|
||||
"taskId": "a32f6bf7-5a6a-4a3b-b675-3bf7bb07cc08",
|
||||
"sessionId": "b0266472-8582-4fe4-9bf7-364c0e968c83",
|
||||
"jobs": [
|
||||
{
|
||||
"id": "0a10a4b2-814f-46ad-bb39-c6363270746b",
|
||||
"status": "passed",
|
||||
"planId": "ff4c8e21-5ace-4e40-a79f-ba3f0b13fb7e",
|
||||
"assertions": {
|
||||
"total": 2,
|
||||
"passed": 2,
|
||||
"failed": 0
|
||||
},
|
||||
"resultSha256": "95db2e4c3107cb6a2f9cacf63e79857f64a1e9e49be438a8ef53befb40253e93",
|
||||
"cleanup": {
|
||||
"pid": 32508,
|
||||
"exited": true,
|
||||
"exitCode": 0,
|
||||
"signal": null,
|
||||
"finishedAt": "2026-10-31T14:46:59.993Z",
|
||||
"previewClosed": true,
|
||||
"snapshotDisposed": true
|
||||
},
|
||||
"screenshots": [
|
||||
{
|
||||
"artifactId": "23f0e58c-b208-4ecd-a998-8f744c326ab0",
|
||||
"sha256": "74f2387b837f5f74bd7e8eef7631865a3b5bb24601c4bbaa93b2478a9e94c6ef",
|
||||
"bytes": 101061,
|
||||
"width": 1280,
|
||||
"height": 900,
|
||||
"label": "initial"
|
||||
},
|
||||
{
|
||||
"artifactId": "39c484dd-afdf-40f5-a804-f609ba8505b6",
|
||||
"sha256": "b1de525bcf042bc1724e8a9c6670042ab3035bed394ff76cf2a934a443ae98f7",
|
||||
"bytes": 85261,
|
||||
"width": 1280,
|
||||
"height": 900,
|
||||
"label": "final"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "08e97273-09d1-4ae3-b61e-ee2dbc75989e",
|
||||
"status": "failed",
|
||||
"planId": "1d9361b2-c74c-405f-8c32-a87f5d75d6e0",
|
||||
"assertions": {
|
||||
"total": 2,
|
||||
"passed": 1,
|
||||
"failed": 1
|
||||
},
|
||||
"resultSha256": "4853092c9b85e0d4d7768e5d26f88d11a494b6eab18ab4043efbf16bda618c65",
|
||||
"cleanup": {
|
||||
"pid": 12784,
|
||||
"exited": true,
|
||||
"exitCode": 0,
|
||||
"signal": null,
|
||||
"finishedAt": "2026-10-31T14:47:07.343Z",
|
||||
"previewClosed": true,
|
||||
"snapshotDisposed": true
|
||||
},
|
||||
"screenshots": [
|
||||
{
|
||||
"artifactId": "f5fedd02-bf03-4933-9743-3658e53d699c",
|
||||
"sha256": "74f2387b837f5f74bd7e8eef7631865a3b5bb24601c4bbaa93b2478a9e94c6ef",
|
||||
"bytes": 101061,
|
||||
"width": 1280,
|
||||
"height": 900,
|
||||
"label": "initial"
|
||||
},
|
||||
{
|
||||
"artifactId": "4cad830b-c171-425c-b34e-d719423b3dd5",
|
||||
"sha256": "b1de525bcf042bc1724e8a9c6670042ab3035bed394ff76cf2a934a443ae98f7",
|
||||
"bytes": 85261,
|
||||
"width": 1280,
|
||||
"height": 900,
|
||||
"label": "failure"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "ebe899f1-66c1-456f-85ae-5a41c82b75f0",
|
||||
"status": "passed",
|
||||
"planId": "1c193b34-df7f-4b99-94c1-272e31cd2cd5",
|
||||
"assertions": {
|
||||
"total": 2,
|
||||
"passed": 2,
|
||||
"failed": 0
|
||||
},
|
||||
"resultSha256": "ecb8b9a9aedb31373fcf335d0e573a4d368cdac981615677bbc58725b3ad80bb",
|
||||
"cleanup": {
|
||||
"pid": 34296,
|
||||
"exited": true,
|
||||
"exitCode": 0,
|
||||
"signal": null,
|
||||
"finishedAt": "2026-10-31T14:47:11.189Z",
|
||||
"previewClosed": true,
|
||||
"snapshotDisposed": true
|
||||
},
|
||||
"screenshots": [
|
||||
{
|
||||
"artifactId": "f52b25da-c226-4180-b15d-1f47c67f651c",
|
||||
"sha256": "93969f1954eec1f7fa4736d7a3035631c6066b6727132ff1179180eadb745602",
|
||||
"bytes": 11011,
|
||||
"width": 390,
|
||||
"height": 844,
|
||||
"label": "initial"
|
||||
},
|
||||
{
|
||||
"artifactId": "d1ba441f-599c-4c33-85f7-bfa4dd495a27",
|
||||
"sha256": "a6908ecfcfd239721ec6bbb916a5054d2f61183b6dea3f014db89216d371d0e3",
|
||||
"bytes": 18832,
|
||||
"width": 390,
|
||||
"height": 844,
|
||||
"label": "final"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "d5483e7a-3283-453c-9b2d-db645c359edf",
|
||||
"status": "interrupted",
|
||||
"planId": "54f5fed3-d482-4ad8-a66e-87c8a271ab98",
|
||||
"assertions": {
|
||||
"total": 3,
|
||||
"passed": 2,
|
||||
"failed": 1
|
||||
},
|
||||
"resultSha256": "8668100df79ec25574a67bf0d1538ec5b7dd46d959a5c85c05f59dbd037f9109",
|
||||
"cleanup": {
|
||||
"pid": 38520,
|
||||
"exited": true,
|
||||
"exitCode": 0,
|
||||
"signal": null,
|
||||
"finishedAt": "2026-10-31T14:47:15.955Z",
|
||||
"previewClosed": true,
|
||||
"snapshotDisposed": true
|
||||
},
|
||||
"screenshots": [
|
||||
{
|
||||
"artifactId": "aaaa0da4-16a2-4124-b475-a8aa15ef8502",
|
||||
"sha256": "93969f1954eec1f7fa4736d7a3035631c6066b6727132ff1179180eadb745602",
|
||||
"bytes": 11011,
|
||||
"width": 390,
|
||||
"height": 844,
|
||||
"label": "initial"
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"scope": "Actual independent Workbench UI plus real Edge static-build executions. The configured production Provider, other projects and keys are not accessed."
|
||||
}
|
||||
@@ -0,0 +1,29 @@
|
||||
{
|
||||
"success": true,
|
||||
"version": "1.17.0",
|
||||
"testedAt": "2026-10-31T14:50:03.375Z",
|
||||
"cases": [
|
||||
"Desktop batch scope, per-file diffs, sealed single-write guard and pre-apply bytes",
|
||||
"Three-file reviewed application preserves exact BOM/CRLF/LF and creates the new file",
|
||||
"Explicit batch rollback restores exact originals and removes only the reviewed new file",
|
||||
"A last-file owner conflict blocks the whole batch and shows the real recovery direction",
|
||||
"Durable partial recovery reads actual hashes and preserves owner content during rollback",
|
||||
"Changed observation fingerprints reject stale acknowledgement without releasing the project",
|
||||
"Explicit owner-content acknowledgement ends recovery without claiming successful application",
|
||||
"Switching projects clears the previous task notification while retaining its durable error and recovery history",
|
||||
"390px step-limit review, same-task continuation entry, rollback confirmation and keyboard close fit the viewport",
|
||||
"A delayed preparation response never opens a different project or changes its selected session",
|
||||
"A failed batch response after its component unmounts cannot display an error in the newly selected project"
|
||||
],
|
||||
"pageErrors": [],
|
||||
"requestFailures": [],
|
||||
"screenshots": [
|
||||
".qa/screenshots/model-protocols-batch-desktop-review.png",
|
||||
".qa/screenshots/model-protocols-batch-desktop-conflict.png",
|
||||
".qa/screenshots/model-protocols-batch-desktop-recovery.png",
|
||||
".qa/screenshots/model-protocols-batch-mobile-review.png",
|
||||
".qa/screenshots/model-protocols-batch-mobile-rollback.png"
|
||||
],
|
||||
"projectRoot": "D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\batch-ui-40c92fcb-1295-4f58-80f8-fbd6efcaff0c",
|
||||
"scope": "Real independent demo host and durable batch states; UI fixtures include actual owner conflict/rollback results. No real model credentials or user project were used."
|
||||
}
|
||||
@@ -0,0 +1,42 @@
|
||||
{
|
||||
"success": true,
|
||||
"version": "1.17.0",
|
||||
"appRoot": "D:\\Ai-project\\REA逆向\\Agent-Workbench",
|
||||
"nodePath": "D:\\Ai-project\\REA逆向\\tools\\node-v24.22.1-win-x64\\node.exe",
|
||||
"nodeSha256": "d702daa815eabbff763c7d3808e235cf031cf2ca70211b56847bc72f51e240e5",
|
||||
"containedRuntime": false,
|
||||
"passed": [
|
||||
"desktop explicitly enables HTTPS and downloads the actual public CA without changing OS trust",
|
||||
"390px real gateway pairing clears fragment, displays scoped task list and registers the same device in desktop",
|
||||
"phone UI confirms original Task/Run approval and the existing Runtime executes one command",
|
||||
"original paginated diff and counted check/report display verified content from the same Task",
|
||||
"disconnect and refresh recover from persistent seq without submitting a new command or Run",
|
||||
"PWA service worker stores the shell and no API/project response",
|
||||
"desktop revocation removes mobile project content immediately on the next read while preserving the original completed tool"
|
||||
],
|
||||
"screenshots": [
|
||||
"D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\companion-ui-f35292a9-e3d2-4433-8d99-f9518b3b48a5\\desktop-certificate.png",
|
||||
"D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\companion-ui-f35292a9-e3d2-4433-8d99-f9518b3b48a5\\desktop-pairing.png",
|
||||
"D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\companion-ui-f35292a9-e3d2-4433-8d99-f9518b3b48a5\\mobile-task-list.png",
|
||||
"D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\companion-ui-f35292a9-e3d2-4433-8d99-f9518b3b48a5\\mobile-original-approval.png",
|
||||
"D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\companion-ui-f35292a9-e3d2-4433-8d99-f9518b3b48a5\\mobile-original-diff.png",
|
||||
"D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\companion-ui-f35292a9-e3d2-4433-8d99-f9518b3b48a5\\mobile-original-report.png",
|
||||
"D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\companion-ui-f35292a9-e3d2-4433-8d99-f9518b3b48a5\\mobile-disconnected.png",
|
||||
"D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\companion-ui-f35292a9-e3d2-4433-8d99-f9518b3b48a5\\tablet-task-list.png",
|
||||
"D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\companion-ui-f35292a9-e3d2-4433-8d99-f9518b3b48a5\\desktop-management-narrow.png",
|
||||
"D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\companion-ui-f35292a9-e3d2-4433-8d99-f9518b3b48a5\\mobile-revoked.png"
|
||||
],
|
||||
"errors": [],
|
||||
"viewports": [
|
||||
1440,
|
||||
390,
|
||||
768
|
||||
],
|
||||
"browser": "Microsoft Edge via Playwright; owned loopback QA relay with actual CA verification",
|
||||
"nativeSafariTrustCookiePWA": "pending-real-iPhone",
|
||||
"osTrustChanged": false,
|
||||
"certificateWarningsBypassed": false,
|
||||
"realModelRequests": 0,
|
||||
"taskId": "82c87cf5-c7ef-480f-8a38-4b9158836ebe",
|
||||
"runId": "60f3e204-241e-4e29-be4c-ed4ad1d5612e"
|
||||
}
|
||||
@@ -0,0 +1,34 @@
|
||||
{
|
||||
"success": true,
|
||||
"version": "1.17.0",
|
||||
"testedAt": "2026-10-31T14:50:27.484Z",
|
||||
"passed": [
|
||||
"desktop request estimate, actual usage, complete goal and history identity are visible",
|
||||
"paged complete history opens actual tool result artifacts without executing tools",
|
||||
"historical and resumed Runs retain their own request records",
|
||||
"390px history, original JSON and sent-body dialogs fit and retain close controls",
|
||||
"late old history failure stays in its original scope after switching projects",
|
||||
"late old Run failure stays in its original scope after switching projects",
|
||||
"late old request-body failure stays in its original scope after switching projects",
|
||||
"context-limit pause explains the intact requirement and explicit continuation",
|
||||
"390px repeated-read pause explains the cause and explicit continuation retains the same Task",
|
||||
"desktop working evidence shows actual code, original result provenance and the current Task",
|
||||
"390px working evidence and sent request stay readable and closing the artifact retains task navigation"
|
||||
],
|
||||
"pageErrors": [],
|
||||
"requestFailures": [],
|
||||
"screenshots": [
|
||||
".qa/screenshots/model-protocols-desktop-plan.png",
|
||||
".qa/screenshots/model-protocols-desktop-history.png",
|
||||
".qa/screenshots/model-protocols-mobile-history.png",
|
||||
".qa/screenshots/model-protocols-mobile-original.png",
|
||||
".qa/screenshots/model-protocols-mobile-request-body.png",
|
||||
".qa/screenshots/model-protocols-mobile-read-pause.png",
|
||||
".qa/screenshots/model-protocols-desktop-working-evidence.png",
|
||||
".qa/screenshots/model-protocols-mobile-working-evidence.png"
|
||||
],
|
||||
"root": "D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\context-ui-fdfeefbd-01db-42ff-ad26-7c487369cc1c",
|
||||
"data": "D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\context-ui-fdfeefbd-01db-42ff-ad26-7c487369cc1c\\data",
|
||||
"mode": "saved synthetic context evidence; no real model or project commands executed by this UI verifier",
|
||||
"browser": "156.0.4314.50"
|
||||
}
|
||||
File diff suppressed because one or more lines are too long
@@ -0,0 +1,31 @@
|
||||
{
|
||||
"success": true,
|
||||
"version": "1.17.0",
|
||||
"testedAt": "2026-10-31T14:49:37.535Z",
|
||||
"browser": "156.0.4314.50",
|
||||
"passed": [
|
||||
"actual failing tests and four source proposals retain the original project and task",
|
||||
"reviewed changes continue the same Task through 14 real tests and model-prepared Git review",
|
||||
"review shows actual paths, diffs, tests, author and policy; cancelling leaves Git unchanged",
|
||||
"an index change after confirmation preview refuses the stale job and preserves both staged owners",
|
||||
"390px confirmation keeps file hashes, diffs and actions inside the modal",
|
||||
"double confirmation creates one four-file commit, retains owner bytes and publishes report with completion",
|
||||
"report viewing and Markdown/JSON downloads preserve verified artifact bytes and truthful acceptance limits",
|
||||
"mobile report preview closes cleanly and keeps the task input fixed",
|
||||
"unknown Git results block new submission and the explicit reconcile action restores the existing commit"
|
||||
],
|
||||
"pageErrors": [],
|
||||
"requestFailures": [],
|
||||
"projectRoot": "D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\delivery-ui-befaa1a4-b7b3-4391-bf8d-368558b56414\\四模块交付工程",
|
||||
"projectId": "adcccec2-db95-4a76-8f5f-34e88d210d81",
|
||||
"sessionId": "590e7a1c-1018-4950-9101-11ae784b9881",
|
||||
"taskId": "72602590-91b9-472d-815a-cca3825b37d4",
|
||||
"commit": "577bb8753846b44e3c71d3dc6668d045f835551b",
|
||||
"recoveryCommit": "de88c98edbb6c802133d24efa0e32fe67405fde3",
|
||||
"testCounts": {
|
||||
"baselineFailed": 13,
|
||||
"passed": 14
|
||||
},
|
||||
"model": "controlled local HTTP fixture",
|
||||
"confirmations": 2
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
{
|
||||
"success": true,
|
||||
"version": "1.17.0",
|
||||
"url": "http://127.0.0.1:65507",
|
||||
"date": "2026-10-31T14:48:08.720Z",
|
||||
"passed": [
|
||||
"model response validation checks the tentative connection before saving",
|
||||
"a model failure after successful tools shows source, attempts and local runtime diagnosis",
|
||||
"explicit retry preserves the completed tool result without replaying the task",
|
||||
"an error question receives the actual session diagnosis and formatted answer",
|
||||
"review state exposes the review action and prevents application during a running request",
|
||||
"approved Unicode commands execute in the declared shell and retain Chinese output",
|
||||
"authentication errors do not retry and error/diagnostic controls fit a 390 px viewport",
|
||||
"stop cancels pending retry and clears running model state",
|
||||
"a transient service failure recovers within the bounded retry policy"
|
||||
],
|
||||
"pageErrors": [],
|
||||
"requestFailures": [],
|
||||
"modelValidation": "controlled local HTTP fixture; actual configured model is verified separately"
|
||||
}
|
||||
@@ -0,0 +1,232 @@
|
||||
{
|
||||
"success": true,
|
||||
"version": "1.17.0",
|
||||
"testedAt": "2026-10-31T14:53:35.280Z",
|
||||
"root": "D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\engines-ui-2c53bb6f-73ae-430d-8e02-619431880326",
|
||||
"browser": "156.0.4314.50",
|
||||
"passed": [
|
||||
"independent desktop engine entry detects the declared installation and clearly separates connection from project execution",
|
||||
"missing installations and edited directories cannot launch a check until detection succeeds",
|
||||
"UI launch seals the selected installation into the current Task/Run and renders actual startup and process-close facts",
|
||||
"original record paging, full download SHA256 and nested-dialog keyboard close preserve the exact evidence",
|
||||
"an explicit new check creates a different process and record while retaining the original check unchanged",
|
||||
"declared pending-initialize fixture stops its owned process and explicit retry preserves the stopped result",
|
||||
"the same product UI verifies the actual installed DeepSeek SDK without a prompt or configured-model call",
|
||||
"390px installation paths, source chain, original record, paging and keyboard close remain readable without page overflow",
|
||||
"late detection failure from a closed panel cannot appear in a different task or reveal its prior engine records",
|
||||
"actual host restart preserves all native and declared-fixture records with no protocol replay or project modification"
|
||||
],
|
||||
"pageErrors": [],
|
||||
"requestFailures": [],
|
||||
"expectedRequestFailures": [],
|
||||
"screenshots": [
|
||||
"D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\engines-ui-2c53bb6f-73ae-430d-8e02-619431880326\\screenshots\\engines-native-desktop.png",
|
||||
"D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\engines-ui-2c53bb6f-73ae-430d-8e02-619431880326\\screenshots\\engines-record-mobile.png",
|
||||
"D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\engines-ui-2c53bb6f-73ae-430d-8e02-619431880326\\screenshots\\engines-native-mobile.png"
|
||||
],
|
||||
"starts": [
|
||||
{
|
||||
"root": "D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\engines-ui-2c53bb6f-73ae-430d-8e02-619431880326\\本地 SDK #",
|
||||
"identitySha256": "b347cc276b525b570cf86db8a7f0fc26a1006a41e88c1777517db59715c5f85f"
|
||||
},
|
||||
{
|
||||
"root": "D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\engines-ui-2c53bb6f-73ae-430d-8e02-619431880326\\本地 SDK #",
|
||||
"retry": true
|
||||
},
|
||||
{
|
||||
"root": "D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\engines-ui-2c53bb6f-73ae-430d-8e02-619431880326\\本地 SDK #",
|
||||
"identitySha256": "b347cc276b525b570cf86db8a7f0fc26a1006a41e88c1777517db59715c5f85f"
|
||||
},
|
||||
{
|
||||
"root": "D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\engines-ui-2c53bb6f-73ae-430d-8e02-619431880326\\本地 SDK #",
|
||||
"retry": true
|
||||
},
|
||||
{
|
||||
"root": "D:\\Ai-project\\DeepSeek-Harness-Desktop\\deepseek-harness",
|
||||
"identitySha256": "65483d7f8aa644d890100a22bc52555ce91c009ba2d213b510f3da1a9ede7ff3"
|
||||
}
|
||||
],
|
||||
"nativeJob": {
|
||||
"schemaVersion": 1,
|
||||
"id": "a3084ece-f45c-40c8-b130-4d221b73b4f5",
|
||||
"sessionId": "f2ea193c-90b1-44ee-b48b-11746b6639e7",
|
||||
"projectId": "1f0114fb-5cf6-4d70-ae5c-9983dfa2388e",
|
||||
"taskId": "ce73a541-54ed-4d57-a35e-a731eb6c584b",
|
||||
"runId": "6c9eef36-dedb-41c7-9855-ef6391316380",
|
||||
"engineId": "deepseek-harness",
|
||||
"scope": "deepseek-sdk-connection-probe",
|
||||
"source": "user",
|
||||
"installation": {
|
||||
"engineId": "deepseek-harness",
|
||||
"name": "DeepSeek Harness",
|
||||
"profile": "sdk-minimal",
|
||||
"expectedVersion": "0.2.1-rc.2",
|
||||
"root": "D:\\Ai-project\\DeepSeek-Harness-Desktop\\deepseek-harness",
|
||||
"capabilities": {
|
||||
"connectionProbe": true,
|
||||
"projectExecution": false,
|
||||
"prompt": false,
|
||||
"singleSessionCancel": false,
|
||||
"protocolVersionNegotiation": false,
|
||||
"cancellationUnit": "owned-process",
|
||||
"tokenStreaming": false
|
||||
},
|
||||
"limitations": [
|
||||
"当前接入核验启动、会话目录、协议错误和关闭;工程执行接入开发中。",
|
||||
"SDK没有单会话cancel或close;停止以所属独立进程为单位。",
|
||||
"SDK serverInfo.version与发行包版本分别核对,没有有效协议版本协商。",
|
||||
"SDK事件覆盖运行时会话树,idle事件不能直接判项目交付完成。"
|
||||
],
|
||||
"status": "available",
|
||||
"version": "0.2.1-rc.2",
|
||||
"entry": "D:\\Ai-project\\DeepSeek-Harness-Desktop\\deepseek-harness\\apps\\cli\\lib\\bin.js",
|
||||
"inputs": [
|
||||
{
|
||||
"path": "D:\\Ai-project\\DeepSeek-Harness-Desktop\\deepseek-harness\\apps\\cli\\package.json",
|
||||
"bytes": 8138,
|
||||
"sha256": "d42efe9f3ff424f602f11598af8805ad25039953ebbbd3c163d67875f7e8658b",
|
||||
"relative": "apps/cli/package.json"
|
||||
},
|
||||
{
|
||||
"path": "D:\\Ai-project\\DeepSeek-Harness-Desktop\\deepseek-harness\\apps\\cli\\lib\\bin.js",
|
||||
"bytes": 11772,
|
||||
"sha256": "80112efe368efb1a39720c7ada0af00442e5131fb2744edaaff8a3e8bb499e6f",
|
||||
"relative": "apps/cli/lib/bin.js"
|
||||
},
|
||||
{
|
||||
"path": "D:\\Ai-project\\DeepSeek-Harness-Desktop\\deepseek-harness\\packages\\boot\\cmdline\\lib\\index.js",
|
||||
"bytes": 6973,
|
||||
"sha256": "7253190611566485f8b6f836fae6e0f2a7abbc59bb807c42e874844016e6d3a0",
|
||||
"relative": "packages/boot/cmdline/lib/index.js"
|
||||
},
|
||||
{
|
||||
"path": "D:\\Ai-project\\DeepSeek-Harness-Desktop\\deepseek-harness\\packages\\bundle\\sdk-app\\lib\\index.js",
|
||||
"bytes": 1921,
|
||||
"sha256": "cc7e6c38b0b38c0e92a0dffb45210c1dac6ef518547436e805e84b772f91a475",
|
||||
"relative": "packages/bundle/sdk-app/lib/index.js"
|
||||
},
|
||||
{
|
||||
"path": "D:\\Ai-project\\DeepSeek-Harness-Desktop\\deepseek-harness\\packages\\bundle\\sdk-minimal\\package.json",
|
||||
"bytes": 2920,
|
||||
"sha256": "0e177cbca85148207343db5afd7f12829723cf3f2afedfa8e85a5c5dcd58f604",
|
||||
"relative": "packages/bundle/sdk-minimal/package.json"
|
||||
},
|
||||
{
|
||||
"path": "D:\\Ai-project\\DeepSeek-Harness-Desktop\\deepseek-harness\\packages\\bundle\\sdk-minimal\\cordis.patch.yml",
|
||||
"bytes": 5636,
|
||||
"sha256": "464ea07c03d3639c2c387fa3a684f5d4001e7ca39f8257b361c604f19487af66",
|
||||
"relative": "packages/bundle/sdk-minimal/cordis.patch.yml"
|
||||
},
|
||||
{
|
||||
"path": "D:\\Ai-project\\DeepSeek-Harness-Desktop\\deepseek-harness\\packages\\sdk\\server\\package.json",
|
||||
"bytes": 2155,
|
||||
"sha256": "2690f99763c3e1e7bd57f680f26e53010c2a3d7c4dffe06ff64a07ae3928cb17",
|
||||
"relative": "packages/sdk/server/package.json"
|
||||
},
|
||||
{
|
||||
"path": "D:\\Ai-project\\DeepSeek-Harness-Desktop\\deepseek-harness\\packages\\sdk\\server\\lib\\index.js",
|
||||
"bytes": 16454,
|
||||
"sha256": "55412923f9f7c279f7e0b2a1e2853eaf238f390b3d84046cb9c94dc5f22a1e4a",
|
||||
"relative": "packages/sdk/server/lib/index.js"
|
||||
},
|
||||
{
|
||||
"path": "D:\\Ai-project\\DeepSeek-Harness-Desktop\\deepseek-harness\\packages\\sdk\\protocol\\package.json",
|
||||
"bytes": 1237,
|
||||
"sha256": "b030b0431d69723b5424dc2519cd4b381ec91acd22b73e0b30d6b8a19f1f2f94",
|
||||
"relative": "packages/sdk/protocol/package.json"
|
||||
},
|
||||
{
|
||||
"path": "D:\\Ai-project\\DeepSeek-Harness-Desktop\\deepseek-harness\\packages\\sdk\\protocol\\lib\\index.js",
|
||||
"bytes": 7336,
|
||||
"sha256": "371c38ca3514797d86df5f4bb78a56c40a2637294b83aababe8145bec6a20253",
|
||||
"relative": "packages/sdk/protocol/lib/index.js"
|
||||
},
|
||||
{
|
||||
"path": "D:\\Ai-project\\DeepSeek-Harness-Desktop\\deepseek-harness\\packages\\sdk\\client\\package.json",
|
||||
"bytes": 1325,
|
||||
"sha256": "ff33f2ecec2d48bb8a24397cc664fc2dd50ba49d5288eb11fea1cb12931e5d5f",
|
||||
"relative": "packages/sdk/client/package.json"
|
||||
},
|
||||
{
|
||||
"path": "D:\\Ai-project\\DeepSeek-Harness-Desktop\\deepseek-harness\\pnpm-lock.yaml",
|
||||
"bytes": 1090222,
|
||||
"sha256": "4637cca3b89c4c8d5a147321710e597aa62ec2444a3772ab542be5690893dfe4",
|
||||
"relative": "pnpm-lock.yaml"
|
||||
},
|
||||
{
|
||||
"path": "D:\\Ai-project\\DeepSeek-Harness-Desktop\\deepseek-harness\\vendor\\cordis\\lib\\index.js",
|
||||
"bytes": 60361,
|
||||
"sha256": "6a9394c0877ff45218818c6e815edd038f8057e1a1deb390a8d43ec81c57691e",
|
||||
"relative": "vendor/cordis/lib/index.js"
|
||||
}
|
||||
],
|
||||
"node": {
|
||||
"path": "D:\\Ai-project\\REA逆向\\tools\\node-v24.22.1-win-x64\\node.exe",
|
||||
"bytes": 93915464,
|
||||
"sha256": "d702daa815eabbff763c7d3808e235cf031cf2ca70211b56847bc72f51e240e5",
|
||||
"version": "24.22.1"
|
||||
},
|
||||
"identitySha256": "65483d7f8aa644d890100a22bc52555ce91c009ba2d213b510f3da1a9ede7ff3",
|
||||
"serverInfo": {
|
||||
"name": "deepseek-harness-sdk-runtime",
|
||||
"version": "0.0.1"
|
||||
},
|
||||
"message": "已找到匹配版本,可验证原程序的连接与停止。"
|
||||
},
|
||||
"sdkSessionId": "93a36095-7405-471d-8a6e-696f6ca89285",
|
||||
"directory": "C:\\Users\\wangy\\AppData\\Local\\Temp\\AgentWorkbench-Engines\\a3084ece-f45c-40c8-b130-4d221b73b4f5",
|
||||
"status": "completed",
|
||||
"startedAt": "2026-10-31T14:53:29.560Z",
|
||||
"finishedAt": "2026-10-31T14:53:30.923Z",
|
||||
"requiresVerification": false,
|
||||
"verification": {
|
||||
"verified": true,
|
||||
"installationSha256": "65483d7f8aa644d890100a22bc52555ce91c009ba2d213b510f3da1a9ede7ff3",
|
||||
"readySignal": "appReady",
|
||||
"sdkSessionId": "93a36095-7405-471d-8a6e-696f6ca89285",
|
||||
"connectionId": "074fc526-5f82-49dc-a11d-316dc91b7c96",
|
||||
"rootExited": true,
|
||||
"modelRequests": 0
|
||||
},
|
||||
"summary": "连接通过,启动、协议返回、会话目录与进程结束均已核对。",
|
||||
"contractArtifactId": "09fb69d8-4737-4b70-a0ef-dedfbad1cc56",
|
||||
"contractSha256": "fc060f625a81bc3b43fc71e538401af63086cebcd3624cf2f95dc233feed66de",
|
||||
"eventId": "24be7482-e8be-4c71-a124-f57115ff2f3a",
|
||||
"pid": 38116,
|
||||
"connectionId": "074fc526-5f82-49dc-a11d-316dc91b7c96",
|
||||
"readiness": {
|
||||
"schemaVersion": 1,
|
||||
"nonce": "32055bd4-2c67-4c12-beec-d68d13cd3c07",
|
||||
"pid": 38116,
|
||||
"signal": "appReady",
|
||||
"readyAt": "2026-10-31T14:53:30.592Z",
|
||||
"artifact": {
|
||||
"path": "C:\\Users\\wangy\\AppData\\Local\\Temp\\AgentWorkbench-Engines\\a3084ece-f45c-40c8-b130-4d221b73b4f5\\ready.json",
|
||||
"bytes": 135,
|
||||
"sha256": "9e199be29f0d3e3fa7f36c9e0c112e25070291a80b9bf9e31de64c1b0809f6cd"
|
||||
}
|
||||
},
|
||||
"serverInfo": {
|
||||
"name": "deepseek-harness-sdk-runtime",
|
||||
"version": "0.0.1"
|
||||
},
|
||||
"reportArtifactId": "c3721b77-e08a-4922-8860-8c91ec74639e",
|
||||
"reportSha256": "11f3b35c0806c931e692eb35273826b5cecf1761ef316dd6f16b47f09a9c1dfc",
|
||||
"reportSummary": {
|
||||
"status": "verified",
|
||||
"modelRequests": 0,
|
||||
"providerRequests": 0,
|
||||
"observedResponses": 6,
|
||||
"rootExited": true
|
||||
}
|
||||
},
|
||||
"actualSdk": {
|
||||
"version": "0.2.1-rc.2",
|
||||
"identitySha256": "65483d7f8aa644d890100a22bc52555ce91c009ba2d213b510f3da1a9ede7ff3",
|
||||
"nativeMethods": 6,
|
||||
"modelRequests": 0,
|
||||
"rootExited": true
|
||||
},
|
||||
"stopScope": "declared private SDK pending-initialize fixture",
|
||||
"scope": "Independent service and data. Original 4318 Provider, tasks and credentials were not accessed."
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
{
|
||||
"success": true,
|
||||
"version": "1.17.0",
|
||||
"testedAt": "2026-10-31T14:48:41.628Z",
|
||||
"browser": "156.0.4314.50",
|
||||
"passed": [
|
||||
"model-selected checks wait for approval and display actual counted results",
|
||||
"verified artifact preview shows the real check log and hash",
|
||||
"changed source invalidates old checks and manual confirmation runs the new plan",
|
||||
"390px task drawer, log preview, close controls and fixed composer remain operable",
|
||||
"reconnection synchronizes missed session changes in other projects",
|
||||
"budget controls pause before tools and preserve the goal across recorded runs",
|
||||
"applied proposals continue the original task into actual engineering verification"
|
||||
],
|
||||
"pageErrors": [],
|
||||
"requestFailures": [],
|
||||
"projectRoot": "D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\execution-ui-d0bf0c10-0ed2-450b-be09-aadd74848307\\project",
|
||||
"model": "controlled local HTTP fixture"
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
✔ saved engine budgets verify all Chat/Responses route variants and reject a mismatched output allowance (592.287ms)
|
||||
✔ owned CC Switch SQLite imports the selected TOML Responses wire_api and leaves unknown clients untouched (710.0598ms)
|
||||
✔ Responses executes one original tool through Runtime, carries encrypted Items, reports raw usage and seals exact wire bytes (1527.7255ms)
|
||||
✔ mixed-protocol pool keeps both actual variants and changes route on 429 without re-executing completed tools (1797.027ms)
|
||||
▶ Responses incomplete, failed, missing call identity and unsupported hosted outputs never dispatch a tool
|
||||
✔ Reject {"body":{"id":"resp_owned","status":"incomplete","output":[{"type":"function_call","id":"fc_partial" (994.4985ms)
|
||||
✔ Reject {"body":{"id":"resp_owned","status":"failed","output":[{"type":"function_call","id":"fc_failed","cal (1145.3472ms)
|
||||
✔ Reject {"body":{"id":"resp_owned","status":"completed","output":[{"type":"function_call","id":"fc_bad","cal (1076.7906ms)
|
||||
✔ Reject {"body":{"id":"resp_owned","status":"completed","output":[{"type":"computer_call","id":"not-supporte (1121.9989ms)
|
||||
✔ Responses incomplete, failed, missing call identity and unsupported hosted outputs never dispatch a tool (4339.4911ms)
|
||||
▶ invalid JSON in a nonstream Chat or Responses tool group prevents every tool and pool replay
|
||||
✔ chat-completions (1174.9831ms)
|
||||
✔ responses (1170.1157ms)
|
||||
✔ invalid JSON in a nonstream Chat or Responses tool group prevents every tool and pool replay (2345.4991ms)
|
||||
✔ wire adapter preserves tool IDs but never passes Responses-private Items to Chat or another endpoint (0.2085ms)
|
||||
✔ connection JSON import accepts explicit Responses protocol and continues to reject unknown protocols without reading client sessions (678.5131ms)
|
||||
ℹ tests 14
|
||||
ℹ suites 0
|
||||
ℹ pass 14
|
||||
ℹ fail 0
|
||||
ℹ cancelled 0
|
||||
ℹ skipped 0
|
||||
ℹ todo 0
|
||||
ℹ duration_ms 12110.7319
|
||||
@@ -0,0 +1,46 @@
|
||||
✔ native text and call IDs preserve original result pairing while unsupported content fails closed (1.1259ms)
|
||||
✔ one engine child freezes real installation identity before creating tasks and rejects a changed source (13098.4525ms)
|
||||
✔ owned native tools return saved results, use real approvals/checks and account each actual request once across new engine Runs (40251.2521ms)
|
||||
✔ forged native scope cannot change a parent file and a final completion claim cannot replace worker proof (20308.5425ms)
|
||||
✔ duplicate native IDs are rejected without replaying a saved tool (16562.3545ms)
|
||||
✔ native input exceeding the context budget is retained and sends no model request (15352.2075ms)
|
||||
✔ stop retains a late reported model usage once and executes no tool from that output (18454.3389ms)
|
||||
✔ a source change after the original model intent stops before native tools execute (11914.2429ms)
|
||||
✔ an engine source change while approval is pending invalidates the old permission before any check starts (20706.4131ms)
|
||||
✔ missing or noncompleted turn end cannot be adopted even with an owned receipt and assistant text (0.6475ms)
|
||||
✔ closed root with an original live descendant remains unknown and retains its process tree (0.2834ms)
|
||||
✔ host recovery reserves original pending attempts exactly once and explicit review preserves unknown outcomes (19007.1945ms)
|
||||
✔ corrupted model reservation cannot reduce recovered budget or permit a new engine request (16877.8397ms)
|
||||
✔ owned CC Switch SQLite imports the selected TOML Responses wire_api and leaves unknown clients untouched (3234.3774ms)
|
||||
✔ Responses executes one original tool through Runtime, carries encrypted Items, reports raw usage and seals exact wire bytes (6488.5999ms)
|
||||
✔ mixed-protocol pool keeps both actual variants and changes route on 429 without re-executing completed tools (4370.0093ms)
|
||||
▶ Responses incomplete, failed, missing call identity and unsupported hosted outputs never dispatch a tool
|
||||
✔ Reject {"body":{"id":"resp_owned","status":"incomplete","output":[{"type":"function_call","id":"fc_partial" (3630.0169ms)
|
||||
✔ Reject {"body":{"id":"resp_owned","status":"failed","output":[{"type":"function_call","id":"fc_failed","cal (2948.9085ms)
|
||||
✔ Reject {"body":{"id":"resp_owned","status":"completed","output":[{"type":"function_call","id":"fc_bad","cal (3278.7186ms)
|
||||
✔ Reject {"body":{"id":"resp_owned","status":"completed","output":[{"type":"computer_call","id":"not-supporte (3321.5086ms)
|
||||
✔ Responses incomplete, failed, missing call identity and unsupported hosted outputs never dispatch a tool (13180.1108ms)
|
||||
▶ invalid JSON in a nonstream Chat or Responses tool group prevents every tool and pool replay
|
||||
✔ chat-completions (3471.3136ms)
|
||||
✔ responses (2630.7028ms)
|
||||
✔ invalid JSON in a nonstream Chat or Responses tool group prevents every tool and pool replay (6102.4523ms)
|
||||
✔ wire adapter preserves tool IDs but never passes Responses-private Items to Chat or another endpoint (0.187ms)
|
||||
✔ connection JSON import accepts explicit Responses protocol and continues to reject unknown protocols without reading client sessions (1186.2766ms)
|
||||
✔ a stop from the event loop interrupts immediate model replies before later tools (3385.4454ms)
|
||||
✔ a stop after the first tool preserves a complete interrupted multi-call protocol group (4073.0285ms)
|
||||
✔ the compatible HTTP loop preserves call IDs, validates errors and waits before executing commands (10411.9691ms)
|
||||
✔ denial prevents command execution and produces a paired denied result (5347.8428ms)
|
||||
✔ stopping during approval interrupts pending calls without executing or replaying them (3917.6167ms)
|
||||
✔ stopping a running fixture command terminates its own process tree and retains uncertain state (3065.8461ms)
|
||||
✔ command output decodes split UTF-8 and excludes the workbench key from child environment (1458.0358ms)
|
||||
✔ model-list testing uses a tentative key without changing the saved connection and redacts service errors (6.6736ms)
|
||||
✔ truncated or repeated model calls fail before execution; tool input validation rejects unknown capabilities (4921.9888ms)
|
||||
✔ request projection keeps exact tool pairs and complete recent user instructions while preserving disk history (2627.5789ms)
|
||||
ℹ tests 36
|
||||
ℹ suites 0
|
||||
ℹ pass 36
|
||||
ℹ fail 0
|
||||
ℹ cancelled 0
|
||||
ℹ skipped 0
|
||||
ℹ todo 0
|
||||
ℹ duration_ms 192697.2863
|
||||
@@ -0,0 +1,19 @@
|
||||
{
|
||||
"checkedAt": "2026-10-31T14:49:46.758Z",
|
||||
"version": "1.17.0",
|
||||
"passed": [
|
||||
"工程索引可独立于任务使用,实际覆盖范围与损坏文件显示明确",
|
||||
"真实AST查询打开原文件第181行并仅滚动代码容器,输入区和顶部保持可见",
|
||||
"查询词保留;过期SHA不高亮旧位置,增量刷新后正确定位第184行",
|
||||
"切换项目时延迟返回的旧索引不覆盖新项目结果",
|
||||
"Python候选、字符串误报排除和局部导入文件候选保留真实证据标签",
|
||||
"两处唯一锚点成为待审核提案,真实界面应用后BOM/CRLF和未选择原字节保留",
|
||||
"390px索引查询、范围和完整指纹可读且无横向溢出,实际文件行定位通过",
|
||||
"索引刷新失败保留归属和解释,旧结果禁用并可重试恢复"
|
||||
],
|
||||
"pageErrors": [],
|
||||
"requestFailures": [],
|
||||
"root": "D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\index-ui-cf2562c0-c35a-4fda-83fd-b65f844c79f1",
|
||||
"primaryProjectId": "f07c27ba-6e41-4eb8-a5d6-6364dbd93acd",
|
||||
"otherProjectId": "e62ad02d-c9c6-4d39-9837-92a99842cfc9"
|
||||
}
|
||||
@@ -0,0 +1,16 @@
|
||||
{
|
||||
"success": true,
|
||||
"assertions": [
|
||||
"actual host crash with no result report exposes its original verified input and unknown status",
|
||||
"390px original-input review clears the job and invocation gates while retaining unknown, no report and no automatic model replay"
|
||||
],
|
||||
"viewportWidths": [
|
||||
1440,
|
||||
390
|
||||
],
|
||||
"pageErrors": [],
|
||||
"requestFailures": [],
|
||||
"localFixtureModelRequests": 1,
|
||||
"realModelRequests": 0,
|
||||
"jobId": "0b90a4df-d7dd-42d3-bb07-7f6efe6c7e7e"
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
{
|
||||
"success": true,
|
||||
"assertions": [
|
||||
"MCP registration stays disabled and shows the hashed local executable and Node entry",
|
||||
"actual stdio tools/list schema is visible; only the reviewed echo tool is enabled",
|
||||
"an enabled program whose source changes can be disabled immediately without executing the changed entry",
|
||||
"approved external call interrupted after dispatch remains unknown and does not replay",
|
||||
"TaskPanel exposes Task/Run source, original report, and human acknowledgement without marking unknown as success",
|
||||
"390px MCP registry, long executable paths, controls, and operation history have no horizontal overflow"
|
||||
],
|
||||
"viewportWidths": [
|
||||
1440,
|
||||
390
|
||||
],
|
||||
"pageErrors": [],
|
||||
"requestFailures": [],
|
||||
"localFixtureModelRequests": 1,
|
||||
"realModelRequests": 0,
|
||||
"unknownJobId": "377d19d0-0c5d-4f9d-bb5c-21db2c39fc88",
|
||||
"screenshots": [
|
||||
"desktop-mcp-registry.png",
|
||||
"mobile-mcp-registry.png"
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,23 @@
|
||||
{
|
||||
"url": "http://127.0.0.1:58898",
|
||||
"viewportWidths": [
|
||||
1440,
|
||||
390
|
||||
],
|
||||
"assertions": [
|
||||
"connection form stores only the volatile credential and renders configured status without exposing it",
|
||||
"pool editor saves weighted members and binds selection for new Runs",
|
||||
"schema-bound import preview, opt-in automatic sync, revision update and sync shutdown keep credentials hidden",
|
||||
"real historical pagination detects the owned corrupted older artifact, retains its identity and leaves original bytes unchanged",
|
||||
"local inspection shows real database and artifact checks and emits zero model requests",
|
||||
"390px model-center modal and connection cards remain operable without horizontal page overflow"
|
||||
],
|
||||
"pageErrors": [],
|
||||
"requestFailures": [],
|
||||
"modelRequests": 0,
|
||||
"screenshots": [
|
||||
"desktop-operations.png",
|
||||
"mobile-operations.png",
|
||||
"mobile-connections.png"
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,41 @@
|
||||
{
|
||||
"success": true,
|
||||
"version": "1.17.0",
|
||||
"appRoot": "D:\\Ai-project\\REA逆向\\Agent-Workbench",
|
||||
"nodePath": "D:\\Ai-project\\REA逆向\\tools\\node-v24.22.1-win-x64\\node.exe",
|
||||
"containedRuntime": false,
|
||||
"nodeSha256": "d702daa815eabbff763c7d3808e235cf031cf2ca70211b56847bc72f51e240e5",
|
||||
"passed": [
|
||||
"actual form saves Responses and declared stream, refresh preserves both and narrow editor remains operable",
|
||||
"separate text, nonce tool and actual SSE probes bind only their requested capability and own attempts",
|
||||
"actual pooled task excludes text-only before dispatch, executes original tool once and downloads SHA-matched original SSE",
|
||||
"desktop and actual paired gateway show same bounded stream; mobile stops original HTTP with zero fragment tool dispatch or replay",
|
||||
"truncated output keeps explicit unknown ledger, partial evidence and no automatic failover or incomplete tool execution"
|
||||
],
|
||||
"screenshots": [
|
||||
"D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\model-protocols-ui-4d113382-58d9-41f4-882e-380cadb7338d\\desktop-editor.png",
|
||||
"D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\model-protocols-ui-4d113382-58d9-41f4-882e-380cadb7338d\\narrow-editor.png",
|
||||
"D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\model-protocols-ui-4d113382-58d9-41f4-882e-380cadb7338d\\desktop-capabilities.png",
|
||||
"D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\model-protocols-ui-4d113382-58d9-41f4-882e-380cadb7338d\\narrow-capabilities.png",
|
||||
"D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\model-protocols-ui-4d113382-58d9-41f4-882e-380cadb7338d\\desktop-ledger.png",
|
||||
"D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\model-protocols-ui-4d113382-58d9-41f4-882e-380cadb7338d\\desktop-streaming.png",
|
||||
"D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\model-protocols-ui-4d113382-58d9-41f4-882e-380cadb7338d\\narrow-streaming.png",
|
||||
"D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\model-protocols-ui-4d113382-58d9-41f4-882e-380cadb7338d\\mobile-streaming.png",
|
||||
"D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\model-protocols-ui-4d113382-58d9-41f4-882e-380cadb7338d\\mobile-stopped.png",
|
||||
"D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\model-protocols-ui-4d113382-58d9-41f4-882e-380cadb7338d\\narrow-stopped.png",
|
||||
"D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\model-protocols-ui-4d113382-58d9-41f4-882e-380cadb7338d\\desktop-incomplete.png",
|
||||
"D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\model-protocols-ui-4d113382-58d9-41f4-882e-380cadb7338d\\narrow-incomplete.png"
|
||||
],
|
||||
"pageErrors": [],
|
||||
"requestFailures": [],
|
||||
"fixtureModelRequests": 7,
|
||||
"realModelRequests": 0,
|
||||
"osTrustChanged": false,
|
||||
"nativeIPhone": "not exercised; paired Edge viewport uses CA-verifying owned relay",
|
||||
"taskId": "5bef2fcc-77cf-4756-b458-02609ebaad18",
|
||||
"runIds": [
|
||||
"55a3fb7d-395f-422a-b5f8-f262a37d8ea1",
|
||||
"dab7b7fb-b25b-4dd6-9f37-8371ae1fcf9f",
|
||||
"0b5ddad4-20da-49d5-8752-72de133fde4a"
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
{
|
||||
"success": true,
|
||||
"url": "http://127.0.0.1:51638",
|
||||
"viewportWidths": [
|
||||
1440,
|
||||
390
|
||||
],
|
||||
"assertions": [
|
||||
"source hashes remain visible and a changed file is explicitly expired with retained provenance",
|
||||
"incomplete inventory marks existing facts unverified and blocks new saves",
|
||||
"390px memory review stays within the viewport with the incomplete-scan warning visible",
|
||||
"desktop and 390px request records expose the exact memory snapshot used by one owned local HTTP request"
|
||||
],
|
||||
"pageErrors": [],
|
||||
"requestFailures": [],
|
||||
"localFixtureModelRequests": 1,
|
||||
"realModelRequests": 0,
|
||||
"screenshots": [
|
||||
"desktop-expired-memory.png",
|
||||
"mobile-incomplete-memory.png",
|
||||
"desktop-sent-memory.png",
|
||||
"mobile-sent-memory.png"
|
||||
]
|
||||
}
|
||||
File diff suppressed because one or more lines are too long
@@ -0,0 +1,35 @@
|
||||
{
|
||||
"success": true,
|
||||
"version": "1.17.0",
|
||||
"at": "2026-10-31T14:51:34.926Z",
|
||||
"browser": "156.0.4314.50",
|
||||
"passed": [
|
||||
"mixed projects retain npm defaults and explicitly seal the selected Python checks",
|
||||
"the confirmation shows the actual interpreter and cancellation executes no checks",
|
||||
"actual unittest failures and the separately passed compilation display their counted results",
|
||||
"the log and original structured result are independently viewable and hash-bound",
|
||||
"a reviewed three-file repair runs new checks while retaining the original failed job and protected files",
|
||||
"390px checks, interpreter paths, original result dialog and composer remain operable without page overflow",
|
||||
"Git confirmation and the Python execution report deliver only the three reviewed sources and preserve owner changes"
|
||||
],
|
||||
"pageErrors": [],
|
||||
"requestFailures": [],
|
||||
"checks": {
|
||||
"before": {
|
||||
"total": 6,
|
||||
"passed": 1,
|
||||
"failed": 5,
|
||||
"skipped": 0,
|
||||
"cancelled": 0
|
||||
},
|
||||
"after": {
|
||||
"total": 6,
|
||||
"passed": 6,
|
||||
"failed": 0,
|
||||
"skipped": 0,
|
||||
"cancelled": 0
|
||||
}
|
||||
},
|
||||
"commit": "a18a17bb02741c5105db0591cabf58fb102b3151",
|
||||
"scope": "Independent host and owned fixture. Real Python processes, reviewed API repairs and actual local Git. No configured model requests or production credentials."
|
||||
}
|
||||
@@ -0,0 +1,76 @@
|
||||
{
|
||||
"success": true,
|
||||
"version": "1.17.0",
|
||||
"at": "2026-10-31T14:52:36.680Z",
|
||||
"browser": "156.0.4314.50",
|
||||
"passed": [
|
||||
"the sealed React TypeScript project builds but its real domain tests first expose the functional failures",
|
||||
"a real browser separately observes that the broken pending filter wrongly shows all three tasks",
|
||||
"the four-file reviewed repair passes all nine domain tests, actual TypeScript checking and the Vite production build",
|
||||
"the production React app passes real filtering, search, single-task toggling and updated statistics",
|
||||
"the same built React app remains operable at 390px without horizontal overflow",
|
||||
"Workbench Git commits only the four reviewed files and binds the actual checks while preserving tests, configuration and owner changes"
|
||||
],
|
||||
"pageErrors": [],
|
||||
"requestFailures": [],
|
||||
"originalTests": {
|
||||
"total": 9,
|
||||
"passed": 2,
|
||||
"failed": 7,
|
||||
"cancelled": 0,
|
||||
"skipped": 0
|
||||
},
|
||||
"repairedTests": {
|
||||
"total": 9,
|
||||
"passed": 9,
|
||||
"failed": 0,
|
||||
"cancelled": 0,
|
||||
"skipped": 0
|
||||
},
|
||||
"beforePending": 3,
|
||||
"checks": [
|
||||
{
|
||||
"name": "test",
|
||||
"status": "passed",
|
||||
"exitCode": 0,
|
||||
"tests": {
|
||||
"total": 9,
|
||||
"passed": 9,
|
||||
"failed": 0,
|
||||
"cancelled": 0,
|
||||
"skipped": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "typecheck",
|
||||
"status": "passed",
|
||||
"exitCode": 0,
|
||||
"tests": null
|
||||
},
|
||||
{
|
||||
"name": "build",
|
||||
"status": "passed",
|
||||
"exitCode": 0,
|
||||
"tests": null
|
||||
}
|
||||
],
|
||||
"commit": "5783f9cec7f1ce54c43bc9e3ddefc9d6c12dd058",
|
||||
"changes": [
|
||||
"src/App.tsx",
|
||||
"src/query.ts",
|
||||
"src/summary.ts",
|
||||
"src/transition.ts"
|
||||
],
|
||||
"protectedFiles": {
|
||||
"package.json": "db3c6bfd79d1ebb1f5692727715e2b48d3f09e0314d9f1d455d335774f5e7db8",
|
||||
"package-lock.json": "48c138a05408667bbd332000731598415c9ae281d5072daa18544025ad89f7ff",
|
||||
"tsconfig.json": "9dcf064d622cb31ae94ef348655f4f7af84076241601148929232314ef1b0c8a",
|
||||
"vite.config.mjs": "0ea84b4e2909b04e4c1cb1b92392a9f85bcb46605472763b15935d3498ba8ed3",
|
||||
"index.html": "37e9fd3172422ecca3902378c567331363dc1277ac749dd9f6bdf74f04540787",
|
||||
"test/domain.test.mjs": "3fbd155cfc4919f3d0570a7ee26ff5ff4243e3e94250f3d0dbebe93c2b925ec0",
|
||||
"src/types.ts": "d2bbcac2cb175e7bcafe087915f02c3cf2e0ea4b65235d6231bbcd4c3a682246",
|
||||
"src/main.tsx": "c9a5787f5544788757ab654e1dfd83b5bc51495144e17aa078c2ee2f4412a302",
|
||||
"src/styles.css": "60bcb8f7f73f4328274096132a3d73b7ef1fc9e6fbf6a3a3477fb9d519f2c25c"
|
||||
},
|
||||
"scope": "Owned frozen React TypeScript project with offline-installed fixed dependencies, real npm/typecheck/build/browser and Workbench review/Git/report. API fixtures supply repairs; configured-model success is separate."
|
||||
}
|
||||
@@ -0,0 +1,138 @@
|
||||
{
|
||||
"success": true,
|
||||
"version": "1.17.0",
|
||||
"at": "2026-10-31T14:53:16.856Z",
|
||||
"browser": "156.0.4314.50",
|
||||
"passed": [
|
||||
"independent desktop research entry detects the installed fixed REA version",
|
||||
"UI registration seals the artifact identity in the current project and creates no model request",
|
||||
"actual REA ASAR analysis renders the verified source chain and downloads the exact original Bundle",
|
||||
"original field selection, paging and limitations retain exact bytes; Escape closes only the top dialog",
|
||||
"raw responses and connection records are readable; repeat analysis reuses the verified original without dispatch",
|
||||
"UI PE operation selection performs actual import and resource analysis with independently verified target hashes",
|
||||
"changed artifact versions refuse dispatch, retain the stale record and require a separately registered version",
|
||||
"390px research, long identities and original evidence dialogs fit the viewport and retain usable close controls",
|
||||
"a real host restart preserves exact target/job identities and serves sealed Evidence without producer replay",
|
||||
"a delayed evidence response cannot reopen a closed research panel or change another task",
|
||||
"declared owned stdio fault fixture stops its root process and explicit retry adds a new record without changing history"
|
||||
],
|
||||
"pageErrors": [],
|
||||
"requestFailures": [],
|
||||
"expectedRequestFailures": [],
|
||||
"root": "D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\research-ui-bdd74198-8ae0-4701-8715-1915a610da36",
|
||||
"actualRea": {
|
||||
"version": "6.8.0",
|
||||
"operations": [
|
||||
"javascript",
|
||||
"pe-imports",
|
||||
"pe-resources"
|
||||
],
|
||||
"jobs": [
|
||||
{
|
||||
"id": "d4c49997-06b6-4e98-a169-ee79811f8fbb",
|
||||
"status": "completed",
|
||||
"evidenceId": "ev_c1ad800281e4666c02bcc6f7d283254e7d3e044b8190c377f075c4b159ba8eba"
|
||||
},
|
||||
{
|
||||
"id": "083be29e-01af-428a-8141-0d0f54ab0e63",
|
||||
"status": "completed",
|
||||
"evidenceId": "ev_c1ad800281e4666c02bcc6f7d283254e7d3e044b8190c377f075c4b159ba8eba"
|
||||
},
|
||||
{
|
||||
"id": "80fe088b-da0b-403e-8862-a120af56a6bc",
|
||||
"status": "completed",
|
||||
"evidenceId": "ev_372977a118d2703e195c0611636df0a6fe9cf1d37bc07a2465cecfb69b9587d1"
|
||||
},
|
||||
{
|
||||
"id": "d5e745d2-aed1-44e7-8b28-dca59f973d3d",
|
||||
"status": "completed",
|
||||
"evidenceId": "ev_d4d6c9b5c355eb84e69c1b8ead12cce214dc83ed178bacf1dafcc5afbc2e23f7"
|
||||
},
|
||||
{
|
||||
"id": "9f222679-32bb-4339-8937-079a8e77fa97",
|
||||
"status": "stale"
|
||||
},
|
||||
{
|
||||
"id": "2d95a873-0c43-445c-b2ad-89d6397c8cbc",
|
||||
"status": "completed",
|
||||
"evidenceId": "ev_6180816e4b9cee0e16668fe0f31a0a8ab281fbad173e445f5b968de9a33d8f70"
|
||||
}
|
||||
],
|
||||
"targetExecution": false,
|
||||
"modelRequests": 0
|
||||
},
|
||||
"stopScope": "explicitly declared private stdio handshake fault fixture",
|
||||
"screenshots": [
|
||||
".qa/screenshots/model-protocols-research-desktop.png",
|
||||
".qa/screenshots/model-protocols-research-mobile.png",
|
||||
".qa/screenshots/model-protocols-research-evidence-mobile.png",
|
||||
".qa/screenshots/model-protocols-research-stop-desktop.png"
|
||||
],
|
||||
"analysisRequests": [
|
||||
{
|
||||
"url": "http://127.0.0.1:61139/api/sessions/7b4c6388-afbe-4292-8407-ba39b4e9f10d/research/jobs",
|
||||
"input": {
|
||||
"targetId": "7747b8a5-7f73-402f-847c-0b6fb8d09ed0",
|
||||
"operation": "javascript",
|
||||
"refresh": false
|
||||
}
|
||||
},
|
||||
{
|
||||
"url": "http://127.0.0.1:61139/api/sessions/7b4c6388-afbe-4292-8407-ba39b4e9f10d/research/jobs",
|
||||
"input": {
|
||||
"targetId": "7747b8a5-7f73-402f-847c-0b6fb8d09ed0",
|
||||
"operation": "javascript",
|
||||
"refresh": false
|
||||
}
|
||||
},
|
||||
{
|
||||
"url": "http://127.0.0.1:61139/api/sessions/7b4c6388-afbe-4292-8407-ba39b4e9f10d/research/jobs",
|
||||
"input": {
|
||||
"targetId": "83779f93-92a7-4052-9afe-b032515f9f7d",
|
||||
"operation": "pe-imports",
|
||||
"refresh": false
|
||||
}
|
||||
},
|
||||
{
|
||||
"url": "http://127.0.0.1:61139/api/sessions/7b4c6388-afbe-4292-8407-ba39b4e9f10d/research/jobs",
|
||||
"input": {
|
||||
"targetId": "83779f93-92a7-4052-9afe-b032515f9f7d",
|
||||
"operation": "pe-resources",
|
||||
"refresh": false
|
||||
}
|
||||
},
|
||||
{
|
||||
"url": "http://127.0.0.1:61139/api/sessions/7b4c6388-afbe-4292-8407-ba39b4e9f10d/research/jobs",
|
||||
"input": {
|
||||
"targetId": "7747b8a5-7f73-402f-847c-0b6fb8d09ed0",
|
||||
"operation": "javascript",
|
||||
"refresh": false
|
||||
}
|
||||
},
|
||||
{
|
||||
"url": "http://127.0.0.1:61139/api/sessions/7b4c6388-afbe-4292-8407-ba39b4e9f10d/research/jobs",
|
||||
"input": {
|
||||
"targetId": "b3e0df51-5c49-4f94-ac13-7ae9bb3811bb",
|
||||
"operation": "javascript",
|
||||
"refresh": false
|
||||
}
|
||||
},
|
||||
{
|
||||
"url": "http://127.0.0.1:56627/api/sessions/6bfae6ec-0181-4790-b76a-50e87f6a16e3/research/jobs",
|
||||
"input": {
|
||||
"targetId": "e4c3871d-2001-4e80-b451-fc6fef517d2a",
|
||||
"operation": "javascript",
|
||||
"refresh": false
|
||||
}
|
||||
},
|
||||
{
|
||||
"url": "http://127.0.0.1:56627/api/sessions/6bfae6ec-0181-4790-b76a-50e87f6a16e3/research/jobs",
|
||||
"input": {
|
||||
"targetId": "e4c3871d-2001-4e80-b451-fc6fef517d2a",
|
||||
"operation": "javascript",
|
||||
"refresh": true
|
||||
}
|
||||
}
|
||||
],
|
||||
"scope": "Independent ports/data and self-owned inert artifacts. No production service, configured-model request or real key read/copied."
|
||||
}
|
||||
@@ -0,0 +1,22 @@
|
||||
{
|
||||
"success": true,
|
||||
"viewportWidths": [
|
||||
1440,
|
||||
390
|
||||
],
|
||||
"assertions": [
|
||||
"desktop registers a disabled skill, reviews original UTF-8 instructions and explicitly enables it with no model calls",
|
||||
"source edit blocks activation until a new reviewed revision is registered; original revision remains available",
|
||||
"390px review and activation retain readable paths, revision history and visible source state without horizontal overflow",
|
||||
"actual local HTTP task reads precisely revision 2 and exposes its SHA-bound sent catalog in task evidence"
|
||||
],
|
||||
"pageErrors": [],
|
||||
"requestFailures": [],
|
||||
"localFixtureModelRequests": 2,
|
||||
"realModelRequests": 0,
|
||||
"screenshots": [
|
||||
"desktop-skill-source.png",
|
||||
"mobile-skill-source.png",
|
||||
"desktop-sent-skill-catalog.png"
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
{
|
||||
"success": true,
|
||||
"passed": [
|
||||
"desktop saves and inspects the full original Task and tool provenance without another model request",
|
||||
"an outside source change is visible and removes the checkpoint continuation action",
|
||||
"390px explicit continuation keeps the same Task and one original read; a second resume of the old checkpoint is blocked"
|
||||
],
|
||||
"viewportWidths": [
|
||||
1440,
|
||||
390
|
||||
],
|
||||
"pageErrors": [],
|
||||
"requestFailures": [],
|
||||
"screenshots": [
|
||||
"desktop-original-checkpoint.png",
|
||||
"mobile-checkpoint-ready.png"
|
||||
],
|
||||
"localFixtureModelRequests": 3,
|
||||
"realModelRequests": 0
|
||||
}
|
||||
@@ -0,0 +1,31 @@
|
||||
{
|
||||
"success": true,
|
||||
"version": "1.17.0",
|
||||
"testedAt": "2026-10-31T14:46:51.918Z",
|
||||
"directory": "D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\task-plan-ui-24c9339b-ab4c-4753-8c41-d4399457fbe4",
|
||||
"passed": [
|
||||
"structured editor seals two real steps and their dependency without running tools",
|
||||
"original owned read verifies the prerequisite and opens its hashed proof without replay",
|
||||
"real two-test and build job supplies the current-source proof and completes the sealed plan",
|
||||
"outside source edits invalidate the current proof, retain the rejection and block a passed delivery report",
|
||||
"390px immutable revision adds a mobile step, preserves old criteria and keeps evidence dialogs operable",
|
||||
"a delayed plan-read failure stays with the original task after selection changes",
|
||||
"controlled tab unmount during a delayed successful save retains the original task result without restoring the closed editor",
|
||||
"real host restart retains all plan revisions, accepted/rejected proofs and the original operations without replay"
|
||||
],
|
||||
"pageErrors": [],
|
||||
"requestFailures": [],
|
||||
"screenshots": [
|
||||
"D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\task-plan-ui-24c9339b-ab4c-4753-8c41-d4399457fbe4\\task-plan-desktop-editor.png",
|
||||
"D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\task-plan-ui-24c9339b-ab4c-4753-8c41-d4399457fbe4\\task-plan-desktop-verified.png",
|
||||
"D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\task-plan-ui-24c9339b-ab4c-4753-8c41-d4399457fbe4\\task-plan-mobile-editor.png",
|
||||
"D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\task-plan-ui-24c9339b-ab4c-4753-8c41-d4399457fbe4\\task-plan-mobile-artifact.png",
|
||||
"D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\task-plan-ui-24c9339b-ab4c-4753-8c41-d4399457fbe4\\task-plan-mobile-steps.png",
|
||||
"D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\task-plan-ui-24c9339b-ab4c-4753-8c41-d4399457fbe4\\task-plan-mobile-overview.png"
|
||||
],
|
||||
"taskId": "5d46257a-0bb7-407a-8740-a5cb277f4fb6",
|
||||
"sessionId": "95869a68-3b50-4e02-a2ac-d48ba9e393c4",
|
||||
"planRevisions": 3,
|
||||
"proofs": 4,
|
||||
"scope": "Independent real Edge desktop/390px Workbench UI, actual npm checks and owned read fixture. No real model credentials or production service access; semantic project acceptance is separate."
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,28 @@
|
||||
{
|
||||
"success": true,
|
||||
"version": "1.17.0",
|
||||
"at": "2026-10-31T14:50:58.841Z",
|
||||
"browser": "156.0.4314.50",
|
||||
"passed": [
|
||||
"desktop preview seals the committed version and shows existing staged/unstaged/untracked source changes",
|
||||
"repeated create clicks execute once and preserve the exact source index and owner files",
|
||||
"opening the child, returning to the source and reloading retain stable project/session identity",
|
||||
"a record owned by another session navigates to that saved owner instead of accepting its actions",
|
||||
"explicit recovery preserves outside content and acknowledgement does not report successful creation",
|
||||
"390×844 preview/create/open/return stays within viewport and preserves source ownership",
|
||||
"late-failure stays with the original session after project switching",
|
||||
"late-success stays with the original session after project switching",
|
||||
"a delayed bootstrap cannot navigate after the task panel has unmounted"
|
||||
],
|
||||
"pageErrors": [],
|
||||
"requestFailures": [],
|
||||
"screenshots": [
|
||||
".qa/screenshots/model-protocols-workspaces-review-desktop.png",
|
||||
".qa/screenshots/model-protocols-workspaces-ready-desktop.png",
|
||||
".qa/screenshots/model-protocols-workspaces-recovery-desktop.png",
|
||||
".qa/screenshots/model-protocols-workspaces-review-mobile.png",
|
||||
".qa/screenshots/model-protocols-workspaces-child-mobile.png"
|
||||
],
|
||||
"projectRoot": "D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\workspace-ui-06d1865c-a007-4814-bf4a-d8461d8e7220",
|
||||
"scope": "Independent synthetic Git projects and real browser flows. Desktop/390×844, actual worktree creation, source preservation, explicit external recovery, ownership navigation, delayed success/error/navigation. No production service or real model credential used."
|
||||
}
|
||||
Generated
+2
-2
@@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "local-agent-workbench",
|
||||
"version": "1.16.0",
|
||||
"version": "1.17.0",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "local-agent-workbench",
|
||||
"version": "1.16.0",
|
||||
"version": "1.17.0",
|
||||
"dependencies": {
|
||||
"@babel/parser": "7.29.11",
|
||||
"@peculiar/x509": "3.1.0",
|
||||
|
||||
+3
-2
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "local-agent-workbench",
|
||||
"version": "1.16.0",
|
||||
"version": "1.17.0",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"description": "A local coding workbench with explicit tool execution, reviewable changes and recoverable sessions.",
|
||||
@@ -38,7 +38,8 @@
|
||||
"verify:desktop-projects": "node scripts/verify-controlled-projects.mjs --desktop",
|
||||
"verify:desktop-lifecycle": "node scripts/verify-controlled-ui.mjs --desktop",
|
||||
"verify:desktop-install": "node scripts/verify-desktop-install.mjs",
|
||||
"verify:companion": "node scripts/verify-companion-ui.mjs"
|
||||
"verify:companion": "node scripts/verify-companion-ui.mjs",
|
||||
"verify:model-protocols": "node scripts/verify-model-protocols-ui.mjs"
|
||||
},
|
||||
"dependencies": {
|
||||
"@babel/parser": "7.29.11",
|
||||
|
||||
@@ -826,3 +826,6 @@ A已提交578cf88。B/C/D新增协议、请求变体、SSE及能力选路,恢
|
||||
|
||||
1.17 E真实操作5组通过,原result与13张截图保持在.qa/model-protocols-ui-5c599342-cbbd-4019-8865-ee1d7b4570e4;主窗口池名、手机接收/停止、390px未知账本已目视核对。补修select明确可访问名称、完成外部API新增后刷新UI夹具、mobile-stream实际选择器、上游停顿的延迟进展、主窗口legacy误判/快速关闭同步、原工件下载字节;37/37、20/20及26/26相关专项分别通过。一次精确替换误命中凭据callback,立即rg核对定位并修正,未执行或部署该中间版本。
|
||||
第一次历史全量UI运行误将Purge打包目录当源码目录,原结果/日志逐字节复制到protocol-first-ui-environment-failure后,以source/upstream重新整轮,session82022;早于最终修复启动的full-tests session80218仅作开发中诊断。响应协议真实工程新门槛使用同一受控Node/React/Python算法、原测试、审批与Git,但走实际Responses SSE+ModelHub。当前session64602执行,结果尚未判定。只读路径version/gateway误猜后已用rg定位diagnostics.mjs与server/companion-gateway.mjs,无数据写入。
|
||||
|
||||
1.17 F诊断:开发整套回归的旧Runtime夹具发送{invalid,现有完整JSON组校验先拒绝,导致旧等待审批断言超时;保留原TAP,旧语义错误用例改为合法JSON/无效query类型,另加Chat/Responses整组无工具/无池重放回归。新Responses真实Harness工程第一次失败原件保留cf832704目录:检查8/8已通过,但worker原输入核验仍只读body.max_tokens;新路由封存的是协议变体。修复为逐变体核对原native.maxTokens与实际输出预留及原预算,不略过SHA、归属或任一线路。此前几个只读大输出截断已改短投影;误猜core/workers路径已按rg定位core/tasks。
|
||||
F冻结前:Responses三工程13节点和父8/8/9/9/10/10、25类191组历史UI通过;相关36/36及新增预算/协议14/14。开发中完整553项552过/1失败原TAP保留,不作为最终结论。私有wrapper替换顺序首次拒绝已修正且保留原脚本;首次build因误用仓库node_modules/npm不存在而失败,固定Node附带npm的retry构建通过。准备提交全部执行输入、冻结并开始最终不可变独立验收。
|
||||
|
||||
@@ -9,6 +9,7 @@ const prefixes = Object.freeze({
|
||||
'1.14.0': 'controlled-engine',
|
||||
'1.15.0': 'desktop-host',
|
||||
'1.16.0': 'mobile-companion',
|
||||
'1.17.0': 'model-protocols',
|
||||
});
|
||||
export function releaseEvidencePrefix(version) {
|
||||
const prefix = prefixes[version];
|
||||
|
||||
@@ -64,7 +64,7 @@ export async function upgradeRunning({ url, pid, currentRoot, newRoot, handlerRo
|
||||
if (!fs.existsSync(originalScript) || !fs.existsSync(upgradeScript) || !fs.existsSync(path.join(next, 'dist-client', 'index.html'))) throw new Error('Upgrade bundles are incomplete.');
|
||||
if (!Number.isInteger(pid) || pid < 1 || pid === process.pid) throw new Error('Invalid target process.');
|
||||
const before = await readUpgradeJson(origin + '/api/bootstrap', 5000, 'Read original Workbench');
|
||||
if (!before.token || !['0.1.0', '1.0.0', '1.1.0', '1.2.0', '1.3.0', '1.4.0', '1.5.0', '1.5.1', '1.6.0', '1.6.1', '1.6.2', '1.7.0', '1.8.0', '1.9.0', '1.10.0', '1.10.1', '1.11.0', '1.12.0', '1.12.1', '1.13.0', '1.14.0', '1.15.0', '1.16.0'].includes(before.version)) throw new Error('This migration accepts a supported existing Workbench.');
|
||||
if (!before.token || !['0.1.0', '1.0.0', '1.1.0', '1.2.0', '1.3.0', '1.4.0', '1.5.0', '1.5.1', '1.6.0', '1.6.1', '1.6.2', '1.7.0', '1.8.0', '1.9.0', '1.10.0', '1.10.1', '1.11.0', '1.12.0', '1.12.1', '1.13.0', '1.14.0', '1.15.0', '1.16.0', '1.17.0'].includes(before.version)) throw new Error('This migration accepts a supported existing Workbench.');
|
||||
if (before.sessions.some(session => ['running', 'awaiting_approval', 'stopping'].includes(session.status))) throw new Error('Wait for existing tasks and approvals before upgrading.');
|
||||
let protocol, socket, enabledHere = false, paused = false;
|
||||
const breakpoints = [];
|
||||
|
||||
+12
-11
@@ -9,27 +9,28 @@ import { randomUUID } from 'node:crypto';
|
||||
if (process.argv.slice(2).some(value => value !== '--help')) throw new Error('Unsupported argument; use --help.');
|
||||
if (process.argv.includes('--help')) { console.log('node scripts/verify-all-ui.mjs - run serial desktop/390px checks in independent local hosts; use explicit QA Python/Harness/Purge source paths for 1.15.'); process.exit(0); }
|
||||
const version = JSON.parse(fs.readFileSync('package.json', 'utf8')).version;
|
||||
assert.ok(['1.6.0', '1.6.1', '1.6.2', '1.7.0', '1.8.0', '1.9.0', '1.10.0', '1.10.1', '1.11.0', '1.12.0', '1.12.1', '1.13.0', '1.14.0', '1.15.0', '1.16.0'].includes(version), 'Review evidence paths when introducing the next release.');
|
||||
assert.ok(['1.6.0', '1.6.1', '1.6.2', '1.7.0', '1.8.0', '1.9.0', '1.10.0', '1.10.1', '1.11.0', '1.12.0', '1.12.1', '1.13.0', '1.14.0', '1.15.0', '1.16.0', '1.17.0'].includes(version), 'Review evidence paths when introducing the next release.');
|
||||
const prefix = releaseEvidencePrefix(version), resultFile = `.qa/${prefix}-all-ui-validation.json`, results = [];
|
||||
const qaEnvironment = Object.fromEntries(['AGENT_WORKBENCH_TEST_PYTHON', 'AGENT_WORKBENCH_DEEPSEEK_ROOT', 'AGENT_WORKBENCH_PURGE_178_ROOT', 'AGENT_WORKBENCH_PURGE_181_ROOT']
|
||||
.filter(name => process.env[name]).map(name => [name, process.env[name]]));
|
||||
for (const [name, value] of Object.entries(qaEnvironment)) assert.ok(path.isAbsolute(value) && fs.existsSync(value), 'Use an existing absolute QA runtime/source path: ' + name);
|
||||
const cases = [
|
||||
...(version === '1.16.0' ? [['verify-companion-ui.mjs', null]] : []),
|
||||
...(['1.14.0', '1.15.0', '1.16.0'].includes(version) ? [['verify-controlled-ui.mjs', null], ['verify-purge-ui.mjs', null]] : []),
|
||||
...(['1.13.0', '1.14.0', '1.15.0', '1.16.0'].includes(version) ? [['verify-workers-ui.mjs', null]] : []),
|
||||
...(['1.11.0', '1.12.0', '1.12.1', '1.13.0', '1.14.0', '1.15.0', '1.16.0'].includes(version) ? [['verify-task-plans-ui.mjs', `docs/${prefix}-task-ui-validation.json`]] : []),
|
||||
...(['1.10.0', '1.10.1', '1.11.0', '1.12.0', '1.12.1', '1.13.0', '1.14.0', '1.15.0', '1.16.0'].includes(version) ? [['verify-browser-ui.mjs', `docs/${prefix}-browser-ui-validation.json`]] : []),
|
||||
...(version === '1.17.0' ? [['verify-model-protocols-ui.mjs', null]] : []),
|
||||
...(['1.16.0', '1.17.0'].includes(version) ? [['verify-companion-ui.mjs', null]] : []),
|
||||
...(['1.14.0', '1.15.0', '1.16.0', '1.17.0'].includes(version) ? [['verify-controlled-ui.mjs', null], ['verify-purge-ui.mjs', null]] : []),
|
||||
...(['1.13.0', '1.14.0', '1.15.0', '1.16.0', '1.17.0'].includes(version) ? [['verify-workers-ui.mjs', null]] : []),
|
||||
...(['1.11.0', '1.12.0', '1.12.1', '1.13.0', '1.14.0', '1.15.0', '1.16.0', '1.17.0'].includes(version) ? [['verify-task-plans-ui.mjs', `docs/${prefix}-task-ui-validation.json`]] : []),
|
||||
...(['1.10.0', '1.10.1', '1.11.0', '1.12.0', '1.12.1', '1.13.0', '1.14.0', '1.15.0', '1.16.0', '1.17.0'].includes(version) ? [['verify-browser-ui.mjs', `docs/${prefix}-browser-ui-validation.json`]] : []),
|
||||
['verify-ui.mjs', '.qa/ui-validation.json'], ['verify-diagnostics-ui.mjs', '.qa/diagnostics-ui-validation.json'],
|
||||
['verify-execution-ui.mjs', '.qa/execution-ui-validation.json'], ['verify-delivery-ui.mjs', '.qa/delivery-ui-validation.json'],
|
||||
['verify-index-ui.mjs', '.qa/index-ui-result.json'], ['verify-change-sets-ui.mjs', `docs/${prefix}-batch-ui-validation.json`],
|
||||
['verify-context-ui.mjs', `.qa/${version === '1.6.0' ? 'workspaces-context' : prefix}-ui-validation.json`],
|
||||
['verify-workspaces-ui.mjs', `docs/${prefix === 'workspaces' ? prefix : prefix + '-workspaces'}-ui-validation.json`],
|
||||
...(['1.7.0', '1.8.0', '1.9.0', '1.10.0', '1.10.1', '1.11.0', '1.12.0', '1.12.1', '1.13.0', '1.14.0', '1.15.0', '1.16.0'].includes(version) ? [['verify-python-ui.mjs', `docs/${prefix}-python-ui-validation.json`]] : []),
|
||||
...(['1.7.0', '1.8.0', '1.9.0', '1.10.0', '1.10.1', '1.11.0', '1.12.0', '1.12.1', '1.13.0', '1.14.0', '1.15.0', '1.16.0'].includes(version) ? [['verify-react-delivery.mjs', version === '1.7.0' ? 'docs/react-engine-delivery-validation.json' : `docs/${prefix}-react-delivery-validation.json`]] : []),
|
||||
...(['1.8.0', '1.9.0', '1.10.0', '1.10.1', '1.11.0', '1.12.0', '1.12.1', '1.13.0', '1.14.0', '1.15.0', '1.16.0'].includes(version) ? [['verify-research-ui.mjs', `docs/${prefix}-research-ui-validation.json`]] : []),
|
||||
...(['1.9.0', '1.10.0', '1.10.1', '1.11.0', '1.12.0', '1.12.1', '1.13.0', '1.14.0', '1.15.0', '1.16.0'].includes(version) ? [['verify-engines-ui.mjs', version === '1.9.0' ? 'docs/engines-ui-validation.json' : `docs/${prefix}-engines-ui-validation.json`]] : []),
|
||||
...(['1.12.0', '1.12.1', '1.13.0', '1.14.0', '1.15.0', '1.16.0'].includes(version) ? ['model-center', 'project-memory', 'task-checkpoints', 'skills', 'mcp', 'mcp-recovery'].map(name => [`verify-${name}-ui.mjs`, null]) : []),
|
||||
...(['1.7.0', '1.8.0', '1.9.0', '1.10.0', '1.10.1', '1.11.0', '1.12.0', '1.12.1', '1.13.0', '1.14.0', '1.15.0', '1.16.0', '1.17.0'].includes(version) ? [['verify-python-ui.mjs', `docs/${prefix}-python-ui-validation.json`]] : []),
|
||||
...(['1.7.0', '1.8.0', '1.9.0', '1.10.0', '1.10.1', '1.11.0', '1.12.0', '1.12.1', '1.13.0', '1.14.0', '1.15.0', '1.16.0', '1.17.0'].includes(version) ? [['verify-react-delivery.mjs', version === '1.7.0' ? 'docs/react-engine-delivery-validation.json' : `docs/${prefix}-react-delivery-validation.json`]] : []),
|
||||
...(['1.8.0', '1.9.0', '1.10.0', '1.10.1', '1.11.0', '1.12.0', '1.12.1', '1.13.0', '1.14.0', '1.15.0', '1.16.0', '1.17.0'].includes(version) ? [['verify-research-ui.mjs', `docs/${prefix}-research-ui-validation.json`]] : []),
|
||||
...(['1.9.0', '1.10.0', '1.10.1', '1.11.0', '1.12.0', '1.12.1', '1.13.0', '1.14.0', '1.15.0', '1.16.0', '1.17.0'].includes(version) ? [['verify-engines-ui.mjs', version === '1.9.0' ? 'docs/engines-ui-validation.json' : `docs/${prefix}-engines-ui-validation.json`]] : []),
|
||||
...(['1.12.0', '1.12.1', '1.13.0', '1.14.0', '1.15.0', '1.16.0', '1.17.0'].includes(version) ? ['model-center', 'project-memory', 'task-checkpoints', 'skills', 'mcp', 'mcp-recovery'].map(name => [`verify-${name}-ui.mjs`, null]) : []),
|
||||
];
|
||||
fs.mkdirSync('.qa', { recursive: true });
|
||||
const externalHost = new Set(['verify-ui.mjs', 'verify-diagnostics-ui.mjs', 'verify-execution-ui.mjs', 'verify-index-ui.mjs']);
|
||||
|
||||
@@ -34,7 +34,7 @@ child.stdout.on('data', value => { output += value; for (const line of output.sp
|
||||
child.stderr.on('data', value => { errors += value; });
|
||||
const wait = async predicate => { const deadline = Date.now() + 15000; while (!await predicate()) { if (Date.now() > deadline) throw new Error('UI fixture timed out: ' + errors); await new Promise(resolve => setTimeout(resolve, 40)); } };
|
||||
await wait(() => url); const bootstrap = await (await fetch(url + '/api/bootstrap')).json();
|
||||
assert.ok(['1.4.0', '1.5.0', '1.5.1', '1.6.0', '1.6.1', '1.6.2', '1.7.0', '1.8.0', '1.9.0', '1.10.0', '1.10.1', '1.11.0', '1.12.0', '1.12.1', '1.13.0', '1.14.0', '1.15.0', '1.16.0'].includes(bootstrap.version)); assert.equal(bootstrap.config.mode, 'demo'); assert.equal(bootstrap.config.hasKey, false);
|
||||
assert.ok(['1.4.0', '1.5.0', '1.5.1', '1.6.0', '1.6.1', '1.6.2', '1.7.0', '1.8.0', '1.9.0', '1.10.0', '1.10.1', '1.11.0', '1.12.0', '1.12.1', '1.13.0', '1.14.0', '1.15.0', '1.16.0', '1.17.0'].includes(bootstrap.version)); assert.equal(bootstrap.config.mode, 'demo'); assert.equal(bootstrap.config.hasKey, false);
|
||||
const proofPrefix = releaseEvidencePrefix(bootstrap.version) + '-batch';
|
||||
const screenshotPrefix = bootstrap.version === '1.4.0' ? '1.4-batch' : proofPrefix;
|
||||
const api = async (route, input) => { const response = await fetch(url + '/api' + route, { method: input === undefined ? 'GET' : 'POST', headers: { 'Content-Type': 'application/json', 'X-Workbench-Token': bootstrap.token, Connection: 'close' },
|
||||
|
||||
@@ -77,7 +77,7 @@ child.stderr.on('data', bytes => { errors += bytes.toString('utf8'); });
|
||||
const until = async predicate => { const deadline = Date.now() + 20000; while (!await predicate()) { if (Date.now() > deadline) throw new Error('Context UI fixture timeout: ' + errors); await new Promise(resolve => setTimeout(resolve, 40)); } };
|
||||
await until(() => url || child.exitCode !== null); assert.ok(url, errors);
|
||||
const bootstrap = await (await fetch(url + '/api/bootstrap', { headers: { Connection: 'close' } })).json();
|
||||
assert.ok(['1.5.0', '1.5.1', '1.6.0', '1.6.1', '1.6.2', '1.7.0', '1.8.0', '1.9.0', '1.10.0', '1.10.1', '1.11.0', '1.12.0', '1.12.1', '1.13.0', '1.14.0', '1.15.0', '1.16.0'].includes(bootstrap.version)); assert.equal(bootstrap.config.mode, 'demo'); assert.equal(bootstrap.config.hasKey, false);
|
||||
assert.ok(['1.5.0', '1.5.1', '1.6.0', '1.6.1', '1.6.2', '1.7.0', '1.8.0', '1.9.0', '1.10.0', '1.10.1', '1.11.0', '1.12.0', '1.12.1', '1.13.0', '1.14.0', '1.15.0', '1.16.0', '1.17.0'].includes(bootstrap.version)); assert.equal(bootstrap.config.mode, 'demo'); assert.equal(bootstrap.config.hasKey, false);
|
||||
const api = async route => { const response = await fetch(url + '/api' + route, { headers: { 'X-Workbench-Token': bootstrap.token, Connection: 'close' } }); const value = await response.json(); assert.ok(response.ok, value.error); return value; };
|
||||
const proofPrefix = bootstrap.version === '1.6.0' ? 'workspaces-context' : releaseEvidencePrefix(bootstrap.version);
|
||||
const browser = await chromium.launch({ channel: 'msedge', headless: true }), page = await browser.newPage({ viewport: { width: 1440, height: 1000 } });
|
||||
|
||||
@@ -13,13 +13,15 @@ import { resolvePurgeSource, assertPurgeModelInput, nativeSystemTexts, PURGE_DEF
|
||||
import { deliveryGit as git, writeDeliveryFile as write } from '../tests/fixtures/delivery-project.mjs';
|
||||
import { ownerBytes } from '../tests/fixtures/workers-project.mjs';
|
||||
import { desktopProjectQa } from './desktop-project-qa.mjs';
|
||||
import { responsesEngineeringHttp } from '../tests/fixtures/responses-engineering-http.mjs';
|
||||
|
||||
const args = process.argv.slice(2);
|
||||
assert.ok(args.every(value => ['--help', '--purge-comparison', '--desktop'].includes(value)), 'Unknown option; use --help.');
|
||||
assert.ok(args.every(value => ['--help', '--purge-comparison', '--desktop', '--responses'].includes(value)), 'Unknown option; use --help.');
|
||||
if (args.includes('--help')) {
|
||||
console.log('node scripts/verify-controlled-projects.mjs [--purge-comparison | --desktop] — actual pinned Harness and owned HTTP model fixture; default verifies Node/React/Python with approvals, Edge and Git. Comparison verifies eight independent Node projects: two Purge versions, two repetitions, enabled/disabled. Set AGENT_WORKBENCH_DEEPSEEK_ROOT; default also requires AGENT_WORKBENCH_TEST_PYTHON, comparison requires AGENT_WORKBENCH_PURGE_178_ROOT / AGENT_WORKBENCH_PURGE_181_ROOT. No configured-model requests.'); process.exit(0);
|
||||
console.log('node scripts/verify-controlled-projects.mjs [--purge-comparison | --desktop] [--responses] — actual pinned Harness and owned HTTP model fixture; default verifies Node/React/Python with approvals, Edge and Git. --responses uses actual Responses SSE and ModelHub routing with the same engineering algorithm. Comparison verifies eight independent Node projects. Set AGENT_WORKBENCH_DEEPSEEK_ROOT and AGENT_WORKBENCH_TEST_PYTHON; comparison requires both Purge source roots. No configured-model requests.'); process.exit(0);
|
||||
}
|
||||
const comparisonMode = args.includes('--purge-comparison'), desktopMode = args.includes('--desktop');
|
||||
const responsesMode = args.includes('--responses'); assert.ok(!responsesMode || !comparisonMode, 'Protocol acceptance and Purge comparison remain separate.');
|
||||
assert.ok(!comparisonMode || !desktopMode, 'Desktop engineering and prompt comparison are separate acceptance runs.');
|
||||
const version = JSON.parse(fs.readFileSync('package.json', 'utf8')).version;
|
||||
const directory = path.resolve('.qa', (desktopMode ? 'desktop-projects-' : comparisonMode ? 'controlled-purge-' : 'controlled-projects-') + randomUUID()), results = [], cases = [];
|
||||
@@ -93,7 +95,7 @@ async function runSample(kind, { sampleName = kind, strategy = null, comparison
|
||||
};
|
||||
if (kind === 'react') install(f.projectRoot, 'parent');
|
||||
let phase = 'new';
|
||||
const model = await compatibleFixture(t, async (route, input) => {
|
||||
const model = await (responsesMode ? responsesEngineeringHttp : compatibleFixture)(t, async (route, input) => {
|
||||
assert.equal(route, '/v1/chat/completions'); assert.equal(input.model, 'owned-controlled-engineering');
|
||||
assertToolPairs(input.messages);
|
||||
const worker = [...f.store.core.worker.values()][0], session = f.store.session(worker.sessionId);
|
||||
@@ -116,7 +118,13 @@ async function runSample(kind, { sampleName = kind, strategy = null, comparison
|
||||
finish_reason: calls.length ? 'tool_calls' : 'stop' }], usage: { prompt_tokens: 123, completion_tokens: 45, total_tokens: 168 } };
|
||||
});
|
||||
f.store.saveConfig({ mode: 'compatible', model: 'owned-controlled-engineering', baseUrl: model.baseUrl, contextLimit: 64000 });
|
||||
const provider = new Provider(); provider.setKey('self-owned-controlled-model-fixture'); runtime = new Runtime(f.store, provider); f.onCleanup(() => runtime.stopAll());
|
||||
const provider = new Provider(); provider.setKey('self-owned-controlled-model-fixture');
|
||||
if (responsesMode) {
|
||||
const hub = provider.modelHub(f.store), connection = hub.upsert({ name: 'Owned Responses engineering', protocol: 'responses', model: 'owned-controlled-engineering', baseUrl: model.baseUrl,
|
||||
requiresKey: false, contextLimit: 64000, streaming: true, capabilityDeclarations: { streaming: 'supported' } });
|
||||
hub.select({ type: 'connection', id: connection.id });
|
||||
}
|
||||
runtime = new Runtime(f.store, provider); f.onCleanup(() => runtime.stopAll());
|
||||
runtime.controlledEngines = new EngineRunner(f.store, provider, { directoryRoot: path.join(f.root, 'owned-engine-runs') });
|
||||
runtime.checkRunner().pythonExecutable = python; const manager = runtime.workerManager();
|
||||
if (desktopMode) desktopQa = await desktopProjectQa({ store: f.store, provider, runtime, sample, kind });
|
||||
@@ -169,6 +177,12 @@ async function runSample(kind, { sampleName = kind, strategy = null, comparison
|
||||
}
|
||||
assert.equal(f.store.core.task.get(child.taskId).usage.requests, modelRequests.length);
|
||||
assert.equal(f.store.core.task.get(child.taskId).usage.totalTokens, modelRequests.length * 168);
|
||||
if (responsesMode) {
|
||||
const attempts = provider.modelHub(f.store).attempts;
|
||||
assert.equal(attempts.length, model.requests.length); assert.ok(attempts.every(value => value.protocol === 'responses' && value.stream?.terminal && value.responseArtifactId));
|
||||
fs.writeFileSync(path.join(sample, 'original-responses-requests.json'), JSON.stringify(model.requests, null, 2));
|
||||
fs.writeFileSync(path.join(sample, 'original-responses-attempts.json'), JSON.stringify(attempts, null, 2));
|
||||
}
|
||||
mark(label + ': actual Harness uses owned model/tool bridges, retains failing tests, and completes reviewed repairs with one accounting entry per HTTP request');
|
||||
await (await manager.collect(f.session, group.id, group.fingerprint)).promise; assert.equal(group.status, 'collected', group.summary);
|
||||
const collection = await manager.readCollection(f.session, group); assert.deepEqual(collection.conflicts, []); assert.deepEqual(collection.paths, writable);
|
||||
@@ -252,7 +266,7 @@ try {
|
||||
assert.equal(results.length, 8); assert.equal(comparisons.length, 4);
|
||||
} else for (const kind of ['node', 'react', 'python']) results.push(await runSample(kind));
|
||||
assert.deepEqual(inputs(), frozen);
|
||||
report = { success: true, targetVersion: version, testedVersion: version, directory, installation, cases, results, comparisons, sources, inputs: frozen, realModelRequests: 0,
|
||||
report = { success: true, targetVersion: version, testedVersion: version, protocol: responsesMode ? 'responses-stream' : 'chat-completions', directory, installation, cases, results, comparisons, sources, inputs: frozen, realModelRequests: 0,
|
||||
scope: 'Pinned actual Harness and workbench execution with an owned HTTP model fixture. Checks, approvals, failures, parent merges, Edge and Git are real. Model reasoning and Purge benefit remain separate.' };
|
||||
} catch (error) { report = { success: false, targetVersion: version, testedVersion: version, directory, installation, cases, results, inputs: frozen, error: error.stack, realModelRequests: 0 }; process.exitCode = 1; }
|
||||
finally { fs.writeFileSync(path.join(directory, 'result.json'), JSON.stringify(report, null, 2)); fs.writeFileSync('.qa/' + (desktopMode ? 'desktop-projects' : comparisonMode ? 'controlled-purge' : 'controlled-projects') + '-location.json', JSON.stringify({ directory, result: path.join(directory, 'result.json') })); console.log('EVIDENCE ' + directory); console.log(JSON.stringify({ success: report.success, cases: cases.length, directory, realModelRequests: 0, error: report.error ?? null })); }
|
||||
finally { fs.writeFileSync(path.join(directory, 'result.json'), JSON.stringify(report, null, 2)); fs.writeFileSync('.qa/' + (responsesMode ? 'responses-engineering' : desktopMode ? 'desktop-projects' : comparisonMode ? 'controlled-purge' : 'controlled-projects') + '-location.json', JSON.stringify({ directory, result: path.join(directory, 'result.json') })); console.log('EVIDENCE ' + directory); console.log(JSON.stringify({ success: report.success, cases: cases.length, directory, realModelRequests: 0, error: report.error ?? null })); }
|
||||
|
||||
@@ -22,7 +22,7 @@ export async function verifyControlledApi({ api, url, token, directory, appRoot,
|
||||
const origin = new URL(url);
|
||||
assert.equal(origin.hostname, '127.0.0.1'); assert.notEqual(origin.port, '4318');
|
||||
const expectedVersion = JSON.parse(fs.readFileSync(path.join(appRoot, fs.existsSync(path.join(appRoot, 'MANIFEST.json')) ? 'MANIFEST.json' : 'package.json'), 'utf8')).version;
|
||||
assert.ok(['1.14.0', '1.15.0', '1.16.0'].includes(expectedVersion));
|
||||
assert.ok(['1.14.0', '1.15.0', '1.16.0', '1.17.0'].includes(expectedVersion));
|
||||
const bootstrap = await api('/bootstrap'); assert.equal(bootstrap.version, expectedVersion);
|
||||
assert.equal(bootstrap.config.mode, 'demo'); assert.equal(bootstrap.config.hasKey, false);
|
||||
const root = path.join(directory, 'compiled-controlled-node'), hooks = path.join(directory, 'disabled-hooks');
|
||||
|
||||
@@ -0,0 +1,159 @@
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import http from 'node:http';
|
||||
import https from 'node:https';
|
||||
import assert from 'node:assert/strict';
|
||||
import { createHash, randomUUID, X509Certificate } from 'node:crypto';
|
||||
import { chromium, expect } from 'playwright/test';
|
||||
import { fixture, launchWorkbench, until } from '../tests/helpers.mjs';
|
||||
|
||||
if (process.argv.includes('--help')) { console.log('node scripts/verify-model-protocols-ui.mjs [--app-root <absolute-package>] — owned HTTP Responses/SSE, connection persistence, scoped detection, pool rejection, original stream download and desktop/mobile stop. Explicit package uses contained Node/server. No real model calls or OS trust changes.'); process.exit(0); }
|
||||
const args = process.argv.slice(2); assert.ok(!args.length || args.length === 2 && args[0] === '--app-root' && path.isAbsolute(args[1]));
|
||||
const appRoot = fs.realpathSync.native(args.length ? args[1] : process.cwd()), nodePath = args.length ? path.join(appRoot, 'runtime/node.exe') : process.execPath;
|
||||
const directory = path.resolve('.qa', 'model-protocols-ui-' + randomUUID()); fs.mkdirSync(directory, { recursive: true });
|
||||
const disposers = [], t = { after: action => disposers.push(action) }, f = fixture(t); f.store.close();
|
||||
const requests = [], sockets = new Set(), held = new Set(); let mode = 'success', closedStreams = 0;
|
||||
const item = text => ({ type: 'message', id: 'msg_owned', status: 'completed', role: 'assistant', content: [{ type: 'output_text', text }] });
|
||||
const responseBody = output => ({ id: 'resp_owned', status: 'completed', output, usage: { input_tokens: 60, output_tokens: 20, total_tokens: 80 } });
|
||||
const modelServer = http.createServer(async (request, response) => {
|
||||
try {
|
||||
const parts = []; for await (const part of request) parts.push(part);
|
||||
const body = JSON.parse(Buffer.concat(parts)); requests.push({ route: request.url, body });
|
||||
assert.equal(request.url, '/v1/responses');
|
||||
const prompt = body.input.findLast(value => value.role === 'user')?.content ?? '';
|
||||
const probe = prompt === 'Reply OK.' || prompt.startsWith('Call workbench_capability_echo');
|
||||
const nonce = prompt.match(/nonce ([a-f0-9-]+)/)?.[1];
|
||||
const tool = { type: 'function_call', id: 'fc_owned', call_id: 'owned_call', name: nonce ? 'workbench_capability_echo' : 'list_files', arguments: JSON.stringify(nonce ? { nonce } : { path: '', depth: 1 }), status: 'completed' };
|
||||
const output = probe ? nonce ? [tool] : [item('OK')] : body.input.some(value => value.type === 'function_call_output') ? [item('项目读取完成。')] : [item('正在核对当前项目。'), tool];
|
||||
if (!body.stream) { response.writeHead(200, { 'Content-Type': 'application/json' }); response.end(JSON.stringify(responseBody(output))); return; }
|
||||
response.writeHead(200, { 'Content-Type': 'text/event-stream' }); let sequence = 0;
|
||||
const send = (type, data) => response.write('data: ' + JSON.stringify({ type, sequence_number: sequence++, ...data }) + '\r\n\r\n');
|
||||
send('response.created', { response: { id: 'resp_owned' } });
|
||||
if (!probe && mode !== 'success') {
|
||||
send('response.output_text.delta', { delta: '正在核对当前项目,尚未执行工具。' });
|
||||
send('response.output_item.added', { item: { ...tool, arguments: '', status: 'in_progress' }, output_index: 1 });
|
||||
send('response.function_call_arguments.delta', { item_id: tool.id, delta: '{"path":' });
|
||||
if (mode === 'truncate') response.end();
|
||||
else { held.add(response); response.on('close', () => { held.delete(response); closedStreams++; }); }
|
||||
return;
|
||||
}
|
||||
for (const value of output) {
|
||||
if (value.type === 'message') send('response.output_text.delta', { delta: value.content[0].text });
|
||||
else { send('response.output_item.added', { item: { ...value, arguments: '', status: 'in_progress' }, output_index: 1 }); send('response.function_call_arguments.delta', { item_id: value.id, delta: value.arguments }); send('response.function_call_arguments.done', { item_id: value.id, arguments: value.arguments }); }
|
||||
}
|
||||
send('response.completed', { response: responseBody(output) }); response.end();
|
||||
} catch { response.destroy(); }
|
||||
});
|
||||
modelServer.on('connection', socket => { sockets.add(socket); socket.on('close', () => sockets.delete(socket)); });
|
||||
await new Promise(resolve => modelServer.listen(0, '127.0.0.1', resolve));
|
||||
const host = await launchWorkbench(t, f.data, {}, { appRoot, nodePath }), browser = await chromium.launch({ channel: 'msedge', headless: true });
|
||||
assert.equal(fs.realpathSync.native(host.bootstrap.runtimeRoot), appRoot); assert.notEqual(new URL(host.url).port, '4318');
|
||||
const page = await browser.newPage({ viewport: { width: 1440, height: 1000 } }), errors = [], failures = [], passed = [], screenshots = [];
|
||||
page.on('pageerror', error => errors.push(error.message)); page.on('requestfailed', request => failures.push(request.url() + ': ' + request.failure()?.errorText));
|
||||
const mark = text => { passed.push(text); console.log('PASS ' + text); };
|
||||
async function shot(target, name) { const file = path.join(directory, name + '.png'); await target.screenshot({ path: file, fullPage: false }); screenshots.push(file); assert.ok(await target.locator('body').evaluate(node => node.scrollWidth <= innerWidth), name + ' page overflow'); }
|
||||
const getSession = () => host.api('/sessions/' + f.session.id);
|
||||
let mobile, relay, cookie = '', gateway;
|
||||
try {
|
||||
await page.addInitScript(({ projectId, sessionId }) => { localStorage.setItem('awb-project', projectId); localStorage.setItem('awb-session', sessionId); }, { projectId: f.project.id, sessionId: f.session.id });
|
||||
await page.goto(host.url); await page.waitForLoadState('networkidle');
|
||||
await page.getByRole('button', { name: '打开模型中心', exact: true }).click();
|
||||
const dialog = page.getByRole('dialog', { name: '模型中心', exact: true });
|
||||
await dialog.getByRole('button', { name: '添加连接', exact: true }).click();
|
||||
const form = dialog.getByRole('form', { name: '编辑模型连接', exact: true });
|
||||
await form.getByLabel('连接名称', { exact: true }).fill('验收 Responses');
|
||||
await form.getByLabel('接口协议', { exact: true }).selectOption('responses');
|
||||
await expect(form.getByLabel('使用流式输出', { exact: true })).toBeDisabled();
|
||||
await form.getByLabel('流式能力', { exact: true }).selectOption('supported'); await form.getByLabel('使用流式输出', { exact: true }).check();
|
||||
await form.getByLabel('接口根地址', { exact: true }).fill('http://127.0.0.1:' + modelServer.address().port + '/v1');
|
||||
await form.getByLabel('实际模型名称', { exact: true }).fill('owned-responses'); await form.getByLabel('服务需要 API Key', { exact: true }).uncheck();
|
||||
await shot(page, 'desktop-editor'); await page.setViewportSize({ width: 390, height: 844 }); await form.scrollIntoViewIfNeeded(); await shot(page, 'narrow-editor');
|
||||
await form.getByRole('button', { name: '保存模型连接', exact: true }).click(); await expect(form).toHaveCount(0);
|
||||
await page.reload(); await page.getByRole('button', { name: '打开模型中心', exact: true }).click();
|
||||
const card = dialog.locator('.mc-connection').filter({ hasText: '验收 Responses' }); await expect(card).toContainText('Responses · 流式开启');
|
||||
let center = await host.api('/model-center'), connection = center.connections.find(value => value.name === '验收 Responses');
|
||||
assert.equal(connection.protocol, 'responses'); assert.equal(connection.streaming, true);
|
||||
mark('actual form saves Responses and declared stream, refresh preserves both and narrow editor remains operable');
|
||||
await page.setViewportSize({ width: 1440, height: 1000 });
|
||||
for (const [scope, label] of [['text', '文本'], ['tools', '工具'], ['streaming', '流式']]) {
|
||||
await card.getByLabel('验收 Responses检测范围', { exact: true }).selectOption(scope);
|
||||
await card.getByRole('button', { name: '验证响应(调用模型)', exact: true }).click();
|
||||
await expect(dialog.locator('.mc-notice')).toContainText(label + '检测通过');
|
||||
await expect(card.locator('.mc-capabilities')).toContainText(label + ':已验证');
|
||||
if (scope === 'text') await expect(card.locator('.mc-capabilities')).toContainText('工具:声明支持');
|
||||
}
|
||||
await shot(page, 'desktop-capabilities'); await page.setViewportSize({ width: 390, height: 844 }); await shot(page, 'narrow-capabilities'); await page.setViewportSize({ width: 1440, height: 1000 });
|
||||
mark('separate text, nonce tool and actual SSE probes bind only their requested capability and own attempts');
|
||||
const text = await host.api('/model-center/connections', { name: '只支持文本', model: 'text-only', baseUrl: connection.baseUrl, requiresKey: false, capabilityDeclarations: { tools: 'unsupported' } });
|
||||
const textId = text.connection?.id ?? (await host.api('/model-center')).connections.find(value => value.name === '只支持文本').id;
|
||||
await page.reload(); await page.getByRole('button', { name: '打开模型中心', exact: true }).click();
|
||||
await dialog.getByRole('button', { name: '模型池', exact: true }).click(); await dialog.getByRole('button', { name: '新建模型池', exact: true }).click();
|
||||
const poolForm = dialog.getByRole('form', { name: '编辑模型池', exact: true }); await poolForm.getByLabel('模型池名称', { exact: true }).fill('按能力选择的验收池');
|
||||
for (const name of ['验收 Responses', '只支持文本']) await poolForm.locator('.mc-member-choice').filter({ hasText: name }).locator('input[type=checkbox]').check();
|
||||
await poolForm.getByRole('button', { name: '保存模型池', exact: true }).click(); await dialog.locator('.mc-pool').filter({ hasText: '按能力选择的验收池' }).getByRole('button', { name: '使用此模型池', exact: true }).click();
|
||||
await page.keyboard.press('Escape');
|
||||
await expect(page.locator('.model-button')).toContainText('按能力选择的验收池');
|
||||
await page.reload(); await expect(page.locator('.model-button')).toContainText('按能力选择的验收池');
|
||||
assert.equal((await host.api('/bootstrap')).config.model, '', 'The unselected legacy model remains untouched.');
|
||||
await page.getByLabel('任务内容', { exact: true }).fill('受控完整流程:读取项目后结束。'); await page.getByRole('button', { name: '发送任务', exact: true }).click();
|
||||
await until(async () => (await getSession()).status === 'completed', 15000);
|
||||
const completed = await getSession(); assert.equal(completed.events.filter(value => value.tool === 'list_files' && value.phase === 'completed').length, 1);
|
||||
center = await host.api('/model-center'); const attempt = center.attempts.find(value => value.runId === completed.runId);
|
||||
assert.equal(attempt.protocol, 'responses'); assert.ok(attempt.rejections.some(value => value.connectionId === textId && value.capability === 'tools')); assert.ok(requests.every(value => value.body.model !== 'text-only'));
|
||||
await page.getByRole('button', { name: '打开模型中心', exact: true }).click(); await dialog.getByRole('button', { name: '请求账本', exact: true }).click();
|
||||
await expect(dialog.locator('.mc-route-rejections').first()).toContainText('只支持文本');
|
||||
const downloadEvent = page.waitForEvent('download'); await dialog.getByRole('button', { name: '下载已保存的响应', exact: true }).first().click();
|
||||
const download = await downloadEvent, saved = path.join(directory, 'original-stream.sse'); await download.saveAs(saved);
|
||||
assert.equal(createHash('sha256').update(fs.readFileSync(saved)).digest('hex'), attempt.responseSha256); await shot(page, 'desktop-ledger');
|
||||
mark('actual pooled task excludes text-only before dispatch, executes original tool once and downloads SHA-matched original SSE');
|
||||
await page.keyboard.press('Escape');
|
||||
|
||||
const initial = await host.api('/companion'); assert.ok(initial.addresses.length);
|
||||
const reservation = http.createServer(); await new Promise(resolve => reservation.listen(0, initial.addresses[0].address, resolve)); const port = reservation.address().port; await new Promise(resolve => reservation.close(resolve));
|
||||
await host.api('/companion/start', { address: initial.addresses[0].address, port }); gateway = await host.api('/companion');
|
||||
const cert = await fetch(host.url + '/api/companion/certificate', { headers: { 'X-Workbench-Token': host.bootstrap.token } }); assert.ok(cert.ok);
|
||||
const ca = new X509Certificate(Buffer.from(await cert.arrayBuffer())).toString(), target = new URL(gateway.origin);
|
||||
const ticket = await host.api('/companion/pair', { projectIds: [f.project.id], permissions: { observe: true, approve: false, stop: true }, expiresInHours: 1 });
|
||||
relay = http.createServer((request, response) => {
|
||||
const forwarded = https.request(new URL(request.url, target), { method: request.method, ca, servername: '', headers: { Host: target.host, Origin: target.origin, Connection: 'close', ...(cookie ? { Cookie: cookie } : {}), ...(request.headers['content-type'] ? { 'Content-Type': request.headers['content-type'] } : {}) } }, incoming => {
|
||||
if (incoming.headers['set-cookie']?.length) cookie = incoming.headers['set-cookie'][0].split(';')[0]; const headers = { ...incoming.headers }; delete headers['set-cookie']; response.writeHead(incoming.statusCode, headers); incoming.pipe(response);
|
||||
}); forwarded.on('error', () => response.destroy()); request.pipe(forwarded);
|
||||
});
|
||||
await new Promise(resolve => relay.listen(0, '127.0.0.1', resolve));
|
||||
mobile = await browser.newPage({ viewport: { width: 390, height: 844 }, isMobile: true, hasTouch: true }); mobile.on('pageerror', error => errors.push(error.message));
|
||||
await mobile.goto('http://127.0.0.1:' + relay.address().port + '/#pair=' + new URLSearchParams(new URL(ticket.url).hash.slice(1)).get('pair'));
|
||||
await mobile.getByLabel('设备名称', { exact: true }).fill('协议验收观察窗口'); await mobile.getByRole('button', { name: '连接这台电脑', exact: true }).click(); await expect(mobile.getByRole('heading', { name: '项目任务', exact: true })).toBeVisible();
|
||||
mode = 'hold'; const count = requests.length;
|
||||
await page.getByLabel('任务内容', { exact: true }).fill('受控停止:保留片段。'); await page.getByRole('button', { name: '发送任务', exact: true }).click();
|
||||
await expect(page.getByLabel('模型输出片段', { exact: true }).last()).toContainText('正在核对当前项目');
|
||||
const active = await getSession(); assert.equal(active.events.filter(value => value.kind === 'tool' && value.runId === active.runId).length, 0);
|
||||
await shot(page, 'desktop-streaming'); await page.setViewportSize({ width: 390, height: 844 }); await shot(page, 'narrow-streaming');
|
||||
await mobile.getByRole('button', { name: '刷新任务', exact: true }).click(); await mobile.locator('.mobile-task-row').first().click();
|
||||
await expect(mobile.locator('.mobile-stream')).toContainText('正在核对当前项目'); await mobile.locator('.mobile-stream').scrollIntoViewIfNeeded(); await shot(mobile, 'mobile-streaming');
|
||||
await mobile.getByRole('button', { name: '停止原任务', exact: true }).click(); await mobile.getByRole('dialog').getByRole('button', { name: '确认', exact: true }).click();
|
||||
await until(async () => (await getSession()).status === 'paused' && closedStreams === 1, 15000);
|
||||
await expect(mobile.locator('.mobile-stream')).toContainText('输出未完成');
|
||||
assert.equal(requests.length, count + 1); await mobile.locator('.mobile-stream').scrollIntoViewIfNeeded(); await shot(mobile, 'mobile-stopped');
|
||||
await expect(page.getByLabel('模型输出片段', { exact: true }).last()).toContainText('输出未完成'); await shot(page, 'narrow-stopped');
|
||||
mark('desktop and actual paired gateway show same bounded stream; mobile stops original HTTP with zero fragment tool dispatch or replay');
|
||||
await page.setViewportSize({ width: 1440, height: 1000 }); mode = 'truncate';
|
||||
await page.getByLabel('任务内容', { exact: true }).fill('受控断流:半截参数不得执行。'); await page.getByRole('button', { name: '发送任务', exact: true }).click();
|
||||
await until(async () => (await getSession()).status === 'failed', 15000);
|
||||
const broken = await getSession(), latest = (await host.api('/model-center')).attempts.find(value => value.runId === broken.runId);
|
||||
assert.equal(latest.status, 'unknown'); assert.equal(latest.stream.terminal, false); assert.equal(requests.length, count + 2);
|
||||
await page.getByRole('button', { name: '打开模型中心', exact: true }).click(); await dialog.getByRole('button', { name: '请求账本', exact: true }).click();
|
||||
await expect(dialog.locator('.mc-attempts article').first()).toContainText('结果待核对'); await expect(dialog.locator('.mc-attempts article').first()).toContainText('响应未完整'); await shot(page, 'desktop-incomplete');
|
||||
await page.setViewportSize({ width: 390, height: 844 }); await shot(page, 'narrow-incomplete');
|
||||
mark('truncated output keeps explicit unknown ledger, partial evidence and no automatic failover or incomplete tool execution');
|
||||
assert.deepEqual(errors, []); assert.deepEqual(failures, []);
|
||||
fs.writeFileSync(path.join(directory, 'result.json'), JSON.stringify({ success: true, version: host.bootstrap.version, appRoot, nodePath, containedRuntime: args.length > 0, nodeSha256: createHash('sha256').update(fs.readFileSync(nodePath)).digest('hex'), passed, screenshots, pageErrors: errors, requestFailures: failures, fixtureModelRequests: requests.length, realModelRequests: 0, osTrustChanged: false, nativeIPhone: 'not exercised; paired Edge viewport uses CA-verifying owned relay', taskId: completed.taskId, runIds: [completed.runId, active.runId, broken.runId] }, null, 2));
|
||||
} catch (error) {
|
||||
await shot(page, 'failure-desktop').catch(() => {}); if (mobile) await shot(mobile, 'failure-mobile').catch(() => {});
|
||||
fs.writeFileSync(path.join(directory, 'result.json'), JSON.stringify({ success: false, error: error.stack, passed, screenshots, pageErrors: errors, requestFailures: failures }, null, 2)); throw error;
|
||||
} finally {
|
||||
console.log('EVIDENCE ' + directory); await browser.close();
|
||||
if (relay) await new Promise(resolve => { relay.closeAllConnections(); relay.close(resolve); });
|
||||
if (gateway) await host.api('/companion/close', {});
|
||||
for (const response of held) response.destroy(); for (const socket of sockets) socket.destroy(); await new Promise(resolve => modelServer.close(resolve));
|
||||
for (const dispose of disposers.reverse()) await dispose();
|
||||
}
|
||||
@@ -34,7 +34,7 @@ for (const name of Object.keys(env)) if (name.toLowerCase() === 'path') delete e
|
||||
const windowsRoot = process.env.SystemRoot || 'C:\\Windows';
|
||||
env.PATH = [windowsRoot, path.join(windowsRoot, 'System32'), path.join(windowsRoot, 'System32', 'WindowsPowerShell', 'v1.0')].join(path.delimiter);
|
||||
let externalGit = null;
|
||||
if (['1.14.0', '1.15.0', '1.16.0'].includes(manifest.version)) {
|
||||
if (['1.14.0', '1.15.0', '1.16.0', '1.17.0'].includes(manifest.version)) {
|
||||
// The controlled project requires external Git. Preserve its observed binary
|
||||
// directory while keeping system Node/npm unavailable to the packaged host.
|
||||
const binary = fs.realpathSync.native(execFileSync('where.exe', ['git.exe'], { env: gitEnvironment(), encoding: 'utf8', windowsHide: true, timeout: 10000 }).trim().split(/\r?\n/)[0]);
|
||||
@@ -88,7 +88,7 @@ try {
|
||||
console.log((await launch()).trim());
|
||||
assert.equal(JSON.parse(fs.readFileSync(path.join(data, 'service-info.json'), 'utf8')).pid, firstInfo.pid);
|
||||
const example = bootstrap.projects.find(project => project.id === bootstrap.exampleProjectId);
|
||||
if (['1.15.0', '1.16.0'].includes(manifest.version)) {
|
||||
if (['1.15.0', '1.16.0', '1.17.0'].includes(manifest.version)) {
|
||||
assert.equal(path.resolve(example.root).toLowerCase(), path.join(data, 'projects', 'starter-project').toLowerCase());
|
||||
const compareTemplate = relative => { for (const entry of fs.readdirSync(path.join(appRoot, 'examples/starter-project', relative), { withFileTypes: true })) {
|
||||
const next = path.join(relative, entry.name); if (entry.isDirectory()) compareTemplate(next);
|
||||
@@ -114,7 +114,7 @@ try {
|
||||
const checkLog = await api('/artifacts/' + checked.results[0].logArtifactId + '?session=' + session.id); assert.match(checkLog.content, /tests 4/);
|
||||
assert.ok(fs.existsSync(path.join(appRoot, 'runtime/node_modules/npm/bin/npm-cli.js')));
|
||||
const actualRea = [];
|
||||
if (['1.8.0', '1.9.0', '1.10.0', '1.10.1', '1.11.0', '1.12.0', '1.12.1', '1.13.0', '1.14.0', '1.15.0', '1.16.0'].includes(manifest.version)) {
|
||||
if (['1.8.0', '1.9.0', '1.10.0', '1.10.1', '1.11.0', '1.12.0', '1.12.1', '1.13.0', '1.14.0', '1.15.0', '1.16.0', '1.17.0'].includes(manifest.version)) {
|
||||
const installation = await api('/research/availability'); assert.equal(installation.status, 'available'); assert.equal(installation.version, '6.8.0');
|
||||
assert.equal(fs.realpathSync.native(installation.packageRoot), fs.realpathSync.native(path.join(appRoot, 'runtime/node_modules/rea-agents')));
|
||||
assert.equal(fs.realpathSync.native(installation.node.path), fs.realpathSync.native(path.join(appRoot, 'runtime/node.exe')));
|
||||
@@ -146,7 +146,7 @@ try {
|
||||
console.log('PASS actual contained REA: ASAR, PE imports and PE resources; zero target execution/model requests');
|
||||
}
|
||||
let actualEngine = null;
|
||||
if (['1.9.0', '1.10.0', '1.10.1', '1.11.0', '1.12.0', '1.12.1', '1.13.0', '1.14.0', '1.15.0', '1.16.0'].includes(manifest.version) && process.argv.includes('--native-engines')) {
|
||||
if (['1.9.0', '1.10.0', '1.10.1', '1.11.0', '1.12.0', '1.12.1', '1.13.0', '1.14.0', '1.15.0', '1.16.0', '1.17.0'].includes(manifest.version) && process.argv.includes('--native-engines')) {
|
||||
const installation = await api('/engines/deepseek/availability'); assert.equal(installation.status, 'available', installation.message);
|
||||
assert.equal(fs.realpathSync.native(installation.node.path), fs.realpathSync.native(path.join(appRoot, 'runtime/node.exe')));
|
||||
const engineRoot = path.join(data, 'engine-project'); fs.mkdirSync(engineRoot); fs.writeFileSync(path.join(engineRoot, 'README.md'), '# Owned portable engine connection\n');
|
||||
@@ -166,8 +166,8 @@ try {
|
||||
assert.equal((await api('/research')).length, 4);
|
||||
const page = await fetch(url); assert.equal(page.status, 200);
|
||||
assert.match(await page.text(), /Agent Workbench/);
|
||||
const actualBrowser = ['1.10.0', '1.10.1', '1.11.0', '1.12.0', '1.12.1', '1.13.0', '1.14.0', '1.15.0', '1.16.0'].includes(manifest.version) ? await verifyBrowserApi({ api, url, token: bootstrap.token, directory: data, appRoot, contained: true }) : null;
|
||||
const actualControlled = ['1.14.0', '1.15.0', '1.16.0'].includes(manifest.version) ? await verifyControlledApi({ api, url, token: bootstrap.token, directory: data, appRoot, contained: true }) : null;
|
||||
const actualBrowser = ['1.10.0', '1.10.1', '1.11.0', '1.12.0', '1.12.1', '1.13.0', '1.14.0', '1.15.0', '1.16.0', '1.17.0'].includes(manifest.version) ? await verifyBrowserApi({ api, url, token: bootstrap.token, directory: data, appRoot, contained: true }) : null;
|
||||
const actualControlled = ['1.14.0', '1.15.0', '1.16.0', '1.17.0'].includes(manifest.version) ? await verifyControlledApi({ api, url, token: bootstrap.token, directory: data, appRoot, contained: true }) : null;
|
||||
console.log((await stop()).trim()); started = false;
|
||||
await until(async () => { try { await fetch(url + '/api/health'); return false; } catch { return true; } });
|
||||
for (const file of manifest.files) {
|
||||
@@ -176,7 +176,7 @@ try {
|
||||
}
|
||||
const result = { success: true, date: new Date().toISOString(), appRoot, version: manifest.version, nodeVersion: manifest.nodeVersion,
|
||||
verifiedFiles: manifest.files.length, url, reusedPid: firstInfo.pid, checks: { jobId, status: checked.status, tests: checked.results[0].tests }, actualRea, actualEngine, actualBrowser, actualControlled, externalGit,
|
||||
steps: ['manifest hashes', 'bundled Node launch', 'same-service reuse', 'packaged example and static page', ...(['1.15.0', '1.16.0'].includes(manifest.version) ? ['starter project copied to owned user data; original template bytes matched'] : []), 'explicit command consent', 'actual bundled Node version', 'actual bundled npm check without system Node/npm', 'check log artifact', ...(actualRea.length ? ['contained REA installation and Node identity', 'actual ASAR/imports/resources', 'original Bundle and target hash binding', 'verified reuse without producer replay'] : []), 'report availability', 'graceful stop', 'every original packaged file remains unchanged'], realModel: false };
|
||||
steps: ['manifest hashes', 'bundled Node launch', 'same-service reuse', 'packaged example and static page', ...(['1.15.0', '1.16.0', '1.17.0'].includes(manifest.version) ? ['starter project copied to owned user data; original template bytes matched'] : []), 'explicit command consent', 'actual bundled Node version', 'actual bundled npm check without system Node/npm', 'check log artifact', ...(actualRea.length ? ['contained REA installation and Node identity', 'actual ASAR/imports/resources', 'original Bundle and target hash binding', 'verified reuse without producer replay'] : []), 'report availability', 'graceful stop', 'every original packaged file remains unchanged'], realModel: false };
|
||||
fs.writeFileSync('.qa/release-validation.json', JSON.stringify(result, null, 2));
|
||||
console.log(JSON.stringify(result, null, 2));
|
||||
} finally { if (started) { try { await stop(); } catch {} } }
|
||||
|
||||
@@ -19,7 +19,7 @@ assert.ok(fs.existsSync(npmCli), 'Use Node with its bundled npm.');
|
||||
const files = (root, relative = '') => fs.readdirSync(path.join(root, relative), { withFileTypes: true }).sort((a, b) => a.name.localeCompare(b.name))
|
||||
.flatMap(entry => entry.isDirectory() ? files(root, path.join(relative, entry.name)) : [path.join(relative, entry.name)]);
|
||||
const digest = file => createHash('sha256').update(fs.readFileSync(file)).digest('hex');
|
||||
const buildFolders = ['dist-client', 'dist-server', ...(['1.15.0', '1.16.0'].includes(evidence.version) ? ['dist-desktop'] : [])];
|
||||
const buildFolders = ['dist-client', 'dist-server', ...(['1.15.0', '1.16.0', '1.17.0'].includes(evidence.version) ? ['dist-desktop'] : [])];
|
||||
const expected = Object.fromEntries(buildFolders.flatMap(folder => files(path.join(workspace, folder)).map(relative => [path.join(folder, relative), digest(path.join(workspace, folder, relative))])));
|
||||
const allowedEnv = /^(path|pathext|systemroot|systemdrive|windir|comspec|temp|tmp|userprofile|homedrive|homepath|home|appdata|localappdata|programfiles(?:\(x86\))?|commonprogramfiles(?:\(x86\))?|lang|lc_all|AGENT_WORKBENCH_TEST_PYTHON|AGENT_WORKBENCH_DEEPSEEK_ROOT|AGENT_WORKBENCH_PURGE_ROOT)$/i;
|
||||
const env = Object.fromEntries(Object.entries(process.env).filter(([key]) => allowedEnv.test(key)));
|
||||
@@ -30,7 +30,7 @@ const directory = path.join(sourceRoot, '.qa', 'package-validation-' + randomUUI
|
||||
fs.mkdirSync(directory, { recursive: true });
|
||||
const inputFolders = ['src', 'tests', 'scripts', ...['benchmarks', 'public', 'examples', 'research'].filter(folder => fs.existsSync(path.join(sourceRoot, folder)))];
|
||||
const inputNames = [...inputFolders.flatMap(folder => files(path.join(sourceRoot, folder)).map(relative => path.join(folder, relative))),
|
||||
'package.json', 'package-lock.json', 'vite.config.ts', 'tsconfig.json', 'index.html', ...(evidence.version === '1.16.0' ? ['mobile.html'] : []), ...['.gitattributes', '.gitignore', 'Start-Desktop.cmd', 'Start-Workbench.cmd', 'Stop-Workbench.cmd'].filter(name => fs.existsSync(path.join(sourceRoot, name)))].sort();
|
||||
'package.json', 'package-lock.json', 'vite.config.ts', 'tsconfig.json', 'index.html', ...(['1.16.0', '1.17.0'].includes(evidence.version) ? ['mobile.html'] : []), ...['.gitattributes', '.gitignore', 'Start-Desktop.cmd', 'Start-Workbench.cmd', 'Stop-Workbench.cmd'].filter(name => fs.existsSync(path.join(sourceRoot, name)))].sort();
|
||||
const snapshot = () => inputNames.map(relative => ({ path: relative.replaceAll('\\', '/'), bytes: fs.statSync(path.join(sourceRoot, relative)).size, sha256: digest(path.join(sourceRoot, relative)) }));
|
||||
const inputs = snapshot(); fs.writeFileSync(path.join(directory, 'inputs.json'), JSON.stringify(inputs, null, 2));
|
||||
let stage = 'environment preflight', python = null;
|
||||
@@ -83,7 +83,7 @@ try {
|
||||
assert.deepEqual(actualFiles.slice().sort(), Object.keys(expected).sort());
|
||||
for (const [relative, sha256] of Object.entries(expected)) assert.equal(digest(path.join(sourceRoot, relative)), sha256, relative);
|
||||
let actualRea = null;
|
||||
if (['1.8.0', '1.9.0', '1.10.0', '1.10.1', '1.11.0', '1.12.0', '1.12.1', '1.13.0', '1.14.0', '1.15.0', '1.16.0'].includes(evidence.version)) {
|
||||
if (['1.8.0', '1.9.0', '1.10.0', '1.10.1', '1.11.0', '1.12.0', '1.12.1', '1.13.0', '1.14.0', '1.15.0', '1.16.0', '1.17.0'].includes(evidence.version)) {
|
||||
stage = 'contained REA';
|
||||
const reaOutput = await run(['scripts/verify-rea.mjs', '--app-root', sourceRoot, '--contained'], 'rea.log');
|
||||
actualRea = JSON.parse(reaOutput.trim().split('\n').at(-1));
|
||||
@@ -91,7 +91,7 @@ try {
|
||||
assert.ok(actualRea.results.every(item => item.rootExited === true));
|
||||
}
|
||||
let actualEngine = null;
|
||||
if (['1.9.0', '1.10.0', '1.10.1', '1.11.0', '1.12.0', '1.12.1', '1.13.0', '1.14.0', '1.15.0', '1.16.0'].includes(evidence.version) && process.argv.includes('--native-engines')) {
|
||||
if (['1.9.0', '1.10.0', '1.10.1', '1.11.0', '1.12.0', '1.12.1', '1.13.0', '1.14.0', '1.15.0', '1.16.0', '1.17.0'].includes(evidence.version) && process.argv.includes('--native-engines')) {
|
||||
stage = 'native DeepSeek SDK';
|
||||
const output = await run(['scripts/verify-engines.mjs'], 'engines.log');
|
||||
actualEngine = JSON.parse(output.trim().split('\n').at(-1));
|
||||
@@ -99,7 +99,7 @@ try {
|
||||
assert.equal(actualEngine.modelRequests, 0); assert.equal(actualEngine.rootExited, true); assert.equal(actualEngine.nativeMethods.length, 6);
|
||||
}
|
||||
let actualBrowser = null;
|
||||
if (['1.10.0', '1.10.1', '1.11.0', '1.12.0', '1.12.1', '1.13.0', '1.14.0', '1.15.0', '1.16.0'].includes(evidence.version)) {
|
||||
if (['1.10.0', '1.10.1', '1.11.0', '1.12.0', '1.12.1', '1.13.0', '1.14.0', '1.15.0', '1.16.0', '1.17.0'].includes(evidence.version)) {
|
||||
stage = 'contained browser runtime';
|
||||
const output = await run(['scripts/verify-browser.mjs', '--app-root', sourceRoot, '--contained'], 'browser.log');
|
||||
actualBrowser = JSON.parse(output.trim().split('\n').at(-1));
|
||||
@@ -109,7 +109,7 @@ try {
|
||||
assert.equal(actualBrowser.modelRequests, 0);
|
||||
}
|
||||
let actualControlled = null;
|
||||
if (['1.14.0', '1.15.0', '1.16.0'].includes(evidence.version)) {
|
||||
if (['1.14.0', '1.15.0', '1.16.0', '1.17.0'].includes(evidence.version)) {
|
||||
stage = 'compiled controlled-engine delivery';
|
||||
const output = await run(['scripts/verify-controlled-release.mjs', '--app-root', sourceRoot], 'controlled-engine.log');
|
||||
actualControlled = JSON.parse(output.trim().split('\n').at(-1));
|
||||
@@ -117,7 +117,7 @@ try {
|
||||
assert.equal(actualControlled.realModelRequests, 0); assert.equal(actualControlled.cases, 4); assert.ok(actualControlled.fixtureModelRequests > 0);
|
||||
}
|
||||
let actualCompanion = null;
|
||||
if (evidence.version === '1.16.0') {
|
||||
if (['1.16.0', '1.17.0'].includes(evidence.version)) {
|
||||
stage = 'contained companion gateway and UI';
|
||||
const output = await run(['scripts/verify-companion-ui.mjs', '--app-root', sourceRoot], 'companion-ui.log');
|
||||
const location = [...output.matchAll(/^EVIDENCE (.+)\r?$/gm)].at(-1)?.[1]?.trim();
|
||||
@@ -127,12 +127,23 @@ try {
|
||||
assert.equal(fs.realpathSync.native(actualCompanion.appRoot), sourceRoot); assert.equal(actualCompanion.containedRuntime, true);
|
||||
assert.equal(actualCompanion.passed.length, 7); assert.deepEqual(actualCompanion.errors, []); assert.equal(actualCompanion.realModelRequests, 0);
|
||||
}
|
||||
let actualProtocols = null;
|
||||
if (evidence.version === '1.17.0') {
|
||||
stage = 'contained model protocols and mobile streams';
|
||||
const output = await run(['scripts/verify-model-protocols-ui.mjs', '--app-root', sourceRoot], 'model-protocols-ui.log');
|
||||
const location = [...output.matchAll(/^EVIDENCE (.+)\r?$/gm)].at(-1)?.[1]?.trim();
|
||||
assert.ok(location && path.resolve(location).startsWith(path.join(sourceRoot, '.qa') + path.sep));
|
||||
actualProtocols = JSON.parse(fs.readFileSync(path.join(location, 'result.json'), 'utf8'));
|
||||
assert.equal(actualProtocols.success, true); assert.equal(actualProtocols.version, evidence.version);
|
||||
assert.equal(fs.realpathSync.native(actualProtocols.appRoot), sourceRoot); assert.equal(actualProtocols.containedRuntime, true);
|
||||
assert.equal(actualProtocols.passed.length, 5); assert.deepEqual(actualProtocols.pageErrors, []); assert.deepEqual(actualProtocols.requestFailures, []);
|
||||
}
|
||||
stage = 'tests';
|
||||
const output = await run(['--test', '--test-concurrency=2', '--test-reporter=tap', ...files(path.join(sourceRoot, 'tests')).filter(file => !file.includes(path.sep) && file.endsWith('.test.mjs')).map(file => path.join('tests', file))], 'tests.tap');
|
||||
const stats = Object.fromEntries([...output.matchAll(/^# (tests|pass|fail|cancelled|skipped|todo) (\d+)\s*$/gm)].map(match => [match[1], Number(match[2])]));
|
||||
assert.ok(stats.tests > 0); assert.equal(stats.pass, stats.tests); for (const key of ['fail', 'cancelled', 'skipped', 'todo']) assert.equal(stats[key], 0);
|
||||
let actualDesktop = null;
|
||||
if (['1.15.0', '1.16.0'].includes(evidence.version)) {
|
||||
if (['1.15.0', '1.16.0', '1.17.0'].includes(evidence.version)) {
|
||||
stage = 'actual contained Electron window'; await run(['scripts/verify-desktop-window.mjs'], 'desktop-window.log');
|
||||
const window = JSON.parse(fs.readFileSync(path.join(sourceRoot, '.qa/desktop-window-latest.json'), 'utf8'));
|
||||
assert.equal(window.success, true); assert.equal(path.resolve(window.appRoot), sourceRoot); assert.equal(window.results.length, 7);
|
||||
@@ -145,9 +156,19 @@ try {
|
||||
assert.deepEqual(projects.results.map(value => value.parentCheck.results[0].tests.passed), [8, 9, 10]);
|
||||
actualDesktop = { window, ui, projects, scope: 'Independently rebuilt source and its shipped Electron binary; original window/engine/engineering assertions. Own credentials/HTTP fixtures; Windows native picker and complete host crash/stop have separate actual evidence.' };
|
||||
}
|
||||
let actualProtocolEngineering = null;
|
||||
if (evidence.version === '1.17.0') {
|
||||
stage = 'Responses SSE actual Node React Python engineering';
|
||||
await run(['scripts/verify-controlled-projects.mjs', '--responses'], 'responses-engineering.log');
|
||||
const location = JSON.parse(fs.readFileSync(path.join(sourceRoot, '.qa/responses-engineering-location.json'), 'utf8'));
|
||||
actualProtocolEngineering = JSON.parse(fs.readFileSync(location.result, 'utf8'));
|
||||
assert.equal(actualProtocolEngineering.success, true); assert.equal(actualProtocolEngineering.testedVersion, evidence.version);
|
||||
assert.equal(actualProtocolEngineering.protocol, 'responses-stream'); assert.equal(actualProtocolEngineering.realModelRequests, 0);
|
||||
assert.deepEqual(actualProtocolEngineering.results.map(value => value.parentCheck.results[0].tests.passed), [8, 9, 10]);
|
||||
}
|
||||
assert.deepEqual(snapshot(), inputs, 'Source package inputs changed during validation.');
|
||||
const result = { success: true, version: evidence.version, testedAt: new Date().toISOString(), sourceRoot, directory, nodeVersion: process.version, python, offlineInstall: true,
|
||||
inputs, inputFiles: inputs.length, logs, buildMatchedFiles: Object.keys(expected).length, buildFiles: Object.entries(expected).map(([path, sha256]) => ({ path: path.replaceAll('\\', '/'), sha256 })), tests: stats, actualRea, actualEngine, actualBrowser, actualControlled, actualDesktop, actualCompanion, credentialsHandling: 'Only build/runtime environment paths are inherited; no configured model credentials are used. The controlled-engine stage supplies a declared owned local response fixture.' };
|
||||
inputs, inputFiles: inputs.length, logs, buildMatchedFiles: Object.keys(expected).length, buildFiles: Object.entries(expected).map(([path, sha256]) => ({ path: path.replaceAll('\\', '/'), sha256 })), tests: stats, actualRea, actualEngine, actualBrowser, actualControlled, actualDesktop, actualCompanion, actualProtocols, actualProtocolEngineering, credentialsHandling: 'Only build/runtime environment paths are inherited; no configured model credentials are used. The controlled-engine stage supplies a declared owned local response fixture.' };
|
||||
save(result); console.log(JSON.stringify({ success: true, version: result.version, sourceRoot, directory, tests: stats, nodeVersion: process.version, python, inputFiles: inputs.length, buildMatchedFiles: result.buildMatchedFiles, rea: !!actualRea?.success, engine: !!actualEngine?.success, browser: !!actualBrowser?.success, controlled: !!actualControlled?.success }));
|
||||
} catch (error) {
|
||||
save({ success: false, version: evidence.version, testedAt: new Date().toISOString(), stage, error: error.message, sourceRoot, directory, nodeVersion: process.version, python, inputs, logs });
|
||||
|
||||
@@ -8,7 +8,7 @@ import { workerHttp } from '../tests/fixtures/workers-http.mjs';
|
||||
import { workerDefinition, ownerBytes, applyWorkerProposals, approveWorkerChecks } from '../tests/fixtures/workers-project.mjs';
|
||||
|
||||
if (process.argv.includes('--help')) { console.log('node scripts/verify-workers-ui.mjs — owned local fixture, real isolated Git/checks/review, desktop/390px, conflicts, stop, recovery and late source errors'); process.exit(0); }
|
||||
const version = JSON.parse(fs.readFileSync('package.json', 'utf8')).version, prefix = version === '1.16.0' ? 'mobile-companion-workers' : version === '1.15.0' ? 'desktop-host-workers' : version === '1.14.0' ? 'controlled-engine-workers' : version === '1.13.0' ? 'workers' : 'workers-development';
|
||||
const version = JSON.parse(fs.readFileSync('package.json', 'utf8')).version, prefix = version === '1.17.0' ? 'model-protocols-workers' : version === '1.16.0' ? 'mobile-companion-workers' : version === '1.15.0' ? 'desktop-host-workers' : version === '1.14.0' ? 'controlled-engine-workers' : version === '1.13.0' ? 'workers' : 'workers-development';
|
||||
const directory = path.resolve('.qa', 'workers-ui-' + randomUUID()); fs.mkdirSync(directory, { recursive: true });
|
||||
const disposers = [], t = { after: dispose => disposers.push(dispose) }, f = fixture(t), host = await workerHttp(f);
|
||||
assert.notEqual(new URL(host.url).port, '4318');
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { engineError, engineCanonicalDigest as controlledDigest } from './engines.mjs';
|
||||
import { requestMetrics } from './model-request.mjs';
|
||||
import { requestMetrics, requestVariants, outputReservation } from './model-request.mjs';
|
||||
|
||||
export const ENGINE_RUN_ACTIVE = new Set(['queued', 'starting', 'initializing', 'running', 'waiting_tool', 'closing', 'stopping']);
|
||||
export function selectedEngineInput(input) {
|
||||
@@ -55,7 +55,9 @@ export function verifyEngineModelInput(store, session, job, request) {
|
||||
if (input.owner.sessionId !== job.sessionId || input.owner.taskId !== job.taskId || input.owner.runId !== job.runId || input.owner.projectId !== job.projectId
|
||||
|| controlledDigest(input.native) !== request.sourceSha256 || metrics.sha256 !== request.requestSha256 || metrics.bytes !== request.requestBytes
|
||||
|| !Number.isSafeInteger(request.maxAttempts) || request.maxAttempts < 1 || request.maxAttempts > 8
|
||||
|| request.reservedPerAttempt !== metrics.estimatedInputTokens + 128 + input.body.max_tokens) throw engineError('原引擎模型输入、归属或预算核对失败。');
|
||||
|| !Number.isSafeInteger(input.native.maxTokens) || input.native.maxTokens < 1
|
||||
|| !requestVariants(input.body).every(body => outputReservation(body) === input.native.maxTokens)
|
||||
|| request.reservedPerAttempt !== metrics.estimatedInputTokens + 128 + input.native.maxTokens) throw engineError('原引擎模型输入、归属或预算核对失败。');
|
||||
return input;
|
||||
}
|
||||
export function assertEngineWorkerResult(store, session, worker) {
|
||||
|
||||
+33
@@ -0,0 +1,33 @@
|
||||
import http from 'node:http';
|
||||
import assert from 'node:assert/strict';
|
||||
import { requestHistory } from '../../src/shared/contracts/model-request.mjs';
|
||||
|
||||
// An owned wire fixture only: the original engineering algorithm receives the
|
||||
// same canonical history. Runtime/Harness/checks/Git still execute for real.
|
||||
export async function responsesEngineeringHttp(t, algorithm) {
|
||||
const requests = [], sockets = new Set();
|
||||
const server = http.createServer(async (request, response) => {
|
||||
try {
|
||||
const parts = []; for await (const part of request) parts.push(part);
|
||||
const wire = JSON.parse(Buffer.concat(parts)); assert.equal(request.url, '/v1/responses'); assert.equal(wire.stream, true);
|
||||
requests.push(wire);
|
||||
const input = { ...wire, messages: requestHistory(wire) }, result = await algorithm('/v1/chat/completions', input);
|
||||
const message = result.choices[0].message;
|
||||
const output = [...(message.content ? [{ id: 'msg_' + requests.length, type: 'message', role: 'assistant', status: 'completed', content: [{ type: 'output_text', text: message.content }] }] : []),
|
||||
...(message.tool_calls ?? []).map((call, index) => ({ type: 'function_call', id: 'fc_' + requests.length + '_' + index, call_id: call.id, name: call.function.name, arguments: call.function.arguments, status: 'completed' }))];
|
||||
const final = { id: 'resp_' + requests.length, status: 'completed', output, usage: { input_tokens: result.usage.prompt_tokens, output_tokens: result.usage.completion_tokens, total_tokens: result.usage.total_tokens } };
|
||||
response.writeHead(200, { 'Content-Type': 'text/event-stream' }); let sequence = 0;
|
||||
const send = (type, data) => response.write('data: ' + JSON.stringify({ type, sequence_number: sequence++, ...data }) + '\n\n');
|
||||
send('response.created', { response: { id: final.id } });
|
||||
for (const item of output) {
|
||||
if (item.type === 'message') send('response.output_text.delta', { delta: item.content[0].text });
|
||||
else { send('response.output_item.added', { item: { ...item, arguments: '', status: 'in_progress' } }); send('response.function_call_arguments.delta', { item_id: item.id, delta: item.arguments }); send('response.function_call_arguments.done', { item_id: item.id, arguments: item.arguments }); }
|
||||
}
|
||||
send('response.completed', { response: final }); response.end();
|
||||
} catch { response.destroy(); }
|
||||
});
|
||||
server.on('connection', socket => { sockets.add(socket); socket.on('close', () => sockets.delete(socket)); });
|
||||
await new Promise(resolve => server.listen(0, '127.0.0.1', resolve));
|
||||
t.after(() => new Promise(resolve => { for (const socket of sockets) socket.destroy(); server.close(resolve); }));
|
||||
return { baseUrl: 'http://127.0.0.1:' + server.address().port + '/v1', requests };
|
||||
}
|
||||
@@ -10,6 +10,28 @@ import { Runtime } from '../src/server/runtime.mjs';
|
||||
import { verifyPlanEvidence } from '../src/core/context/request-plan.mjs';
|
||||
import { completionBody, requestMetrics } from '../src/shared/contracts/model-request.mjs';
|
||||
import { ConfigImporter } from '../src/core/models/config-import.mjs';
|
||||
import { verifyEngineModelInput } from '../src/shared/contracts/engine-execution.mjs';
|
||||
import { engineCanonicalDigest } from '../src/shared/contracts/engines.mjs';
|
||||
|
||||
test('saved engine budgets verify all Chat/Responses route variants and reject a mismatched output allowance', t => {
|
||||
const f = fixture(t), run = f.store.beginRun(f.session, 'Owned engine input proof', false, f.store.config);
|
||||
const owner = { sessionId: f.session.id, projectId: f.project.id, taskId: run.taskId, runId: run.id };
|
||||
const native = { messages: [], maxTokens: 512 };
|
||||
const verify = body => {
|
||||
const metrics = requestMetrics(body), request = { id: 'owned', bridgeTraceId: 'owned-trace', sourceSha256: engineCanonicalDigest(native),
|
||||
requestSha256: metrics.sha256, requestBytes: metrics.bytes, estimatedInputTokens: metrics.estimatedInputTokens,
|
||||
reservedPerAttempt: metrics.estimatedInputTokens + 128 + native.maxTokens, maxAttempts: 1 };
|
||||
const artifact = f.store.artifactStore.write(f.session, run, { owner, native, body, plan: request }, { kind: 'engine-model-input' });
|
||||
f.store.core.artifact.set(artifact.id, artifact); request.inputArtifactId = artifact.id; request.inputSha256 = artifact.sha256;
|
||||
return () => verifyEngineModelInput(f.store, f.session, owner, request);
|
||||
};
|
||||
const config = { model: 'owned', maxOutputTokens: 512 };
|
||||
for (const protocol of ['chat-completions', 'responses']) assert.doesNotThrow(verify(completionBody({ ...config, protocol }, [], [])));
|
||||
const mixed = completionBody({ ...config, modelRouting: { members: [{ id: 'chat', model: 'owned', protocol: 'chat-completions' },
|
||||
{ id: 'responses', model: 'owned', protocol: 'responses' }] } }, [], []);
|
||||
assert.doesNotThrow(verify(mixed)); mixed.variants[1].body.max_output_tokens++;
|
||||
assert.throws(verify(mixed), /预算核对失败/);
|
||||
});
|
||||
|
||||
test('owned CC Switch SQLite imports the selected TOML Responses wire_api and leaves unknown clients untouched', t => {
|
||||
const f = fixture(t), file = path.join(f.root, 'owned-switch.db'), db = new DatabaseSync(file);
|
||||
@@ -87,6 +109,23 @@ test('Responses incomplete, failed, missing call identity and unsupported hosted
|
||||
}
|
||||
});
|
||||
|
||||
test('invalid JSON in a nonstream Chat or Responses tool group prevents every tool and pool replay', async t => {
|
||||
for (const protocol of ['chat-completions', 'responses']) await t.test(protocol, async sub => {
|
||||
const f = fixture(sub), value = protocol === 'responses' ? response([fn('valid'), fn('broken', 'search_files', '{invalid')])
|
||||
: { choices: [{ message: { role: 'assistant', content: '', tool_calls: [
|
||||
{ id: 'valid', type: 'function', function: { name: 'list_files', arguments: '{"path":""}' } },
|
||||
{ id: 'broken', type: 'function', function: { name: 'search_files', arguments: '{invalid' } },
|
||||
] }, finish_reason: 'tool_calls' }] };
|
||||
const server = await compatibleFixture(sub, () => value), unused = await compatibleFixture(sub, () => { throw new Error('Must not replay'); });
|
||||
const provider = new Provider(), hub = provider.modelHub(f.store), first = connect(hub, server, protocol), second = connect(hub, unused, protocol);
|
||||
const pool = hub.pool({ name: 'Owned validation boundary', members: [{ connectionId: first.id }, { connectionId: second.id }] });
|
||||
hub.select({ type: 'pool', id: pool.id }); const runtime = new Runtime(f.store, provider); sub.after(() => runtime.stopAll());
|
||||
await runUntilIdle(runtime, f.session);
|
||||
assert.equal(server.requests.length, 1); assert.equal(unused.requests.length, 0); assert.equal(f.store.core.tool.size, 0);
|
||||
assert.notEqual(f.session.status, 'completed'); assert.equal(hub.view().attempts[0].status, 'failed');
|
||||
});
|
||||
});
|
||||
|
||||
test('wire adapter preserves tool IDs but never passes Responses-private Items to Chat or another endpoint', () => {
|
||||
const messages = [{ role: 'user', content: 'Inspect' }, { role: 'assistant', content: '', tool_calls: [{ id: 'call_own', type: 'function', function: { name: 'read_file', arguments: '{"path":"README.md"}' } }],
|
||||
providerState: { protocol: 'responses', endpoint: 'https://owned.invalid/v1', model: 'm', items: [{ type: 'reasoning', encrypted_content: 'owned-private' }] } },
|
||||
|
||||
@@ -73,7 +73,7 @@ test('the compatible HTTP loop preserves call IDs, validates errors and waits be
|
||||
round++;
|
||||
if (round === 1) return completion([
|
||||
call('list', 'list_files', { path: '', depth: 2 }),
|
||||
call('malformed', 'search_files', '{invalid'),
|
||||
call('malformed', 'search_files', { query: 42 }),
|
||||
call('unknown', 'unknown_tool', {}),
|
||||
]);
|
||||
if (round === 2) {
|
||||
|
||||
Reference in New Issue
Block a user