test: verify controlled Harness delivery across three real local projects

This commit is contained in:
Agent Workbench Dev
2026-10-31 05:16:08 +08:00
parent f0eecbda47
commit 213845e72e
11 changed files with 71166 additions and 29 deletions
+14 -18
View File
@@ -1,29 +1,25 @@
# 1.14 受控外部引擎验收
# 1.14 受控外部引擎验收记录
2026-10-31,接续60b63bc/正式1.13.0。本阶段按PLAN-1.14开发,当前尚未完成完整发行或正式升级;本地模型夹具、实际工具链、真实模型推理分别判定。
2026-10-31。本阶段在feat/controlled-engines-1.14开发;正式4318仍为1.13.0,原PID23172及Provider保持。本文件随切片更新,局部通过不等于发行完成。真实模型推理、Purge实际收益继续按用户既有选择独立待验。
## A:固定版本与协议
## A与B:固定协议和所属执行内核
受控适配器锁定已有Harness 0.2.1-rc.2、sdk-minimal、原SDK/CLI/Node及七份工具/代理/LLM/Include关键运行器。每个Run生成独立home、agents、叠加插件和JSON patch;禁用原生文件、工作目录工具、持久终端、原生模型与重试插件,实际appReady后再次核对完整工具schema、唯一代理provider、所属PID/nonce/SDK session/Run和工作目录。
A已由a01c418封存固定Harness0.2.1-rc.2、关键输入、受限叠加插件及原SDK协议。7/7专项和实际原生程序/本地固定模型2请求1工具通过;不支持的单会话取消、反向审批和安全原生续接如实保留。
原SDK不提供单会话取消或反向审批。受控桥通过独立本地HTTP返回工具/模型请求,原SDK仍使用initialize、session/prompt、session/wait与shutdown;收据必须在所属agent/inbox/spliced原事件中出现,输出来自同一活动的assistant/message,不能用idle或回复文字判工程交付。停止/恢复与真实工程的完整接入仍由后续切片验收。
B由f0eecbd封存原模型/工具桥、自有账本与所属引擎输入、逐请求预算、工具原结果反馈、结束前封存及全树关闭。原首轮17项15通过/2断言失败保留,相关兼容67/67、最新内核与HTTP25/25及已退出桥守卫7/7分别记录,不拼接不同轮统计。最后全量回归仍在F按最终固定输入独立判定。
首轮7/7专项通过:关键运行器变化、Include patch、完整目录与重复schema、进程/Run/SDK身份及Origin/认证、重复/越界/格式错误/来源变化、模型目录/输出预算和停止/超长正文守卫。原TAP为controlled-engine-core-first.tap。
## C:三个真实功能工程
实际Harness加本地固定模型夹具通过:完整启动目录核对、两次代理模型请求、一次原工具调用、prompt入队收据、所属原活动输出与正常退出;0真实模型请求、0stderr。原输入/协议/通知/桥数据与退出保存在controlled-engine-native-first.json及对应-log.txt。本轮只读取独立夹具README,没有开放用户工程或现有真实连接。
实际原生Harness和自有本地HTTP模型夹具从新b24613a9目录完成三个独立Git工程13节点;源码、脚本、测试、基准、构建全程冻结一致。每个Engine Run核对完整prompt收据、assistant原消息、turn/end(completed)、空stderr及所属进程树关闭;原模型/工具输入输出均保留,Task请求数/用量与实际HTTP一致。
## B:执行账本与权限接入
- Node:原8项测试实际初次失败,修复两个源模块后子/父8/8及ESM构建通过,父审查/Git只提交这两个模块。
- React/TypeScript:原9项测试和配置保持,修复4个源文件后子/父9/9、类型/构建通过。实际Edge1280/390各8项原断言覆盖筛选、裁剪搜索、单项切换及完整统计,所属浏览器结束;父Git只提交4个源文件。
- Python:固定独立3.10解释器,保留原2项及预先固定8项新增功能测试。错误实现实际失败后只修复两个模块,子/父10/10和编译通过,Git提交2个模块与2份固定新增测试。
原引擎工具进入所属Runtime的校验/审批/原结果账本,调用ID以引擎Run命名空间保存原sourceCallId。下一轮直接核对savedToolResult并封存真正反馈的工件;读取/提案/检查与工作台既有范围一致。引擎来源、所属叠加输入在接收工具、开始工具和批准后执行前复核,等待审批期间的变化不能使用旧许可。模型Key留在工作台Provider,原Native输入与实际兼容Chat请求分别保存。
父已有暂存、工作副本和未跟踪文件原字节/原索引保持;保护文件不变,失败/修复、子审批、父变更集、检查和Git均有原工件。模型响应为预先声明的本地夹具,不能据此报告真实模型自主完成率。
首轮17项15通过/2断言失败,原spec输出保持;按既有check_status/message契约修正。第二轮相关兼容67/67通过;随后最新内核25/25通过,包括等待审批来源变化、损坏报告拒绝交付、原模型预算恢复单次记账、未知核对和完整HTTP原记录分页/下载/归属。各轮分别保存,不拼接统计。界面代码和原生三工程尚待独立验收。
首轮350b1623使用测试连接默认16K上下文,在完整检查结果反馈后被预算守卫拒绝,整轮失败,原记录/原字节保留。新测试连接在开始前明确64K容量和累计200K预算;产品上下文与超限守卫未改变,每个Run须立即正常完成,不能用后续Run补过失败。原报告见controlled-engine-projects-first、first-node和validation及对应原日志。
关闭原SDK进程后的桥请求也拒绝进入模型/工具回调,追加7/7桥专项通过。原记录分别为controlled-engine-worker-first-spec-log.txt、controlled-engine-worker-compatible.tap、controlled-engine-worker-kernel.tap和controlled-engine-live-bridge.tap;来源字节和当前内核输入索引为controlled-engine-kernel-evidence.json。
## 未完成节点
本轮模型实际尝试由引擎桥记账,外层合成工具步骤不再重复记账。重启对原reservedPerAttempt/maxAttempts工件复核后保守记录未封存尝试,第二次恢复不重复计账;损坏预算不能减少累计用量。正常输出前关闭并封存引擎,worker成果仍需原报告、实际提案/检查和父审查。
已加入启动/运行中原创建时间和后代身份观察,关闭后复核完整所属树;活后代或观察失败保留unknown。原SDK会话不自动resume,人工核对独立保存review,原输出不改写为成功。下一步实际Harness+HTTP固定模型三工程及桌面/390px交付验证。
## 尚待验收
C:实际Harness独立工程/父审查;D:恢复/桌面/390px;E:Purge来源/兼容/同题对照;F:完整回归、独立发行及保留状态升级。正式4318继续使用不可变1.13.0。
D实际桌面/390px界面、停止/显式核对正在验收;E默认关闭的Purge来源/提示实验与同题对照、F最终版本/完整回归/独立发行/保留状态升级仍待完成。
+8 -10
View File
@@ -1,15 +1,13 @@
# 1.14 阶段回顾
# 1.14 节点回顾
2026-10-31。每个切片按计划、实现、原证据、限制和本地Git回顾;此文随实际结果更新,阶段尚未完整交付。
2026-10-31。每切片记录实际能力、失败及下一前置;正式发行按QA-1.14最后状态判定。
| 切片 | 当前实现/验证 | 遗留与下一步 |
| --- | --- | --- |
| A:版本/协议 | 新受控安装身份、七份关键运行器、独立profile叠加/桥;7/7边界专项和实际Harness固定模型2请求/1工具/收据/wait/正常退出通过 | 原SDK没有单会话取消或反向审批;桥不能单凭启动成功宣称工程交付 |
| B:工具桥 | 原目录/身份桥接自有Runtime/账本,命名空间保留sourceCallId,反馈核对真实结果;等待审批的来源复核、单次尝试计账、报告损坏/预算恢复/HTTP分页已通过最新25项;原67项兼容保持 | 实际Harness三工程及最终冻结发行单独验收;已关闭原进程的桥请求追加拒绝 |
| C–F | 尚未完成 | 独立工程/停止恢复/父审查、真实界面、Purge同题对照、完整发行与状态保持升级 |
## C:真实工程与原生协议共同验收
前置研究确认sdk-minimal仍含danger-full-access策略和本机工具,不能把profile名字当权限边界;采用宿主受限桥和实际目录检查。Harness现有DeepSeek provider使用Messages API,本工作台现有连接是兼容Chat层;自有代理provider保持两层分离,使引擎不取得模型Key,也不把一种协议冒充另一种协议。
原生idle/最终文字和父测试通过均不能单独表示本次引擎成功。首轮默认16K模型预算拒绝之后,后续新Run可以观察已完成工具,但不能拿最后检查补称原失败Run正常完成;验收器现在逐Run立即核对完整结束,整轮门槛保持。
查询期间过宽文件枚举、几处推测的不存在路径、一次PowerShell foreach直连管道解析失败已保存计划/发现并修正为已确认文件、明确变量结果;这些查询不当作功能失败或验收结果。首轮实际协议与本地专项成功原记录保留,后续实现变化采用新的结果目录。
新明确64K容量的自有HTTP夹具完成三工程13节点、父8/8、9/9、10/10、实际构建/编译/Git及两视口8项断言。工具完整原结果/调用身份、每次实际模型请求及累计用量、正常进程树关闭均参与判定;原失败日志/输入/现场保存。仍未证明真实模型语义成功率。
正式服务/原模型连接保持,真实模型推理和Purge收益仍按用户既有决定暂缓。每个已完成切片检查diff后本地提交,不把计划或夹具成功登记为整个产品完成。
一次旧package脚本将--help误作普通打包,已在复制阶段终止,自有临时目录保留且不采用。没有改变本轮冻结输入或正式进程;F需增加参数校验,帮助查询不得产生文件/进程副作用。
下一节点继续实际控制台、原报告分页下载、停止/未知核对和390px操作。Purge完整插件含自愈/宿主修改入口,先实现独立提示装配实验及来源锁定,保留宿主审查权限,未经重复收益不默认启用。
@@ -0,0 +1,2 @@
PASS node: pinned native engine and one owned Git worktree preserve parent staging and protected bytes
{"success":false,"cases":1,"directory":"D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\controlled-projects-350b1623-f39b-4820-8a0e-ca89aaa7aec5","realModelRequests":0,"error":"AssertionError [ERR_ASSERTION]: 原引擎完整模型输入或累计尝试超出封存预算,未发送模型请求。\n+ actual - expected\n\n+ 'failed'\n- 'completed'\n\n at runSample (file:///D:/Ai-project/REA%E9%80%86%E5%90%91/Agent-Workbench/scripts/verify-controlled-projects.mjs:141:61)\n at async file:///D:/Ai-project/REA%E9%80%86%E5%90%91/Agent-Workbench/scripts/verify-controlled-projects.mjs:190:64"}
File diff suppressed because one or more lines are too long
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,14 @@
PASS node: pinned native engine and one owned Git worktree preserve parent staging and protected bytes
PASS node: actual Harness uses owned model/tool bridges, retains failing tests, and completes reviewed repairs with one accounting entry per HTTP request
PASS node: separately reviewed parent merge passes all 8 functional tests and actual build/compile
PASS node: local Git contains only reviewed functional changes and preserves original staged/unstaged owner bytes
PASS react: pinned native engine and one owned Git worktree preserve parent staging and protected bytes
PASS react: actual Harness uses owned model/tool bridges, retains failing tests, and completes reviewed repairs with one accounting entry per HTTP request
PASS react: separately reviewed parent merge passes all 9 functional tests and actual build/compile
PASS react: real Edge desktop/390px filtering, trimmed search, one-task toggle and complete totals pass
PASS react: local Git contains only reviewed functional changes and preserves original staged/unstaged owner bytes
PASS python: pinned native engine and one owned Git worktree preserve parent staging and protected bytes
PASS python: actual Harness uses owned model/tool bridges, retains failing tests, and completes reviewed repairs with one accounting entry per HTTP request
PASS python: separately reviewed parent merge passes all 10 functional tests and actual build/compile
PASS python: local Git contains only reviewed functional changes and preserves original staged/unstaged owner bytes
{"success":true,"cases":13,"directory":"D:\\Ai-project\\REA逆向\\Agent-Workbench\\.qa\\controlled-projects-b24613a9-9043-452d-84da-f32a2a472228","realModelRequests":0,"error":null}
File diff suppressed because one or more lines are too long
+4
View File
@@ -2,6 +2,10 @@
## 2026-10-31 1.14 受控引擎前置核对
- E源码入口已区分:独立upstream为dsh-purge1.1.78,Desktop/plugins为1.1.81,两者dshTarget均0.2.1-rc.2、Apache-2.0/YuJunZhiXue。原index.apply即使autoApply关闭仍有默认种子、profile自愈、runtime env和工具注册;原完整plugin不能直接作为受限SDK叠加加载。1.1.81新增apply-progress/profile-insert等调用,必须分别锁定来源。可独立采纳的是明确来源与提示装配/同题对照机制;默认关闭实验策略,保留现有宿主权限、原提示与完整账本,不开放原purge_apply、hooksDenyBypass、自动续跑或宿主补丁。
- 实际Harness三工程首轮350b1623说明工作台原检查结果完整反馈会增加SDK上下文;本地默认16K模型在新Run追加plan_checks/run_checks结果后超过容量,原预算守卫未发出拒绝请求,进程树仍正确关闭。全验收要求每个原生Run正常完成,故保留失败、不只按最后父检查通过判定成功。自有本地工程/界面夹具明确配置64K容量,产品默认与超限守卫不改变;真实模型容量及推理效果继续另验。
- B进一步审查发现仅在桥收到工具时核对来源,无法覆盖用户等待检查审批期间的变化。现增加Runtime开始工具和批准后执行之前的所属引擎/叠加输入复核;变化旧许可不能启动检查。实际模型事件与引擎轮次分开,原模型请求/预算工件参与恢复计账与成果核对。
- B草稿审查确认外层Runtime与引擎model回调会双重记账,worker.output又早于引擎关闭封存;需按实际请求单次计账并先finish再成果校验。现有Store.recordResponse只改变内存,必须与每次attempt标记同次save保持恢复一致。Owned process tree复用src/core/check-runner.mjs,不能只检查根进程已退出。
- 恢复查询的一次不存在checks/check-runner路径和一次无效findings标题补丁均未写入产品;已按实际import/标题修正。公开源码分析沿用普通文件工具,不重新跑A原验证。
+5
View File
@@ -2,6 +2,11 @@
## 2026-10-31 1.14 受控外部引擎阶段
- C第二轮b24613a9完整三工程13节点通过,原生完整协议/全树关闭/空stderr与实际HTTP请求逐次记账核对;父Node8/8、React9/9、Python10/10、实际构建/编译/Git以及React两视口各8项原断言通过,原owner暂存/工作副本/未跟踪字节与保护文件保持,全部冻结输入一致。首轮原失败与Node完整现场分别复制docs,真实模型0请求。
- 一次尝试查询旧打包脚本--help意外进入复制阶段,因为该脚本没有参数校验;自有会话74297立即终止,两份未验收临时1.13目录保留,不作发行。没有重启正式PID23172或改写受测src/scripts/tests/dist输入,C最终冻结比较仍通过;后续F为打包等旧入口增加help/未知参数拒绝,避免误启动。
- 接续f0eecbd:两个新增验证脚本的语法/help及最新TS/构建通过,原构建日志controlled-build-current-log.txt保存。实际Harness三工程首轮350b1623在Node检查原结果反馈后触发16K上下文守卫,整轮失败,原日志/原会话/完整报告保留;测试连接显式配置64K容量后重新验收,累计200K预算及全部原协议/进程树/工程检查门槛保持。每轮原生运行立即核对正常结束,不能靠后续Run绕过本轮失败。正式服务仍未变更。只读查询误用scripts/lib和三处推测路径,已按rg枚举的实际路径修正。
- 新相关整轮67/67通过、0失败/取消/跳过,原TAP controlled-worker-second.tap保存;含原引擎探针和worker/停止/恢复/合并/HTTP兼容。该轮期间开发了界面及追加「等待审批期间来源变化」补强,最新补强会单独验收,F按最终冻结输入另验。实际Harness三工程HTTP夹具验收脚本已准备,尚未运行;源文件/TS静态检查通过,正式1.13保持。
- B/C首轮17项15通过/2失败,原输出controlled-worker-first.tap保留(Node默认spec格式)。两个失败来自专项断言错误:run_checks实际返回completed与check_status:passed,错误文本字段是message;已读实际源码修正,没有改低产品完成门槛。引擎计划、真实工具反馈/单次记账、停止迟到输出、来源变化及所属树专项已实现;后续用显式TAP新轮验证。池既有上限8次,原模型输入验证同步尊重该封存上限。
- 接续a01c418:A原证据沿用;B草稿尚未测试且未接Runtime/Store/WorkerManager。先完成实际模型请求单次记账、引擎封存与worker成果的顺序、所属进程树和恢复未知状态,再做独立工程/UI/Purge/发行。规划文件保持当前状态,正式4318不重启。恢复时一次将.qa纳入文件枚举产生过宽输出,改为限定src/tests/scripts及具体证据文件。
+196
View File
@@ -0,0 +1,196 @@
import assert from 'node:assert/strict';
import fs from 'node:fs';
import path from 'node:path';
import { randomUUID, createHash } from 'node:crypto';
import { spawnSync } from 'node:child_process';
import { fixture, compatibleFixture, call, assertToolPairs } from '../tests/helpers.mjs';
import { Provider } from '../src/server/provider.mjs';
import { Runtime } from '../src/server/runtime.mjs';
import { EngineRunner } from '../src/core/engines/engine-runner.mjs';
import { resolveControlledInstallation } from '../src/core/engines/controlled-installation.mjs';
import { originalEngineRun } from '../src/shared/contracts/engine-execution.mjs';
import { deliveryGit as git, writeDeliveryFile as write } from '../tests/fixtures/delivery-project.mjs';
import { ownerBytes } from '../tests/fixtures/workers-project.mjs';
if (process.argv.includes('--help')) {
console.log('node scripts/verify-controlled-projects.mjs — actual pinned Harness, owned HTTP model fixture, three real Node/React/Python worktrees, failing tests/repairs, approvals, parent validation, Edge and Git; no configured-model requests. Requires AGENT_WORKBENCH_DEEPSEEK_ROOT and AGENT_WORKBENCH_TEST_PYTHON.'); process.exit(0);
}
assert.equal(process.argv.length, 2);
const version = JSON.parse(fs.readFileSync('package.json', 'utf8')).version;
const directory = path.resolve('.qa', 'controlled-projects-' + randomUUID()), results = [], cases = [];
const hash = bytes => createHash('sha256').update(bytes).digest('hex');
const installation = resolveControlledInstallation(process.cwd(), process.env.AGENT_WORKBENCH_DEEPSEEK_ROOT);
assert.equal(installation.controlledStatus, 'available', installation.message);
const python = process.env.AGENT_WORKBENCH_TEST_PYTHON;
assert.ok(python && path.isAbsolute(python) && fs.existsSync(python), 'Provide the existing isolated Python 3.10+ interpreter.');
fs.mkdirSync(directory, { recursive: true });
const npmCli = path.join(path.dirname(process.execPath), 'node_modules/npm/bin/npm-cli.js');
function inputs() {
const files = ['package.json', 'package-lock.json'];
const walk = root => { for (const entry of fs.readdirSync(root, { withFileTypes: true })) {
const file = path.join(root, entry.name); if (entry.isDirectory()) walk(file); else if (entry.isFile()) files.push(file);
} };
for (const root of ['src', 'scripts', 'tests', 'benchmarks', 'dist-client', 'dist-server']) walk(root);
return files.sort().map(file => ({ path: file, sha256: hash(fs.readFileSync(file)) }));
}
const frozen = inputs(), mark = message => { cases.push(message); console.log('PASS ' + message); };
function specification(kind, root) {
if (kind === 'node') return { tests: 8, checks: ['test', 'build'], initial: null, fixes: {
'src/add.mjs': 'export function add(left, right) {\n if (!Number.isFinite(left) || !Number.isFinite(right)) throw new TypeError("finite numeric inputs required");\n return left + right;\n}\n',
'src/range.mjs': 'export function inclusiveRange(start, end) {\n if (!Number.isSafeInteger(start) || !Number.isSafeInteger(end) || end < start || end - start >= 10000) throw new RangeError("bounded ascending integer range required");\n return Array.from({ length: end - start + 1 }, (_, index) => start + index);\n}\n',
} };
if (kind === 'react') {
const app = fs.readFileSync(path.join(root, 'src/App.tsx'), 'utf8'); assert.ok(app.includes("toggleTask(current, 'missing')"));
return { tests: 9, checks: ['test', 'typecheck', 'build'], initial: null, fixes: {
'src/query.ts': "import type { Task } from './types.ts';\nexport function selectTasks(tasks: Task[], filter: string, query: string): Task[] {\n const text = query.trim().toLocaleLowerCase();\n return tasks.filter(task => (filter === 'done' ? task.done : filter === 'pending' ? !task.done : true) && task.title.toLocaleLowerCase().includes(text));\n}\n",
'src/summary.ts': "import type { Task } from './types.ts';\nexport function summarizeTasks(tasks: Task[]) {\n return { total: tasks.length, completed: tasks.filter(task => task.done).length, pendingHours: tasks.filter(task => !task.done).reduce((sum, task) => sum + task.hours, 0) };\n}\n",
'src/transition.ts': "import type { Task } from './types.ts';\nexport function toggleTask(tasks: Task[], id: string): Task[] {\n return tasks.map(task => task.id === id ? { ...task, done: !task.done } : task);\n}\n",
'src/App.tsx': app.replace("toggleTask(current, 'missing')", 'toggleTask(current, task.id)'),
} };
}
const test = file => fs.readFileSync(path.join('benchmarks/workers-delivery-v1/python-tests', file), 'utf8');
return { tests: 10, checks: ['python:test', 'python:compile'], initial: {
'money.py': 'def to_cents(value):\n return int(float(value) * 100)\n',
'schedule.py': 'def normalize_schedule(values):\n return list(values)\n',
'tests/test_money.py': test('test_money.py'), 'tests/test_schedule.py': test('test_schedule.py'),
}, fixes: {
'money.py': 'from decimal import Decimal, InvalidOperation, ROUND_HALF_UP\n\ndef to_cents(value):\n try:\n amount = Decimal(str(value))\n if not amount.is_finite():\n raise ValueError("finite amount required")\n return int((amount * 100).quantize(Decimal("1"), rounding=ROUND_HALF_UP))\n except (InvalidOperation, OverflowError, TypeError) as error:\n raise ValueError("valid finite amount required") from error\n',
'schedule.py': 'import re\nfrom datetime import date\n\ndef normalize_schedule(values):\n dates = []\n for value in values:\n if not isinstance(value, str) or not re.fullmatch(r"\\d{4}-\\d{2}-\\d{2}", value):\n raise ValueError("ISO calendar date required")\n dates.append(date.fromisoformat(value))\n return [value.isoformat() for value in sorted(set(dates))]\n',
} };
}
async function runSample(kind) {
const disposers = [], t = { after: value => disposers.push(value) }, f = fixture(t), sample = path.join(directory, kind);
fs.mkdirSync(sample); let runtime, group, result; const reviews = [], modelRequests = [];
try {
const baseline = kind === 'react' ? 'benchmarks/react-delivery-v1/baseline' : 'benchmarks/workers-delivery-v1/' + kind;
fs.cpSync(path.resolve(baseline), f.projectRoot, { recursive: true, filter: file => !file.split(path.sep).includes('node_modules') });
write(f, '.gitignore', 'node_modules/\ndist/\n.awb-*\n__pycache__/\n.venv/\n.pytest_cache/\n');
write(f, 'owner-staged.txt', 'Owner baseline\n'); write(f, 'owner-unstaged.txt', 'Owner baseline\n');
git(f, ['init', '--initial-branch=main']); git(f, ['config', 'core.autocrlf', 'false']); git(f, ['config', 'user.name', 'Workbench QA']); git(f, ['config', 'user.email', 'qa@workbench.invalid']);
git(f, ['add', '--', '.']); git(f, ['commit', '--no-gpg-sign', '-m', 'Seal ' + kind + ' controlled-engine acceptance']);
write(f, 'owner-staged.txt', 'Owner staged change\n'); git(f, ['add', '--', 'owner-staged.txt']);
write(f, 'owner-staged.txt', 'Owner working copy after staging\n'); write(f, 'owner-unstaged.txt', 'Owner unstaged change\n'); write(f, 'owner-untracked.txt', 'Owner untracked file\n');
const spec = specification(kind, f.projectRoot), writable = [...new Set([...Object.keys(spec.initial ?? {}), ...Object.keys(spec.fixes)])].sort();
const original = ownerBytes(f), protectedNames = git(f, ['ls-files', '-z']).split('\0').filter(file => file && !writable.includes(file) && !file.startsWith('owner-'));
const protectedFiles = Object.fromEntries(protectedNames.map(file => [file, fs.readFileSync(path.join(f.projectRoot, file))]));
const verify = root => { for (const [file, bytes] of Object.entries(protectedFiles)) assert.deepEqual(fs.readFileSync(path.join(root, file)), bytes, 'Protected ' + file); };
const install = (root, label) => {
const child = spawnSync(process.execPath, [npmCli, 'ci', '--offline', '--no-audit', '--no-fund'], { cwd: root, windowsHide: true, timeout: 120000, maxBuffer: 8 * 1024 * 1024 });
fs.writeFileSync(path.join(sample, label + '-npm-ci-log.txt'), child.stdout ?? Buffer.alloc(0)); fs.writeFileSync(path.join(sample, label + '-npm-ci-stderr-log.txt'), child.stderr ?? Buffer.alloc(0));
if (child.error) throw child.error; assert.equal(child.status, 0, 'Inspect the original offline installation logs.'); verify(root);
};
if (kind === 'react') install(f.projectRoot, 'parent');
let phase = 'new';
const model = await compatibleFixture(t, async (route, input) => {
assert.equal(route, '/v1/chat/completions'); assert.equal(input.model, 'owned-controlled-engineering');
assertToolPairs(input.messages);
const worker = [...f.store.core.worker.values()][0], session = f.store.session(worker.sessionId);
const checks = [...f.store.core.check_job.values()].filter(job => job.taskId === worker.taskId), job = checks.at(-1);
const plan = [...f.store.core.check_plan.values()].filter(plan => plan.taskId === worker.taskId).at(-1);
const propose = files => Object.entries(files).map(([file, content]) => call(randomUUID(), 'propose_file', { path: file, content, reason: 'Fixed local acceptance repair; retain all tests and declarations.' }));
let calls = [];
if (session.changes.some(change => change.status === 'pending')) { /* Explicit review follows native completion. */ }
else if (phase === 'new' && spec.initial) { calls = propose(spec.initial); phase = 'initial'; }
else if (phase !== 'fixed' && job) {
assert.equal(job.status, 'failed', job.summary); assert.ok(job.results.some(result => result.tests?.failed > 0)); calls = propose(spec.fixes); phase = 'fixed';
} else if (phase === 'fixed' && job?.status === 'passed') assert.equal(job.results[0].tests.passed, spec.tests);
else if (!plan || job?.planId === plan.id) calls = [call(randomUUID(), 'plan_checks', { names: spec.checks })];
else calls = [call(randomUUID(), 'run_checks', { plan_id: plan.id })];
modelRequests.push({ requestBody: input, callIds: calls.map(call => call.id), phase });
return { choices: [{ message: { role: 'assistant', content: calls.length ? '' : 'Owned native run complete; host proof is still required.', ...(calls.length ? { tool_calls: calls } : {}) },
finish_reason: calls.length ? 'tool_calls' : 'stop' }], usage: { prompt_tokens: 123, completion_tokens: 45, total_tokens: 168 } };
});
f.store.saveConfig({ mode: 'compatible', model: 'owned-controlled-engineering', baseUrl: model.baseUrl, contextLimit: 64000 });
const provider = new Provider(); provider.setKey('self-owned-controlled-model-fixture'); runtime = new Runtime(f.store, provider); f.onCleanup(() => runtime.stopAll());
runtime.controlledEngines = new EngineRunner(f.store, provider, { directoryRoot: path.join(f.root, 'owned-engine-runs') });
runtime.checkRunner().pythonExecutable = python; const manager = runtime.workerManager();
group = await manager.plan(f.session, { goal: 'Deliver ' + kind + ' with the pinned controlled Harness, retained failures, explicit review, parent validation and local Git.',
engine: { id: 'deepseek-harness', root: installation.root, controlledSha256: installation.controlledSha256 },
budget: { maxTokens: 200000, maxSteps: 64, maxToolCalls: 128, maxDurationMs: 600000 },
children: [{ label: 'engine-' + kind, goal: 'Repair and validate the complete ' + kind + ' functional suite without replacing tests.', paths: writable }] });
await (await manager.prepare(f.session, group.id, group.fingerprint)).promise; assert.equal(group.status, 'ready', group.summary);
const child = group.children[0], childRoot = f.store.project(child.projectId).root;
assert.notEqual(childRoot, f.projectRoot); assert.deepEqual(ownerBytes(f), original); verify(childRoot); if (kind === 'react') install(childRoot, 'child');
mark(kind + ': pinned native engine and one owned Git worktree preserve parent staging and protected bytes');
for (let turn = 0; turn < 4 && group.status !== 'ready_to_collect'; turn++) {
const session = f.store.session(child.sessionId), ids = session.changes.filter(change => change.status === 'pending').map(change => change.id);
if (ids.length) {
assert.equal(group.status, 'awaiting_review', group.summary); const set = await runtime.changeSetManager().prepare(session, ids);
reviews.push({ operation: 'review child batch', id: set.id, fingerprint: set.fingerprint, paths: set.files.map(file => file.path) });
await (await runtime.changeSetManager().start(session, set.id, set.fingerprint)).promise; assert.equal(set.status, 'completed', set.summary);
}
const started = manager.start(f.session, group.id, group.fingerprint); let done = false, failure;
started.promise.then(() => { done = true; }, error => { failure = error; done = true; });
while (!done) {
for (const approval of session.approvals.filter(approval => approval.status === 'pending')) {
assert.equal(f.store.core.tool.get(approval.invocationId).tool, 'run_checks');
const plan = [...f.store.core.check_plan.values()].filter(plan => plan.taskId === child.taskId).at(-1);
assert.deepEqual(plan.checks.map(check => check.name), spec.checks);
reviews.push({ operation: 'approve original check', id: approval.id, taskId: child.taskId, planId: plan.id, contractSha256: plan.contractSha256 }); runtime.decide(session.id, approval.id, true);
}
await new Promise(resolve => setTimeout(resolve, 30));
}
if (failure) throw failure;
const native = [...f.store.core.engine_run.values()].filter(job => job.taskId === child.taskId).at(-1);
assert.equal(native.status, 'completed', native.error);
}
assert.equal(group.status, 'ready_to_collect', group.summary); assert.deepEqual(ownerBytes(f), original); verify(childRoot);
const childChecks = [...f.store.core.check_job.values()].filter(job => job.taskId === child.taskId);
assert.ok(childChecks.some(job => job.status === 'failed' && job.results.some(result => result.tests?.failed > 0)));
assert.equal(childChecks.at(-1).results[0].tests.passed, spec.tests); assert.equal(childChecks.at(-1).status, 'passed');
const engineRuns = [...f.store.core.engine_run.values()].filter(job => job.taskId === child.taskId), childSession = f.store.session(child.sessionId);
for (const job of engineRuns) {
originalEngineRun(f.store, childSession, job); assert.equal(job.status, 'completed', job.error); assert.equal(job.native.cleanup.treeConfirmed, true);
assert.equal(job.native.stderr, ''); assert.equal(job.native.activity.endEvents[0].data.reason.kind, 'completed');
}
assert.equal(f.store.core.task.get(child.taskId).usage.requests, modelRequests.length);
assert.equal(f.store.core.task.get(child.taskId).usage.totalTokens, modelRequests.length * 168);
mark(kind + ': actual Harness uses owned model/tool bridges, retains failing tests, and completes reviewed repairs with one accounting entry per HTTP request');
await (await manager.collect(f.session, group.id, group.fingerprint)).promise; assert.equal(group.status, 'collected', group.summary);
const collection = await manager.readCollection(f.session, group); assert.deepEqual(collection.conflicts, []); assert.deepEqual(collection.paths, writable);
await (await manager.prepareMerge(f.session, group.id, group.fingerprint)).promise; assert.equal(group.status, 'merge_prepared', group.summary);
const merge = f.store.core.change_set.get(group.merge.changeSetId); reviews.push({ operation: 'review parent batch', id: merge.id, fingerprint: merge.fingerprint });
await (await runtime.changeSetManager().start(f.session, merge.id, merge.fingerprint)).promise; assert.equal(merge.status, 'completed', merge.summary);
const checkPlan = await runtime.checkRunner().plan(f.session, spec.checks), check = await (await runtime.checkRunner().start(f.session, checkPlan.id)).promise;
assert.equal(check.status, 'passed', check.summary); assert.equal(check.results[0].tests.passed, spec.tests); verify(f.projectRoot);
mark(kind + ': separately reviewed parent merge passes all ' + spec.tests + ' functional tests and actual build/compile');
const browserJobs = [];
if (kind === 'react') for (const viewport of [{ width: 1280, height: 900 }, { width: 390, height: 844 }]) {
const steps = [ { action: 'assert_count', selector: '.task-list article', count: 3 },
{ action: 'assert_text', selector: '.summary > div:nth-child(3) strong', text: viewport.width > 600 ? '8小时' : '8\n小时' },
{ action: 'select', selector: 'select[aria-label="任务状态"]', value: 'pending' }, { action: 'assert_count', selector: '.task-list article', count: 2 },
{ action: 'fill', selector: 'input[aria-label="任务搜索"]', value: ' 编写 ' }, { action: 'assert_count', selector: '.task-list article', count: 1 },
{ action: 'fill', selector: 'input[aria-label="任务搜索"]', value: '' }, { action: 'select', selector: 'select[aria-label="任务状态"]', value: 'all' },
{ action: 'click', selector: '.task-list article[data-task="api"] button' }, { action: 'assert_text', selector: '.task-list article[data-task="api"] .tag', text: '已完成' },
{ action: 'assert_text', selector: '.task-list article[data-task="tests"] .tag', text: '待完成' }, { action: 'assert_text', selector: '.summary > div:nth-child(2) strong', text: '2' },
{ action: 'assert_text', selector: '.summary > div:nth-child(3) strong', text: viewport.width > 600 ? '5小时' : '5\n小时' } ];
const plan = await runtime.browserManager().plan(f.session, { viewport, steps }), job = await runtime.browserManager().start(f.session, plan.id).promise;
assert.equal(job.status, 'passed', job.summary); assert.equal(job.cleanup.exited, true); browserJobs.push(job);
for (const image of job.screenshots) fs.writeFileSync(path.join(sample, 'page-' + viewport.width + '-' + image.artifactId + '.png'), f.store.artifactStore.readBytes(f.store.core.artifact.get(image.artifactId)));
}
if (kind === 'react') mark('react: real Edge desktop/390px filtering, trimmed search, one-task toggle and complete totals pass');
const deliveryPlan = await runtime.gitDelivery().plan(f.session, { message: 'Reviewed ' + kind + ' controlled-engine delivery' }), delivery = await (await runtime.gitDelivery().start(f.session, deliveryPlan.id)).promise;
assert.equal(delivery.status, 'completed', delivery.summary); await manager.finalize(f.session, group.id, group.fingerprint); assert.equal(group.status, 'completed');
assert.deepEqual(ownerBytes(f).files, original.files); assert.equal(git(f, ['show', ':owner-staged.txt']), 'Owner staged change\n');
assert.deepEqual(git(f, ['show', '--format=', '--name-only', delivery.commit]).trim().split(/\r?\n/).sort(), writable);
mark(kind + ': local Git contains only reviewed functional changes and preserves original staged/unstaged owner bytes');
result = { success: true, scope: 'actual-native-Harness-with-owned-HTTP-model-fixture', version, kind, parentTaskId: group.taskId, groupId: group.id,
child: f.store.core.worker.get(child.id), engineRuns, checks: [...f.store.core.check_job.values()], parentCheck: check, browserJobs, commit: delivery.commit,
writable, reviews, modelRequests, protectedFiles: Object.fromEntries(Object.entries(protectedFiles).map(([file, bytes]) => [file, hash(bytes)])), realModelRequests: 0 };
return result;
} catch (error) { result = { success: false, version, kind, error: error.stack, group, reviews, modelRequests, engineRuns: [...f.store.core.engine_run.values()], checks: [...f.store.core.check_job.values()], realModelRequests: 0 }; throw error; }
finally {
await runtime?.stopAll(); f.store.storage.database.exec('PRAGMA wal_checkpoint(TRUNCATE)');
fs.cpSync(f.root, path.join(sample, 'original-fixture'), { recursive: true, filter: file => !file.split(path.sep).includes('node_modules') });
for (const session of f.store.sessions.values()) fs.writeFileSync(path.join(sample, 'session-' + session.id + '.json'), JSON.stringify({ project: f.store.project(session.projectId), session, execution: f.store.executionSnapshot(session) }, null, 2));
fs.writeFileSync(path.join(sample, 'result.json'), JSON.stringify(result, null, 2));
for (const dispose of disposers.reverse()) await dispose();
}
}
let report;
try {
for (const kind of ['node', 'react', 'python']) results.push(await runSample(kind)); assert.deepEqual(inputs(), frozen);
report = { success: true, targetVersion: '1.14.0', testedVersion: version, directory, installation, cases, results, inputs: frozen, realModelRequests: 0,
scope: 'Pinned actual Harness and workbench execution with an owned HTTP model fixture. Checks, approvals, failures, parent merges, Edge and Git are real. Model reasoning and Purge benefit remain separate.' };
} catch (error) { report = { success: false, targetVersion: '1.14.0', testedVersion: version, directory, installation, cases, results, inputs: frozen, error: error.stack, realModelRequests: 0 }; process.exitCode = 1; }
finally { fs.writeFileSync(path.join(directory, 'result.json'), JSON.stringify(report, null, 2)); fs.writeFileSync('.qa/controlled-projects-location.json', JSON.stringify({ directory, result: path.join(directory, 'result.json') })); console.log(JSON.stringify({ success: report.success, cases: cases.length, directory, realModelRequests: 0, error: report.error ?? null })); }
+1 -1
View File
@@ -6,7 +6,7 @@
1. [complete] A:新受控安装身份固定原SDK/CLI/Node及七份关键运行器,独立profile叠加禁用原生项目/终端/模型/重试插件。7/7边界专项和实际Harness+本地固定模型2请求/1工具、prompt收据/原活动/wait/正常退出通过;完整原输入和日志封存。原SDK缺失单会话取消/反向审批/安全恢复如实保留,尚不计工程交付。
2. [complete] B:原工具/模型桥接自有Runtime/Store/WorkerManager;原调用命名空间与savedToolResult工件反馈,校验/等待审批后来源核对、单次实际尝试记账、原输入/报告/HTTP归属。67项相关兼容及最新25项内核通过;原首轮15/17断言失败保留,关闭原进程的桥请求追加复核。
3. [in_progress] C:已接独立worker工程、累计预算、原输出封存、原进程树与未知恢复;继续实际Harness+本地HTTP模型夹具和真实Node/React/Python功能工程/父审查/Git验收。
3. [complete] C:实际Harness+自有本地HTTP模型夹具完成Node/React/Python三工程13节点,父8/8、9/9、10/10与构建/编译/Git通过,React1280/390各8项原断言通过;每个原生Run正常结束、全树关闭、完整原结果/逐请求用量核对。默认16K夹具首轮失败保留,明确64K容量后新冻结轮通过,产品守卫不改动。
4. [in_progress] D:执行方式/实际安装检测、单引擎工程和分页原运行面板已实现,TS/构建通过;继续真实桌面/390px、停止/核对与窄屏标题栏验收。
5. [pending] E:分别核对Purge 1.1.78/1.1.81来源、兼容性、启停和生效点,建立同题/同预算对照与指标;本地夹具与真实模型收益分开记录,未经重复验证不默认启用。
6. [pending] F:按最终固定代码/基准完成相关与完整回归、真实工程/UI、独立源码/便携、发行与保留状态升级;各切片保存原失败、回顾和本地commit。