docs(qa): retain Claude identity typo and DeepSeek reasoning cap failure

This commit is contained in:
Agent Workbench Dev
2026-11-04 23:34:08 +08:00
parent 409d30f6e2
commit 324ec46623
8 changed files with 204 additions and 1 deletions
+2
View File
@@ -1,5 +1,7 @@
# Agent Workbench项目执行规范
Z两路线已封存:apikey-Claude首预检1请求176tokens,手误claude-opus-5.5对上游claude-opus-5-5,mismatch停止0工具;用户明确已自行改为claude-opus-5-5,当前连接修订2/凭据修订2,保留新配置,另冻Z2,不续跑原Z。DeepSeek预检1请求146tokens通过,新Node5请求47740tokens,末次SSE有DONE但finish_reason=length、2048输出全reasoning,原protocol失败/identity incomplete,不执行工具、不续跑。原九基线失败/build通过、0提案/交付;配置/预算/池/原状态恢复通过,连接1→2→3、凭据1保持。两样本共7请求48062tokens、151份82988849字节证据离线通过,docs/QA-LIVE-PROVIDER-Z.md。当前72项目88会话6697记录14254事件/outbox、3912工件,原PID36552/Provider/9229关闭保持。DeepSeek官方证实默认high推理,Chat thinking开关当前白名单未支持,兼容性缺口另列,不能宣称已修或以网络不稳定代替原因。下一用户更正Claude的新单次预检/原题有界Node。
Y已结束未闭环:正式1.25.2/apikey GPT SSE,预检1请求110tokens;Node8请求64500tokens全部matched、原九基线失败/build通过、三源码模型提案封存并监督审查应用,最后inspect_session后step_limit,未做修改后检查/Git,不续跑。原收尾在恢复配置后因3条外部诊断追加失败;独立只读逐行/全原导出/工件/配置对账通过,开场快照与驱动配置冻结之间外部加DeepSeek入池(6→7)及两次复核错误完整保留,不回滚外部变更。69项目85会话6607记录13944事件/outbox、3857工件,原PID36552/Provider、凭据1、原身份来源保持,9229关闭。见docs/QA-LIVE-DELIVERY-Y.md。用户最新要求尝试当前apikey-Claude和DeepSeek;另冻每线路单次严格身份预检,通过才新Node,旧Y/X/W等不续跑,预算和原失败不改。
最新真实实验选择:X在预冻结时发现原jojocode被外部停用(revision12),在创建样本目录/诊断/模型之前拒绝,0请求。用户明确改用当前启用的apikey并“另冻新实验,不沿用X”,执行docs/LIVE-DELIVERY-Y-PLAN.md及scripts/live-y。连接93d0cf6d/Chat SSE/gpt-6.1-sol/凭据修订1,保留原身份来源https://api.apikey.fan;目标/8请求90000tokens/16K2048保持,不启用jojocode、不续跑X或旧样本。下面“下一X”按历史计划理解。
+26
View File
@@ -0,0 +1,26 @@
# Z:apikey-Claude 与 DeepSeek 的独立验证
两条路线都收到真实上游响应,均未证明Node自主交付。Claude停在配置型号拼写差异;DeepSeek停在推理输出耗尽2048上限。两类结果与网络故障分开。冻结来源409d30f6、正式1.25.2/source d510cb70/候选20261104134035379,原PID36552/Provider保持;计划[LIVE-PROVIDER-Z-PLAN.md](LIVE-PROVIDER-Z-PLAN.md)。
| 路线 | 真实请求与用量 | 结果 |
|---|---|---|
| apikey-Claude / Anthropic Messages SSE | 单次预检1请求、176报告tokens | 配置claude-opus-5.5,上游claude-opus-5-5;严格身份mismatch停止,0工具/工程/修改,不重试 |
| DeepSeek / Chat SSE | 预检1请求146tokens通过;新Node5请求47740tokens | 前4工程请求completed/matched;第5完整SSE含[DONE]但finish_reason=length,10522输入+2048输出=12570;2048输出全部为reasoning_tokens,无工具执行,协议完整性守卫拒绝 |
合计7请求、48062报告tokens、0未知消费、无自动重放/切池。DeepSeek最后一条modelIdentity是incomplete/complete=false,虽原SSE中的model均为deepseek-flash,不能把它写成5请求全部完整matched。最后的流格式收尾存在,不表示模型答复完整;不把原protocol失败改成网络超时或已完成。
## 工程与状态
DeepSeek自行规划原test/build并保存真实基线:九项测试0通过9失败,build通过;读测试和源码后第5请求终止。0提案/批次/修改/修改后检查/Git交付;原HEAD、owner暂存/工作区/未跟踪字节保持。原任务不续跑、不增加输出或累计预算、不代写补检。Node37秒,原八请求/90000tokens上限内失败;不是步数或累计tokens阻断。
两个预检前后的全部原导出、原工件、sessions/records/events/outbox原行和原metadata(仅新增所属尝试除外)逐项一致,配置/选线/池均未写。DeepSeek工程收尾独立核对87原导出、3865原工件、244原尝试和原行精确保持;原预算/池/选线/record身份/空别名/空来源恢复,连接revision1→2→3、凭据1保持。外部诊断按完整原前缀核验,没有新追加。9229关闭、原PID36552、hasKey=true。
用户随后明确指出Claude型号是手误,并自行更正。新公开配置为claude-opus-5-5、connectionRevision2、credentialRevision2;不读取或推断凭据内容,也不恢复旧配置。原Z-Claude失败保留,新配置另冻Z2-Claude;此变化不回填原样本。其它线路与预算保持。
## 复核与后续
原请求/响应、用量、身份、真实工具/原检查和恢复证据位于.qa/live-provider-z/claude及deepseek,各自MANIFEST封存;[证据身份](live-provider-z-evidence-identity.json)记录长度/SHA和离线核对。代码/文档与阶段回顾本地提交,不推送。
只读查阅DeepSeek官方[Thinking Mode](https://api-docs.deepseek.com/guides/thinking_mode/)并保存117778字节原HTML:deepseek-flash默认开启high推理,Chat接口的显式开关为thinking.type=enabled/disabled;不是推测网络坏了。当前产品Chat原生参数白名单尚不支持thinking,而Responses的reasoning不适用于当前Chat连接。此兼容性缺口另列修复,不能伪造参数已发出或改Z预算。供应商根站公开页面返回404,不提供型号映射依据;未新增别名。
阶段回顾:两条API均能响应;本次没有出现连接超时或5xx。Claude由用户确认输入手误,DeepSeek的停止与默认推理及输出上限有原用量和官方契约支持。长期稳定性、完整自主交付、真实异常剔除收益和Purge仍未证明;React/Python/Purge保持Node闭环前置。下一优先用户已更正的Claude新样本,DeepSeek原失败及原预算保持。
@@ -0,0 +1,51 @@
{
"schemaVersion": 1,
"executionSource": "409d30f6",
"formalVersion": "1.25.2",
"manifests": [
{
"target": "claude",
"path": ".qa/live-provider-z/claude/MANIFEST.json",
"sha256": "0a35b6381861f2b2e3b9ab30f75e7f452a5385e917fe6cb58e9c45a75d91604a",
"files": 28,
"bytes": 30618075,
"offlinePassed": true
},
{
"target": "deepseek",
"path": ".qa/live-provider-z/deepseek/MANIFEST.json",
"sha256": "6e4fd9220103fad25a5c6a086b218cc3958b3ab50328ff9838ab61ad9de69bca",
"files": 123,
"bytes": 52370774,
"offlinePassed": true
}
],
"publicDocs": [
{
"path": ".qa/provider-z-public-docs/apikey-home.html",
"bytes": 130,
"sha256": "15421f4622106a4cc24dab91db95db9a50d3bdf4d0e32a6b5cb3d9d61b726b8b"
},
{
"path": ".qa/provider-z-public-docs/deepseek-thinking-excerpt.txt",
"bytes": 12513,
"sha256": "dd33ee2a438560369e8ea53b555673bad3d153409d02235e260bddd877992df7"
},
{
"path": ".qa/provider-z-public-docs/deepseek-thinking.html",
"bytes": 117778,
"sha256": "40c1515941a67a16102a3f98a0f0a5e4e395b29bb432607d8e74d695c522b571"
}
],
"requests": 7,
"reportedTokens": 48062,
"engineeringSuccess": false,
"originalFailuresPreserved": true,
"claudeUserCorrection": {
"model": "claude-opus-5-5",
"connectionRevision": 2,
"credentialRevision": 2,
"oldSampleUnchanged": true,
"noCredentialValueReadOrSavedByAgent": true
}
}
+2
View File
@@ -1,5 +1,7 @@
# 汇总发现
Z:Claude请求5.5/返回5-5的型号手误,用户已确认并保存新连接/凭据修订2,首样本1请求176tokens保留。DeepSeek默认high推理;预检146tokens通过,Node5请求47740tokens末轮finish_reason=length,2048输出全部reasoning_tokens,完整SSE也不能当完整答复;identity incomplete、0提案/修改/交付,原九基线失败/build通过。两轮151份82988849字节证据离线通过,原状态/预算/选线/凭据恢复,不续跑旧Task。官方Thinking Mode原HTML117778字节和SHA保存,当前Chat白名单不支持thinking是明确兼容缺口,不能假装已发送该参数或归因网络。
Y:1预检110tokens与8Node64500tokens全部matched,0上游失败/未知,三源码模型提案并封存应用,最后inspect_session/step_limit,未检查或Git交付。原Node收尾已恢复配置但因3条其它网络诊断追加失败;原11条诊断前缀按完整metadata行SHA精确重建。开场顺序快照池6、稍后驱动冻结池7(外部加DeepSeek)均保留;两次只读复核误判和准确时点对照不覆盖。独立对账84原导出/3777原工件/234原尝试/原行及配置通过;当前69/85/6607/13944、3857工件,原PID36552/Provider、凭据1、身份来源、9229关闭保持。用户指定apikey-Claude/DeepSeek另冻,不续跑Y。
## 1.25.2 起点
+51
View File
@@ -0,0 +1,51 @@
// Offline analysis of the original receipts; never reruns a failed request.
import fs from 'node:fs';
import path from 'node:path';
import assert from 'node:assert/strict';
import { root, save, sha, experiment } from './common.mjs';
const read = name => JSON.parse(fs.readFileSync(path.join(root, name)));
const preflight = read('preflight/result.json');
const requests = [];
const add = (stage, attempts, captures) => {
for (const attempt of attempts) {
const response = captures.find(value => value.attemptId === attempt.id && value.kind === 'response');
let done = false, finishReasons = [], reasoningDeltas = 0;
if (response) {
const raw = fs.readFileSync(path.join(root, response.path));
assert.equal(raw.length, response.bytes); assert.equal(sha(raw), response.sha256);
const text = raw.toString('utf8'); done = text.includes('[DONE]') || text.includes('event: message_stop');
for (const line of text.split(/\r?\n/).filter(value => value.startsWith('data: ') && value !== 'data: [DONE]')) {
const chunk = JSON.parse(line.slice(6));
for (const choice of chunk.choices ?? []) {
if (choice.finish_reason) finishReasons.push(choice.finish_reason);
if (choice.delta?.reasoning_content) reasoningDeltas++;
}
if (chunk.delta?.stop_reason) finishReasons.push(chunk.delta.stop_reason);
}
}
requests.push({ stage, id: attempt.id, status: attempt.status, requested: attempt.model,
reported: attempt.modelIdentity?.reportedModels, identity: attempt.modelIdentity?.status,
identityComplete: attempt.modelIdentity?.complete, usageStatus: attempt.usageStatus, usage: attempt.usage,
requestUrl: attempt.requestUrl, protocol: attempt.protocol, done, finishReasons, reasoningDeltas });
}
};
add('preflight', read('preflight/attempts.json'), preflight.captures);
let node;
if (fs.existsSync(path.join(root, 'node/result.json'))) {
node = read('node/result.json');
assert.equal(sha(fs.readFileSync(path.join(root, 'node/goal.txt'))), 'b5421d0bb502126f49a5153729027f13ef82db86ab19a9e5b0c23a49070bb905');
add('node', read('node/attempts.json'), node.captures);
}
const result = { route: experiment.name, preflightPassed: preflight.passed,
engineeringStarted: !!node, engineeringSuccess: node?.success ?? false, requests,
requestCount: requests.length, reportedTokens: requests.reduce((sum, value) => sum + (value.usage?.total_tokens ?? 0), 0),
everyReportedIdentityMatched: requests.every(value => value.identity === 'matched' && value.identityComplete),
everyAttemptCompleted: requests.every(value => value.status === 'completed'),
note: 'Model identity, transport/stream completion and engineering success are separate observations.',
toolCallsOnPreflight: preflight.actualTools, failure: node?.failure, noAutomaticReplay: true, verifierModelRequests: 0 };
save('analysis.json', result);
console.log(JSON.stringify({ route: result.route, requestCount: result.requestCount,
reportedTokens: result.reportedTokens, engineeringSuccess: result.engineeringSuccess,
everyReportedIdentityMatched: result.everyReportedIdentityMatched,
everyAttemptCompleted: result.everyAttemptCompleted, lastFinish: requests.at(-1)?.finishReasons,
lastReasoningTokens: requests.at(-1)?.usage?.completion_tokens_details?.reasoning_tokens }));
+42
View File
@@ -0,0 +1,42 @@
// Verify already-saved preflight snapshots; no host/model request or config write.
import fs from 'node:fs';
import path from 'node:path';
import assert from 'node:assert/strict';
import { root, save, sha, experiment } from './common.mjs';
const read = file => JSON.parse(fs.readFileSync(path.join(root, file)));
const before = read('preflight/before.json'), after = read('preflight/after.json');
const result = read('preflight/result.json'), owned = read('preflight/owned.json');
const attempts = read('preflight/attempts.json'), session = read('preflight/session.json');
for (const field of ['pid', 'version', 'runtimeRoot', 'hasKey', 'configSha256']) assert.equal(after[field], before[field], field);
assert.deepEqual(before.active, []); assert.deepEqual(after.active, []);
assert.equal(after.projects, before.projects + 1); assert.equal(after.exports.length, before.exports.length + 1);
for (const item of before.exports) assert.deepEqual(after.exports.find(value => value.id === item.id), item);
for (const table of ['sessions', 'records', 'events', 'outbox']) {
assert.equal(after.database.tables[table].schemaSha256, before.database.tables[table].schemaSha256);
const rows = new Map(after.database.tables[table].rows.map(value => [value.id, value]));
for (const row of before.database.tables[table].rows) assert.deepEqual(rows.get(row.id), row, table + '/' + row.id);
}
const metadata = new Map(after.database.tables.metadata.rows.map(value => [value.id, value]));
for (const row of before.database.tables.metadata.rows.filter(value => value.id !== 'modelAttempts')) {
assert.deepEqual(metadata.get(row.id), row, 'No preflight config or diagnostic write: ' + row.id);
}
for (const row of before.persistent.artifacts) assert.deepEqual(after.persistent.artifacts.find(value => value.path === row.path), row);
assert.deepEqual(after.modelCenter.selected, before.modelCenter.selected);
assert.deepEqual(after.modelCenter.pools, before.modelCenter.pools);
assert.equal(attempts.length, 1); assert.equal(result.requests, 1);
assert.equal(attempts[0].sessionId, owned.session.id); assert.equal(attempts[0].poolId, null); assert.equal(attempts[0].ordinal, 1);
assert.equal(attempts[0].model, experiment.model); assert.equal(attempts[0].protocol, experiment.protocol);
assert.equal(result.actualTools, 0); assert.equal(session.execution.run.toolCalls, 0);
assert.equal(read('preflight/inspector-final.json').closed, true);
for (const capture of result.captures) {
const bytes = fs.readFileSync(path.join(root, capture.path));
assert.equal(bytes.length, capture.bytes); assert.equal(sha(bytes), capture.sha256);
}
const verification = { passed: true, route: experiment.name, engineeringAllowed: result.passed,
originalExportsExact: before.exports.length, originalArtifactFilesExact: before.persistent.artifacts.length,
originalRowsExact: ['sessions', 'records', 'events', 'outbox'], publicMetadataExactExceptOneNewAttempt: true,
configSelectionPoolAndCredentialsUnchanged: true, originalPidPreserved: after.pid, inspectorClosed: true,
preflightRequests: 1, preflightReportedTokens: result.usage.total_tokens, toolsExecuted: 0,
verificationModelRequests: 0, stopReason: result.stopReason };
save('preflight/verification.json', verification);
console.log(JSON.stringify(verification));
+28
View File
@@ -0,0 +1,28 @@
import fs from 'node:fs';
import path from 'node:path';
import assert from 'node:assert/strict';
import { root, sha, experiment } from './common.mjs';
const read = name => JSON.parse(fs.readFileSync(path.join(root, name)));
const manifest = read('MANIFEST.json');
for (const row of manifest.files) {
const file = path.resolve(root, row.path); assert.ok(file.startsWith(root + path.sep));
const bytes = fs.readFileSync(file); assert.equal(bytes.length, row.bytes); assert.equal(sha(bytes), row.sha256);
}
assert.equal(read('preflight/verification.json').passed, true);
const analysis = read('analysis.json');
for (const stage of ['preflight', 'node']) {
const file = path.join(root, stage, 'result.json'); if (!fs.existsSync(file)) continue;
const result = JSON.parse(fs.readFileSync(file));
for (const row of result.captures) {
const bytes = fs.readFileSync(path.join(root, row.path));
assert.equal(bytes.length, row.bytes); assert.equal(sha(bytes), row.sha256);
}
if (stage === 'node') {
assert.equal(read('node/restoration.json').passed, true);
assert.equal(result.usage.requests <= 8, true); assert.equal(result.usage.totalTokens <= 90000, true);
assert.equal(sha(fs.readFileSync(path.join(root, 'node/goal.txt'))), 'b5421d0bb502126f49a5153729027f13ef82db86ab19a9e5b0c23a49070bb905');
}
}
console.log(JSON.stringify({ passed: true, route: experiment.name, files: manifest.files.length, bytes: manifest.totalBytes,
requests: analysis.requestCount, reportedTokens: analysis.reportedTokens, engineeringSuccess: analysis.engineeringSuccess,
modelRequestsExecutedByVerifier: 0 }));
+2 -1
View File
@@ -9,7 +9,8 @@
3. [complete] 混合进展按同轮实际新增区间结算,独立无变化保护保持;新v3读取批次释放小来源剩余预算给文件/日志、变更操作原序不动。联合43/43、补强2/2及吞吐最终38/38与构建通过,首失败保留。
4. [complete] 1.25.2/source d510不可变独立重建22/22、便携/隔离安装12组+75UI、正式保留PID36552升级及网页/原生后全部原状态exact。完整898/899+原断言1/1,原超时保留;1339份证据离线通过,阶段回顾及限定本地提交。
5. [complete] X预冻结拒绝0请求保持;用户选择另冻Y,预检通过、Node8请求64500tokens三源码模型封存并应用,step_limit前未检查/交付。原诊断元数据收尾失败、开场外部池变化及两次只读复核错误保持;独立对账通过,不续跑Y。
6. [in_progress] 用户新指定apikey-Claude/DeepSeek,另冻逐连接单次身份预检;通过才相同原题新Node。旧样本不续跑,不增加预算筛成功;Node闭环后才React/Python和Purge。
6. [complete] 用户指定Z两路线完成:Claude手误型号mismatch,DeepSeek预检通过但新Node第5请求2048输出全用于reasoning后length失败,0提案/交付;151份证据离线通过,状态恢复、原失败和回顾保持。
7. [in_progress] 用户自行更正Claude型号为claude-opus-5-5(连接/凭据修订2),另冻Z2单次预检/原题有界Node;DeepSeek thinking原生参数兼容缺口另列。旧样本不续跑,Node闭环后才React/Python/Purge。
细节和阶段回顾见docs/RELIABILITY-1.25.2.md。开始时session-catchup无待同步输出;计划文件首次补丁因findings标题上下文不符被拒且无文件变化,已按原标题修正。以下按历史时点理解。