feat(runtime): recover a bounded number of output-ceiling truncations
When a turn stops at the output-token ceiling (output_truncated), the run no longer dies. The truncated turn is still dropped whole - no tool is executed and the invalid assistant message is never appended - but the model is given a standalone instruction to split its proposals and the run continues. Recoveries are capped per run (2); past that the original diagnostic still fails the run. The connection is not cooled down for a truncation, since it responded fine and only the model overran its budget, so the recovery turn can reach it. Both the failed attempt and the recovering turn are accounted; neither is unknown. Fixture regression covers recover-then-continue and the exhausted cap.
This commit is contained in:
@@ -424,6 +424,11 @@ export class ModelHub {
|
||||
// A user stop is not evidence that the provider is unhealthy. Keep
|
||||
// the attempt, release its lease, and leave the connection eligible.
|
||||
this.health.set(this.identity(member), { ...health, state: 'interrupted', lastInterruptedAt: stamp() });
|
||||
} else if (diagnostic?.reason === 'output_truncated') {
|
||||
// The connection responded correctly; the model only overran its output
|
||||
// budget. That is not a health failure, so do not cool the member down -
|
||||
// the bounded recovery turn must be able to reach the same connection.
|
||||
this.health.set(this.identity(member), { ...health, state: 'responded', lastSuccessAt: stamp(), cooldownUntil: 0, failures: 0 });
|
||||
} else {
|
||||
const blocked = [401, 403].includes(attempt.httpStatus) || ['quota', 'model_identity'].includes(diagnostic?.kind);
|
||||
const failedAt = Date.now();
|
||||
|
||||
Reference in New Issue
Block a user