feat(codex): mid-turn Steer via app-server turn/steer (#888) (#1606)

* feat(shared): steer capability gates and live steered signal schemas

- STEERING_SUPPORTED_FLAVORS / isSteeringSupportedForSession gate which
  agents can deliver queued messages into the active turn (pi, codex,
  cursor ACP; legacy stream-json cursor excluded)
- AgentState.steeringActive, DecryptedMessage.steered and
  messages-consumed  live signal (never persisted by the hub)

* feat(cli): queue reservations and steered messages-consumed option

- MessageQueue2 gains takeByLocalId/restoreReservation/
  beginReservationDispatch/commitReservation so an async steer can reserve
  a queued row without racing the main loop's turn/start drain
- emitMessagesConsumed accepts steered: true to mark mid-turn delivery

* feat(codex): mid-turn steer via app-server turn/steer (#888)

- CodexAppServerClient.steerTurn + TurnSteerParams/Response types
- CodexRemoteLauncher registers the steer-queued-message RPC handler:
  reserves the queued row, validates it against the active turn (no
  control commands, matching mode hash), injects via turn/steer with an
  epoch guard that invalidates in-flight steers on abort/cleanup
- steeringActive agent state tracks the active-turn window
- hub syncEngine gate opens to codex; messages-consumed relays steered

* feat(web): Steered badge and steer gating for codex sessions

- HappyUserMessage shows a ↳ Steered badge fed by the live
  messages-consumed steered signal, preserved across server echoes and
  refetches (mergeMessages carries the optimistic marker)
- SessionChat gates canSteer via isSteeringSupportedForSession instead of
  the pi-only check
- clearStaleQueuedStatus normalizes a queued status on an invoked message
- fix(web): drop duplicate showSessionSummaryInChat in markdown test
  (upstream typecheck breakage)

* fix(codex,shared): address bot findings on steer gate and ambiguous turn/steer

- STEERING_SUPPORTED_FLAVORS / isSteeringSupportedForSession advertise
  codex and pi only; cursor joins when its soft-steer handler lands (#1609)
- turn/steer now splits dispatch (stdin accepted) from completion (turn
  finished): the hub RPC acks once dispatch succeeds — never on the
  concurrent turn's completion, which can exceed the 30s RPC window
- queue row commits only after the turn settles; a rejected/aborted steer
  restores the row so the message still delivers via turn/start, and a
  dispatched steer is never restored (no duplicate delivery)
- steer carries clientUserMessageId (echoed as userMessage.clientId) so
  ambiguous transport failures can reconcile the thread later
- client tests cover dispatch/complete split and stdin-write failure

* fix(codex): reconcile dispatched steers before restoring; align error copy

- A dispatched turn/steer whose completion fails (disconnect / protocol
  error) is now reconciled via thread/read by clientUserMessageId before
  the queued row is restored — the instruction is only re-delivered by
  turn/start when the thread never received it
- Reconcile targets the pinned steer thread, not whichever turn is
  current when completion fails
- syncEngine unsupported-flavor error now matches the capability gate
  (Pi and Codex only until the cursor handler lands)
- launcher tests cover steer success (ack on dispatch), reconcile-accepted
  and reconcile-rejected outcomes

* fix(codex): consume the row at dispatch; drop background reconcile

- The hub RPC acks and the queue row is consumed as soon as stdin accepts
  turn/steer; completion is background-only logging. A dispatched steer is
  never restored, so the same localId cannot be re-delivered via turn/start
  after the caller was told the steer succeeded
- Dispatch failure (stdin write error) still restores the row and reports
  failure
- steer.completed rejection is always handled (no unhandled rejection on
  the dispatch-failure path)
- tests updated: completion failure after dispatch keeps the row consumed;
  dispatch failure restores it

* fix(codex): distinguish definite rejection from indeterminate completion

- Transport-level failures (timeout, abort, disconnect, spawn, protocol)
  carry an indeterminate marker; explicit JSON-RPC error responses do not
- After a dispatched steer, turn completion resolves → commit + consumed;
  a definite app-server rejection restores the row (instruction was never
  accepted, so turn/start cannot duplicate it); an indeterminate outcome
  leaves the row reserved so it can never be delivered twice
- Completion handling registers before awaiting dispatch so the
  dispatch-failure path cannot leak an unhandled rejection
- client/launcher tests cover explicit rejection (restore), indeterminate
  outcome (row stays reserved) and dispatch failure

* fix(codex): reconcile indeterminate steers instead of a permanent reservation

- After an indeterminate completion (disconnect/protocol), reconcile the
  thread by clientUserMessageId immediately: accepted → commit + consumed,
  provably rejected → restore, still unreadable → keep the reservation and
  retry from the main-loop top on later passes (post-reconnect)
- A row never sits in dispatching forever: the hub cannot stamp it invoked
  while the instruction may never have been accepted
- tests: indeterminate keeps reserved while thread unreadable; accepted
  reconciliation consumes; rejected path restores

* fix(codex): accept all thread item shapes; retry reconcile; ack through abort

- Reconcile matcher accepts userMessage/user_message with clientId/
  client_id, matching the shapes the thread parser supports — an accepted
  steer can no longer be misclassified as rejected
- A pending reconciliation schedules a wakeLoop retry, so a temporary
  app-server outage cannot strand the reservation behind waitForTurnOrRecovery
- The success-path ACK no longer checks the steer epoch: the hub already
  reported steered on dispatch, so commit + messages-consumed must reach
  it even when an abort resets the queue in between

* fix(codex): reinit reconnected app-server; keep reconcile retries alive

- thread/read after a disconnect auto-connects a fresh app-server, which
  must be initialized before any request — reconcile now ensures
  connect + initialize (isConnected getter added to the client)
- every still-unknown loop-top reconciliation schedules the next retry,
  so recovery without external traffic is eventually observed
- launcher mock gains isConnected

* fix(codex): timer-driven reconciliation; init tracking; abort-safe ACK

- Reconciliation runs on a self-rescheduling 1s timer independent of the
  main loop (wakes it too), so idle loops and waitForTurnOrRecovery still
  observe app-server recovery; abort clears nothing implicitly — the ACK
  path commits and consumes even when the reservation was cancelled
- Absence of a durable client id is ambiguous: unmatched reads stay
  'unknown' and keep retrying instead of restoring the row
- CodexAppServerClient tracks initialized state (reset on disconnect/exit)
  so ensureAppServerInitialized re-initializes a fresh process before
  thread/read; initialize failures leave the flag false for the next retry
- tests: accepted reconciliation via scheduled timer, indeterminate
  keeps reserved, explicit rejection restores

* fix(codex): bind reconciliation to the launcher lifecycle

- runSteerReconciliation clears any armed retry timer on entry and never
  installs a second one, so loop-top and timer-driven passes cannot
  multiply
- shuttingDown is set when the main loop ends: timers are cleared and the
  pending map is dropped, so an unresolved steer can never respawn an
  app-server after cleanup (remote-to-local switch included)

* fix(codex): report steered only after app-server acceptance

- The handler now awaits steer.completed (the inject-acceptance response):
  an explicit JSON-RPC rejection surfaces as failed and restores the row
  for the normal turn/start path instead of a false steered
- Transport failure after dispatch reports 'Steer outcome is being
  reconciled' and keeps the row reserved while the timer-driven thread
  reconciliation runs
- dispatch-failure path also swallows the paired completion rejection

* fix(steer): tri-state cancel, clear-safe reservations, bounded acceptance wait

- MessageQueue2.cancelByLocalId returns 'in-flight' for a dispatching
  steer reservation: the hub neither deletes the row nor stamps invoked_at
  (new CancelMessageResponse 'busy' status; web restores the optimistic
  row); pushIsolateAndClear and reset/close share cancelReservations so
  /clear-style commands cannot have a rejected steer resurrect a discarded
  prompt
- turn/steer acceptance wait bounded at 25s (< hub 30s RPC timeout): a
  lost response is indeterminate and funnels into thread reconciliation
  instead of stranding the reservation
- tests updated for the tri-state cancel contract

* fix(codex,web): busy-aware edit flow; bound reconciliation reads

- QueuedMessagesBar edit flow treats a 'busy' cancel as unsuccessful: it
  never prefills the composer when the row is inside an async steer, so a
  second client cannot send a duplicate
- reconcileSteerByClientId bounds thread/read with a 5s timeout so a
  connected-but-silent app-server cannot hold the reservation in-flight
  indefinitely

* fix(steer): inFlight-dominated cancel acks; bounded reconciliation

- hub cancel-queued-message acks check inFlight before removed: a stale
  duplicate socket reporting removed can no longer delete the durable row
  while another socket is dispatching the steer
- reconciliation entries expire after 60s and mark delivered: after the
  rejection window, a dispatched steer that the app-server never proved
  (client ids dropped on restart) is committed instead of polling
  thread/read forever
- pre-dispatch failures (abort before write included) never enter
  reconciliation — they restore the row and report failure

* fix(steer): persist indeterminate outcomes without replay

* fix(steer): make ambiguous delivery restart-safe

* fix(steer): recover crash-held rows and preserve retry dedup

* fix(steer): ack retries and bound stdin dispatch

* fix(steer): reconcile indeterminate dispatches and serialize retries

* fix(codex): classify stdin callback failures as indeterminate

* fix(steer): recheck indeterminate cancels after ACK

* fix(steer): close retry and abort races

* fix(steer): serialize live retries and abort admission

* fix(steer): distinguish live dispatching from unknown

* fix(steer): keep ACK failures held and reconcile busy cancel

* fix(steer): distinguish held cancel from removal

* fix(store): combine schema v24 migrations

* fix(store): reserve schema v25 for steer delivery state

* fix(steer): keep held cancel state and notify requeue

* fix(steer): release explicitly cancelled unknown reservations

* fix(codex): reject cancelled reservations before native steer

* fix(codex): make reservation restore atomic with state

* fix(codex): terminate abandoned transport writes

* fix(steer): own abandoned app-server lifecycle and consume races

* fix(codex): confirm dispatch and recover abandoned turns

* test(codex): mock abandoned transport callback

* fix(codex): clear visible turn state on transport loss

* fix(steer): claim retries and cover native delivery state

* fix(native): preserve indeterminate state on Android hydration

* fix(steer): make retry claims single-winner

* fix(steer): serialize concurrent retry claims

* fix(socket): tolerate missing steer-state ACK callbacks

* fix(native): serialize retry operations

* docs(web): document unknown steer delivery and retry controls

* fix(steer): handle retry failures and abort-before-connect

* fix(steer): reinitialize after transport loss and finish iOS retry errors

* fix(steer): preserve indeterminate rows across reconnect gaps

* test(web): mock indeterminate queued recovery state

* fix(steer): recover consumed ACK tombstones

* fix(steer): expose consumed cancel tombstones
This commit is contained in:
SSU-WEI HUANG
2026-08-19 20:07:39 +08:00
committed by GitHub
parent 2fbd98dfe4
commit f0e5ba9c0f
82 changed files with 2735 additions and 227 deletions
+218 -56
View File
@@ -25,6 +25,8 @@ import type {
TurnInterruptResponse,
ThreadRollbackParams,
ThreadRollbackResponse,
TurnSteerParams,
TurnSteerResponse,
ThreadCompactStartParams,
ThreadCompactStartResponse,
ThreadGoalSetParams,
@@ -63,9 +65,18 @@ type RequestHandler = (params: unknown) => Promise<unknown> | unknown;
type PendingRequest = {
resolve: (value: unknown) => void;
reject: (error: Error) => void;
rejectDispatched: (error: Error) => void;
cleanup: () => void;
};
/** Marks transport-level failures whose steer outcome is unknown. */
export const INDETERMINATE_SYMBOL = Symbol('codex-app-server-indeterminate');
export function isIndeterminateError(error: unknown): boolean {
return typeof error === 'object' && error !== null
&& (error as Record<symbol, unknown>)[INDETERMINATE_SYMBOL] === true;
}
type CodexAppServerClientOptions = {
cwd?: string;
};
@@ -164,10 +175,22 @@ function resolveCodexAppServerCommand(): string {
export class CodexAppServerClient extends JsonLineParser {
private process: ChildProcessWithoutNullStreams | null = null;
private connected = false;
private initialized = false;
/** True while the app-server process is connected. */
isConnected(): boolean {
return this.connected;
}
/** True after a successful initialize round-trip on the current process. */
isInitialized(): boolean {
return this.initialized;
}
private nextId = 1;
private readonly pending = new Map<number, PendingRequest>();
private readonly requestHandlers = new Map<string, RequestHandler>();
private notificationHandler: ((method: string, params: unknown) => void) | null = null;
private transportAbandonedHandler: (() => void) | null = null;
private stderrHandler: ((text: string) => void) | null = null;
private protocolError: Error | null = null;
@@ -188,7 +211,7 @@ export class CodexAppServerClient extends JsonLineParser {
const codexCommand = resolveCodexAppServerCommand();
logger.debug(`[CodexAppServer] Starting ${codexCommand} app-server`);
this.process = spawn(codexCommand, ['app-server'], {
const child = spawn(codexCommand, ['app-server'], {
cwd: this.options.cwd,
env: Object.keys(process.env).reduce((acc, key) => {
const value = process.env[key];
@@ -199,12 +222,16 @@ export class CodexAppServerClient extends JsonLineParser {
shell: process.platform === 'win32',
windowsHide: process.platform === 'win32'
});
this.process = child;
this.process.stdout.setEncoding('utf8');
this.process.stdout.on('data', (chunk) => this.feed(chunk));
child.stdout.setEncoding('utf8');
child.stdout.on('data', (chunk) => {
if (this.process === child) this.feed(chunk);
});
this.process.stderr.setEncoding('utf8');
this.process.stderr.on('data', (chunk) => {
child.stderr.setEncoding('utf8');
child.stderr.on('data', (chunk) => {
if (this.process !== child) return;
const text = chunk.toString().trim();
if (text.length > 0) {
logger.debug(`[CodexAppServer][stderr] ${text}`);
@@ -212,16 +239,20 @@ export class CodexAppServerClient extends JsonLineParser {
}
});
this.process.on('exit', (code, signal) => {
child.on('exit', (code, signal) => {
if (this.process !== child) return;
const message = `Codex app-server exited (code=${code ?? 'null'}, signal=${signal ?? 'null'})`;
logger.debug(message);
this.rejectAllPending(new Error(message));
this.connected = false;
this.initialized = false;
this.resetParserState();
this.process = null;
this.transportAbandonedHandler?.();
});
this.process.on('error', (error) => {
child.on('error', (error) => {
if (this.process !== child) return;
logger.debug('[CodexAppServer] Process error', error);
const message = error instanceof Error ? error.message : String(error);
this.rejectAllPending(new Error(
@@ -231,6 +262,7 @@ export class CodexAppServerClient extends JsonLineParser {
this.connected = false;
this.resetParserState();
this.process = null;
this.transportAbandonedHandler?.();
});
this.connected = true;
@@ -241,6 +273,10 @@ export class CodexAppServerClient extends JsonLineParser {
this.notificationHandler = handler;
}
setTransportAbandonedHandler(handler: (() => void) | null): void {
this.transportAbandonedHandler = handler;
}
registerRequestHandler(method: string, handler: RequestHandler): void {
this.requestHandlers.set(method, handler);
}
@@ -248,6 +284,7 @@ export class CodexAppServerClient extends JsonLineParser {
async initialize(params: InitializeParams): Promise<InitializeResponse> {
const response = await this.sendRequest('initialize', params, { timeoutMs: 30_000 });
this.sendNotification('initialized');
this.initialized = true;
return response as InitializeResponse;
}
@@ -351,6 +388,24 @@ export class CodexAppServerClient extends JsonLineParser {
return response as ThreadRollbackResponse;
}
async steerTurn(
params: TurnSteerParams,
options?: { signal?: AbortSignal }
): Promise<{ dispatched: Promise<void>; completed: Promise<TurnSteerResponse> }> {
// Dispatch/complete split: the caller awaits acceptance before
// reporting success. Bound the wait below the hub's 30s RPC timeout so
// a lost response cannot strand the row in dispatching — a timeout is
// indeterminate and funnels into thread reconciliation.
const request = await this.sendRequestWithDispatch('turn/steer', params, {
signal: options?.signal,
timeoutMs: 20_000
});
return {
dispatched: request.dispatched,
completed: request.completed.then((response) => response as TurnSteerResponse)
};
}
async compactThread(
params: ThreadCompactStartParams,
options?: { signal?: AbortSignal }
@@ -413,6 +468,7 @@ export class CodexAppServerClient extends JsonLineParser {
} finally {
this.rejectAllPending(new Error('Codex app-server disconnected'));
this.connected = false;
this.initialized = false;
this.resetParserState();
}
@@ -424,9 +480,30 @@ export class CodexAppServerClient extends JsonLineParser {
params?: unknown,
options?: { signal?: AbortSignal; timeoutMs?: number }
): Promise<unknown> {
const request = await this.sendRequestWithDispatch(method, params, options);
void request.dispatched.catch(() => {});
return request.completed;
}
/**
* Split a request into transport dispatch (stdin accepted) and completion
* (JSON-RPC response). Lets callers commit queue state once stdin accepted
* the request without waiting for the (possibly long-running) response.
*/
private async sendRequestWithDispatch(
method: string,
params?: unknown,
options?: { signal?: AbortSignal; timeoutMs?: number }
): Promise<{ dispatched: Promise<void>; completed: Promise<unknown> }> {
if (options?.signal?.aborted) {
throw createAbortError();
}
if (!this.connected) {
await this.connect();
}
if (options?.signal?.aborted) {
throw createAbortError();
}
const id = this.nextId++;
const payload: JsonRpcLiteRequest = {
@@ -437,60 +514,134 @@ export class CodexAppServerClient extends JsonLineParser {
const timeoutMs = options?.timeoutMs ?? CodexAppServerClient.DEFAULT_TIMEOUT_MS;
return new Promise((resolve, reject) => {
let timeout: ReturnType<typeof setTimeout> | null = null;
let aborted = false;
const cleanup = () => {
if (timeout) {
clearTimeout(timeout);
}
if (options?.signal) {
options.signal.removeEventListener('abort', onAbort);
}
};
const onAbort = () => {
if (aborted) return;
aborted = true;
this.pending.delete(id);
cleanup();
reject(createAbortError());
};
let timeout: ReturnType<typeof setTimeout> | null = null;
let resolveDispatched!: () => void;
let rejectDispatched!: (error: Error) => void;
let resolveCompleted!: (value: unknown) => void;
let rejectCompleted!: (error: Error) => void;
const dispatched = new Promise<void>((resolve, reject) => {
resolveDispatched = resolve;
rejectDispatched = reject;
});
const completed = new Promise<unknown>((resolve, reject) => {
resolveCompleted = resolve;
rejectCompleted = reject;
});
let aborted = false;
let dispatchSettled = false;
const cleanup = () => {
if (timeout) {
clearTimeout(timeout);
}
if (options?.signal) {
if (options.signal.aborted) {
onAbort();
options.signal.removeEventListener('abort', onAbort);
}
};
const abandonUnconfirmedDispatch = (error: Error) => {
const child = this.process;
this.process = null;
this.connected = false;
this.initialized = false;
try {
child?.stdin.destroy();
} catch (destroyError) {
logger.debug('[CodexAppServer] Error destroying stalled stdin', destroyError);
}
this.rejectAllPending(error);
this.resetParserState();
this.transportAbandonedHandler?.();
};
const failRequest = (error: Error, abandon = false) => {
if (abandon) {
abandonUnconfirmedDispatch(error);
return;
}
this.pending.delete(id);
cleanup();
if (!dispatchSettled) {
dispatchSettled = true;
rejectDispatched(error);
}
rejectCompleted(error);
};
const onAbort = () => {
if (aborted) return;
aborted = true;
const error = this.markIndeterminate(createAbortError());
if (dispatchSettled) {
failRequest(error);
} else {
failRequest(error, true);
}
};
if (options?.signal) {
if (options.signal.aborted) {
onAbort();
return { dispatched, completed };
}
options.signal.addEventListener('abort', onAbort, { once: true });
}
if (Number.isFinite(timeoutMs) && !aborted) {
timeout = setTimeout(() => {
if (this.pending.has(id)) {
const error = this.markIndeterminate(new Error(`Codex app-server request '${method}' timed out after ${timeoutMs}ms`));
failRequest(error, !dispatchSettled);
}
}, timeoutMs);
timeout.unref();
}
this.pending.set(id, {
resolve: (value) => {
cleanup();
if (!dispatchSettled) {
dispatchSettled = true;
resolveDispatched();
}
resolveCompleted(value);
},
reject: (error) => {
cleanup();
if (!dispatchSettled) {
dispatchSettled = true;
resolveDispatched();
}
rejectCompleted(error);
},
rejectDispatched: (error) => {
if (!dispatchSettled) {
dispatchSettled = true;
rejectDispatched(error);
}
},
cleanup
});
try {
const serialized = JSON.stringify(payload);
this.process?.stdin.write(`${serialized}\n`, (error) => {
if (error) {
const writeError = error instanceof Error ? error : new Error(String(error));
failRequest(this.markIndeterminate(writeError), !dispatchSettled);
return;
}
options.signal.addEventListener('abort', onAbort, { once: true });
}
if (Number.isFinite(timeoutMs)) {
timeout = setTimeout(() => {
if (this.pending.has(id)) {
this.pending.delete(id);
cleanup();
reject(new Error(`Codex app-server request '${method}' timed out after ${timeoutMs}ms`));
}
}, timeoutMs);
timeout.unref();
}
this.pending.set(id, {
resolve: (value) => {
cleanup();
resolve(value);
},
reject: (error) => {
cleanup();
reject(error);
},
cleanup
if (!dispatchSettled) {
dispatchSettled = true;
resolveDispatched();
}
});
} catch (error) {
const writeError = error instanceof Error ? error : new Error(String(error));
failRequest(writeError);
}
this.writePayload(payload);
});
return { dispatched, completed };
}
private sendNotification(method: string, params?: unknown): void {
@@ -604,6 +755,15 @@ export class CodexAppServerClient extends JsonLineParser {
pending.resolve(response.result);
}
/** Marks transport-level failures (timeout, disconnect, spawn, protocol)
* whose outcome is unknown — unlike an explicit JSON-RPC error response.
* Steer completion uses this to distinguish definite rejection from
* indeterminate outcomes. */
private markIndeterminate(error: Error): Error {
Object.defineProperty(error, INDETERMINATE_SYMBOL, { value: true });
return error;
}
private writePayload(payload: JsonRpcLiteRequest | JsonRpcLiteNotification | JsonRpcLiteResponse): void {
const serialized = JSON.stringify(payload);
this.process?.stdin.write(`${serialized}\n`);
@@ -615,8 +775,10 @@ export class CodexAppServerClient extends JsonLineParser {
}
private rejectAllPending(error: Error): void {
for (const { reject, cleanup } of this.pending.values()) {
error = this.markIndeterminate(error);
for (const { reject, rejectDispatched, cleanup } of this.pending.values()) {
cleanup();
rejectDispatched(error);
reject(error);
}
this.pending.clear();