mirror of
https://github.com/wu736139669/hapi.git
synced 2026-08-08 07:17:39 +00:00
fix(cursor): requeue user message on transient agent exit (auth, rate limit) (#823)
* fix(cursor): requeue user message on transient agent exit (auth, rate limit)
cursorLegacyRemoteLauncher.runMainLoop popped a user message off the queue
before spawning `agent` and silently discarded it whenever `agent` exited
non-zero (auth expiry, rate limit, transient network). The wrapper logged
the failure at debug level only, never surfaced it to the web UI, and
emitted `ready` as if a normal turn had ended.
Capture stderr from the spawned process; classify exit-1 with a transient
signature (Authentication required, rate limit, ETIMEDOUT, ECONNRESET,
EAI_AGAIN) as recoverable; re-head the message via `queue.unshift`, surface
a friendly banner via `sendSessionEvent({type:'message',...})`, and backoff
~2s before the loop picks it up again. Cap at 5 consecutive transient
failures, after which the message is dropped with a clear "resolve and
resend" event so we never spin forever on a genuinely broken auth.
Non-transient non-zero exits also surface the stderr to the UI now (instead
of only the local ring buffer), so a real crash is visible to the operator.
Backoff is overridable via CURSOR_LEGACY_TRANSIENT_BACKOFF_MS for tests.
Tests cover: success path, transient auth requeue + banner, rate-limit
banner, non-transient crash surfaced without requeue, and the 5-failure
drop cap.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(cursor): preserve slash-command isolation on requeue; wait for stderr flush
Two findings from the cold-review bot on the requeue path:
1. `enqueueCursorUserMessage` uses `pushIsolated` for pass-through slash
commands (e.g. `/compress`) so they never batch with sibling prompts.
The transient-requeue path used plain `unshift`, which dropped the
isolate bit and allowed the next collected batch to merge the slash
command with a sibling - changing command semantics. Add
`MessageQueue2.unshiftIsolated` and use it when the popped batch was
isolated or when `parseCursorSpecialCommand` recognises the message.
2. `runAgentProcess` resolved on `child.on('exit', ...)`. Node may emit
`exit` while the stderr pipe is still draining, so a fast "auth
required" error printed-and-exited could be classified as
non-transient with empty stderr and silently drop the user message -
the exact bug this PR was supposed to fix. Resolve on `close` instead,
which waits for stdio streams to flush.
Adds a unit test that requeues `/compress` after a transient auth failure
and asserts the second spawn still receives the slash command alone (not
batched with a sibling).
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(cursor): restrict transient retry to exit code 1 only
Upstream codex review #823 (Minor): the helper treated any non-zero exit
with matching stderr as transient, which could requeue a signal-killed
(SIGTERM 143, SIGKILL 137) or crashed (SIGABRT 134) process whose stderr
happens to contain a keyword like "rate limit". Documented contract is
exit-1-for-transient; tighten the classifier accordingly.
Adds regression test covering exit 143 + rate-limit stderr → no retry.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(cursor): clean up transientBackoff abort listener on timer completion
Upstream codex review #823 (Minor): transientBackoff added an abort
listener with { once: true } but only removed it when the abort fired.
Because the launcher reuses one AbortController, repeated transient
retries accumulated stale listeners until the next abort.
Switch to a single completion path that clears the timer AND removes the
abort listener whichever side wins.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(cursor): cap in-memory stderr capture at 8 KB
Upstream codex review #823 (Minor): runAgentProcess accumulated every
stderr chunk for the full child lifetime. A noisy `agent` failure could
grow CLI process memory without bound even though only the first 400
chars are ever displayed. Cap the retained copy at 8 KB; debug log of the
full stream is unchanged.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
@@ -11,6 +11,7 @@ import {
|
||||
type RemoteLauncherExitReason
|
||||
} from '@/modules/common/remote/RemoteLauncherBase';
|
||||
import type { CursorSession } from './session';
|
||||
import type { EnhancedMode } from './loop';
|
||||
// TODO(cursor-acp): remove legacy stream-json resume path after migration window.
|
||||
// New Cursor sessions use ACP only. This path exists because pre-ACP Cursor
|
||||
// session_id values are not loadable via ACP session/load.
|
||||
@@ -19,6 +20,54 @@ import type { CursorStreamEvent } from './utils/cursorLegacyEventConverter';
|
||||
import { parseCursorEvent, convertCursorEventToAgentMessage } from './utils/cursorLegacyEventConverter';
|
||||
import { cursorPassThroughStatusMessage, parseCursorSpecialCommand } from './cursorSpecialCommands';
|
||||
|
||||
// Transient `agent` failures (auth expiry, rate limits, transient network) come back
|
||||
// as exit code 1 with a recognisable stderr signature. We requeue and retry instead
|
||||
// of silently swallowing the user message.
|
||||
const TRANSIENT_STDERR_PATTERN = /authentication required|please run ['"]?agent login['"]?|rate limit|ETIMEDOUT|ECONNRESET|EAI_AGAIN/i;
|
||||
const AUTH_STDERR_PATTERN = /authentication required|please run ['"]?agent login['"]?/i;
|
||||
const RATE_LIMIT_STDERR_PATTERN = /rate limit/i;
|
||||
const DEFAULT_TRANSIENT_BACKOFF_MS = 2_000;
|
||||
const MAX_CONSECUTIVE_TRANSIENT_FAILURES = 5;
|
||||
const STDERR_DISPLAY_LIMIT = 400;
|
||||
// In-memory stderr cap. Display only uses STDERR_DISPLAY_LIMIT chars; this is a
|
||||
// safety bound so a chatty `agent` failure cannot balloon CLI process memory.
|
||||
const STDERR_CAPTURE_LIMIT = 8_192;
|
||||
|
||||
function getTransientBackoffMs(): number {
|
||||
const raw = process.env.CURSOR_LEGACY_TRANSIENT_BACKOFF_MS;
|
||||
if (raw === undefined) return DEFAULT_TRANSIENT_BACKOFF_MS;
|
||||
const parsed = Number.parseInt(raw, 10);
|
||||
if (Number.isNaN(parsed) || parsed < 0) return DEFAULT_TRANSIENT_BACKOFF_MS;
|
||||
return parsed;
|
||||
}
|
||||
|
||||
function isTransientAgentError(exitCode: number, stderr: string): boolean {
|
||||
// Known transient failures (auth expiry, rate limit, transient network)
|
||||
// all come back as exit code 1. Keep the retry path narrow to that contract;
|
||||
// signal-kills (137 SIGKILL, 143 SIGTERM) and crashes (134 SIGABRT, etc.)
|
||||
// should never auto-retry even if their stderr happens to contain a matching
|
||||
// keyword.
|
||||
return exitCode === 1 && TRANSIENT_STDERR_PATTERN.test(stderr);
|
||||
}
|
||||
|
||||
function truncateStderrForDisplay(stderr: string): string {
|
||||
const trimmed = stderr.trim();
|
||||
if (!trimmed) return '(no stderr)';
|
||||
return trimmed.length > STDERR_DISPLAY_LIMIT
|
||||
? `${trimmed.slice(0, STDERR_DISPLAY_LIMIT)}...`
|
||||
: trimmed;
|
||||
}
|
||||
|
||||
function friendlyTransientMessage(exitCode: number, stderr: string): string {
|
||||
if (AUTH_STDERR_PATTERN.test(stderr)) {
|
||||
return "Cursor authentication expired. Re-run 'agent login' or set CURSOR_API_KEY. Your message is queued and will retry automatically.";
|
||||
}
|
||||
if (RATE_LIMIT_STDERR_PATTERN.test(stderr)) {
|
||||
return 'Cursor rate limit hit. Your message is queued and will retry automatically.';
|
||||
}
|
||||
return `Cursor agent failed transiently (exit ${exitCode}). Your message is queued and will retry automatically.`;
|
||||
}
|
||||
|
||||
function buildAgentArgs(opts: {
|
||||
message: string;
|
||||
cwd: string;
|
||||
@@ -57,6 +106,7 @@ class CursorRemoteLauncher extends RemoteLauncherBase {
|
||||
private readonly session: CursorSession;
|
||||
private abortController = new AbortController();
|
||||
private displayPermissionMode: string | null = null;
|
||||
private consecutiveTransientFailures = 0;
|
||||
|
||||
constructor(session: CursorSession) {
|
||||
super(process.env.DEBUG ? session.logPath : undefined);
|
||||
@@ -102,7 +152,7 @@ class CursorRemoteLauncher extends RemoteLauncherBase {
|
||||
break;
|
||||
}
|
||||
|
||||
const { message, mode } = batch;
|
||||
const { message, mode, isolate: batchIsolated } = batch;
|
||||
const specialCommand = parseCursorSpecialCommand(message);
|
||||
|
||||
const { mode: agentMode, yolo } = permissionModeToAgentArgs(mode.permissionMode as string);
|
||||
@@ -128,7 +178,7 @@ class CursorRemoteLauncher extends RemoteLauncherBase {
|
||||
session.onThinkingChange(true);
|
||||
|
||||
try {
|
||||
const exitCode = await this.runAgentProcess(args, session.path, (event) => {
|
||||
const { exitCode, stderr } = await this.runAgentProcess(args, session.path, (event) => {
|
||||
if (event.type === 'system' && event.subtype === 'init' && event.session_id) {
|
||||
cursorSessionId = event.session_id;
|
||||
session.onSessionFoundWithProtocol(event.session_id, 'stream-json');
|
||||
@@ -162,11 +212,19 @@ class CursorRemoteLauncher extends RemoteLauncherBase {
|
||||
}
|
||||
});
|
||||
|
||||
if (exitCode !== 0 && exitCode !== null) {
|
||||
logger.debug(`[cursor-remote] Agent exited with code ${exitCode}`);
|
||||
messageBuffer.addMessage(`Agent exited with code ${exitCode}`, 'status');
|
||||
if (exitCode === 0 || exitCode === null) {
|
||||
this.consecutiveTransientFailures = 0;
|
||||
} else if (isTransientAgentError(exitCode, stderr)) {
|
||||
await this.handleTransientAgentFailure(exitCode, stderr, message, mode, batchIsolated);
|
||||
} else {
|
||||
this.consecutiveTransientFailures = 0;
|
||||
const errMsg = `Agent exited (${exitCode}): ${truncateStderrForDisplay(stderr)}`;
|
||||
logger.warn(`[cursor-remote] ${errMsg}`);
|
||||
session.sendSessionEvent({ type: 'message', message: errMsg });
|
||||
messageBuffer.addMessage(errMsg, 'status');
|
||||
}
|
||||
} catch (error) {
|
||||
this.consecutiveTransientFailures = 0;
|
||||
logger.warn('[cursor-remote] Agent run failed', error);
|
||||
const errMsg = error instanceof Error ? error.message : String(error);
|
||||
session.sendSessionEvent({ type: 'message', message: `Cursor Agent failed: ${errMsg}` });
|
||||
@@ -184,7 +242,7 @@ class CursorRemoteLauncher extends RemoteLauncherBase {
|
||||
args: string[],
|
||||
cwd: string,
|
||||
onEvent: (event: ReturnType<typeof parseCursorEvent> & object) => void
|
||||
): Promise<number | null> {
|
||||
): Promise<{ exitCode: number | null; stderr: string }> {
|
||||
return new Promise((resolve, reject) => {
|
||||
const child = spawn('agent', args, {
|
||||
cwd,
|
||||
@@ -194,9 +252,11 @@ class CursorRemoteLauncher extends RemoteLauncherBase {
|
||||
windowsHide: process.platform === 'win32'
|
||||
});
|
||||
|
||||
let stderrCapture = '';
|
||||
|
||||
const abortHandler = () => {
|
||||
killProcessByChildProcess(child, false).catch(() => {});
|
||||
resolve(null);
|
||||
resolve({ exitCode: null, stderr: stderrCapture });
|
||||
};
|
||||
this.abortController.signal.addEventListener('abort', abortHandler);
|
||||
|
||||
@@ -209,9 +269,14 @@ class CursorRemoteLauncher extends RemoteLauncherBase {
|
||||
reject(err);
|
||||
});
|
||||
|
||||
child.on('exit', (code, signal) => {
|
||||
// `close` (not `exit`) waits for the stdio streams to flush before
|
||||
// firing. Otherwise stderr from an agent that prints + exits quickly
|
||||
// (e.g. "Authentication required" → exit 1) can arrive after we
|
||||
// already classified the failure, turning a transient error into a
|
||||
// dropped message.
|
||||
child.on('close', (code, signal) => {
|
||||
cleanup();
|
||||
resolve(code);
|
||||
resolve({ exitCode: code, stderr: stderrCapture });
|
||||
});
|
||||
|
||||
const rl = createInterface({ input: child.stdout, crlfDelay: Infinity });
|
||||
@@ -224,6 +289,9 @@ class CursorRemoteLauncher extends RemoteLauncherBase {
|
||||
|
||||
child.stderr?.on('data', (chunk) => {
|
||||
const text = chunk.toString();
|
||||
if (stderrCapture.length < STDERR_CAPTURE_LIMIT) {
|
||||
stderrCapture += text.slice(0, STDERR_CAPTURE_LIMIT - stderrCapture.length);
|
||||
}
|
||||
if (text.trim()) {
|
||||
logger.debug('[cursor-remote] agent stderr:', text.trim());
|
||||
}
|
||||
@@ -231,6 +299,78 @@ class CursorRemoteLauncher extends RemoteLauncherBase {
|
||||
});
|
||||
}
|
||||
|
||||
private async handleTransientAgentFailure(
|
||||
exitCode: number,
|
||||
stderr: string,
|
||||
message: string,
|
||||
mode: EnhancedMode,
|
||||
batchIsolated: boolean
|
||||
): Promise<void> {
|
||||
const session = this.session;
|
||||
const messageBuffer = this.messageBuffer;
|
||||
this.consecutiveTransientFailures += 1;
|
||||
|
||||
if (this.consecutiveTransientFailures >= MAX_CONSECUTIVE_TRANSIENT_FAILURES) {
|
||||
const summary = truncateStderrForDisplay(stderr);
|
||||
const dropMsg = `Cursor agent failed ${MAX_CONSECUTIVE_TRANSIENT_FAILURES} times in a row (${summary}). Dropping the queued message; resolve the issue ('agent login', wait out rate limit, etc.) and resend.`;
|
||||
logger.warn(
|
||||
`[cursor-remote] transient agent failures hit cap (${MAX_CONSECUTIVE_TRANSIENT_FAILURES}); dropping message`,
|
||||
{ exitCode, stderr: stderr.slice(0, STDERR_DISPLAY_LIMIT) }
|
||||
);
|
||||
session.sendSessionEvent({ type: 'message', message: dropMsg });
|
||||
messageBuffer.addMessage(dropMsg, 'status');
|
||||
this.consecutiveTransientFailures = 0;
|
||||
return;
|
||||
}
|
||||
|
||||
logger.warn(
|
||||
'[cursor-remote] transient agent failure, requeueing user message',
|
||||
{
|
||||
exitCode,
|
||||
attempt: this.consecutiveTransientFailures,
|
||||
stderr: stderr.slice(0, STDERR_DISPLAY_LIMIT)
|
||||
}
|
||||
);
|
||||
// Preserve isolation when the original batch was isolated (e.g. a
|
||||
// pass-through slash command queued via pushIsolated). Without this the
|
||||
// requeued command could be batched with a sibling prompt on retry and
|
||||
// change semantics. parseCursorSpecialCommand is the same gate
|
||||
// enqueueCursorUserMessage uses to decide isolation in the first place.
|
||||
const requeueIsolated = batchIsolated || parseCursorSpecialCommand(message).type !== null;
|
||||
if (requeueIsolated) {
|
||||
session.queue.unshiftIsolated(message, mode);
|
||||
} else {
|
||||
session.queue.unshift(message, mode);
|
||||
}
|
||||
const friendly = friendlyTransientMessage(exitCode, stderr);
|
||||
session.sendSessionEvent({ type: 'message', message: friendly });
|
||||
messageBuffer.addMessage(friendly, 'status');
|
||||
await this.transientBackoff(getTransientBackoffMs());
|
||||
}
|
||||
|
||||
private async transientBackoff(ms: number): Promise<void> {
|
||||
if (ms <= 0) return;
|
||||
const signal = this.abortController.signal;
|
||||
if (signal.aborted) return;
|
||||
await new Promise<void>((resolve) => {
|
||||
let timer: ReturnType<typeof setTimeout> | null = null;
|
||||
// Single completion path so the abort listener is always removed,
|
||||
// whether the timer or the abort wins. Without this, repeated
|
||||
// transient retries on the same AbortController accumulate stale
|
||||
// listeners until the next abort fires them in bulk.
|
||||
const finish = () => {
|
||||
if (timer !== null) {
|
||||
clearTimeout(timer);
|
||||
timer = null;
|
||||
}
|
||||
signal.removeEventListener('abort', finish);
|
||||
resolve();
|
||||
};
|
||||
timer = setTimeout(finish, ms);
|
||||
signal.addEventListener('abort', finish, { once: true });
|
||||
});
|
||||
}
|
||||
|
||||
private applyDisplayMode(permissionMode: string | undefined): void {
|
||||
if (permissionMode && permissionMode !== this.displayPermissionMode) {
|
||||
this.displayPermissionMode = permissionMode;
|
||||
|
||||
Reference in New Issue
Block a user