9 Commits

Include renames

Author	SHA1	Message	Date
3 people	86c88df93b	fix(runner,hub): cross-platform orphan reap on archive / tracking loss (#1911) ... * fix(runner,hub): reap detached driver CLI on archive and tracking loss Archive always verifies via stopRunnerSession; runner stopSession scans argv orphans and awaits webhook-timeout tree-kill so PPID=1 children cannot accumulate after KillSession miss or map drop. Fixes #1910 Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli): keep --hapi-session-id reap-only; test orphan kill path Claude/cursor swallow the stamp without bootstrap reuse, and reapRunnerSpawnedOrphans is unit-tested as the stopSession map-miss path. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli,hub): address #1911 CI and bot Majors Use Vitest in orphanReap tests, add Pi session.on mock, refuse archive when machine StopSession is unreachable, trust full tree-kill results, and argv-sweep orphans after every mapped stop. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(runner): gate late-webhook kill on timeout orphans; fail closed on scan error Track webhook-timeout PIDs separately so recovered shared roots are not killed, and treat ps-list failures as still_alive rather than empty. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(runner): keep shared Codex wrapper alive when archiving one root StopSession must detach a single shared root without tree-killing the wrapper PID while sibling roots remain, including after KillSession has already marked the runtime inactive. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(runner): keep shared Codex siblings after restart tracking loss Before persisted-PID or argv orphan kills, consult the durable runtime registry for active sibling roots on the same wrapper PID so archiving one root after a runner restart cannot tree-kill live shared sessions. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(hub,cli): deliver hub-authored archive metadata to reconnecting CLIs When KillSession misses and archive is written via markSessionArchivedFromHub, emit Socket.IO update-session to the session room. On reconnect, ApiSessionClient reconciles GET /cli/sessions/:id metadata so hub-archived still fires. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(runner): argv-scan orphans even when shared siblings remain Remove session-wide early returns that skipped orphan reaping whenever any shared wrapper had active siblings. Keep only the PID-specific filter so older untracked CLIs with the same HAPI id are still reaped. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(runner): protect tracked shared wrappers when runtime registry is empty Argv orphan sweeps must exclude in-memory shared wrapper PIDs that still host sibling roots, not only durable registry siblings — otherwise a missing runtime record lets the sweep tree-kill the wrapper while archiving one root. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(runner): adopt untracked nonshared CLIs after runner restart When a runner-spawned Claude webhook arrives with no TrackedSession (typical after restart mid-bootstrap), durably adopt the PID instead of ignoring it so StopSession can reap. Keep killing only nonshared timeout orphans; never kill shared Codex wrappers on late webhooks. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(runner): keep recovered shared wrappers for registry siblings After restart adoption tracks only a newly reported Codex root, archiving it must consult the durable registry before tree-killing the wrapper PID so older active sibling roots survive. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(hub): do not archive on KillSession ack when StopSession is unknown KillSession returns before cleanupAndExit finishes. When the runner cannot find the PID (unknown), wait for the session KillSession handler to drop before archiving; refuse if the CLI socket remains reachable. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(hub,cli): confirm KillSession pid via StopSession before archive KillSession now returns the CLI pid. When session-id StopSession is unknown, archive confirms that OS pid (without treating socket loss as exit) and still refuses when confirmation stays unknown or still_alive. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(runner,hub): tree-kill adopted CLIs and require PID start markers Adopted post-restart runner sessions now use killProcessTreeByPid when no ChildProcess handle exists. KillSession carries a process-start marker so raw PID confirmation cannot tree-kill a reused OS pid. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(runner,hub): verify adopted PID markers and hub-archive after stop Adopted sessions compare the persisted start marker before tree-kill and drop stale tracking on mismatch. After confirmed StopSession, hub-authors archive metadata when the CLI did not finish its flush. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(runner): refuse unconfirmed shared/adopted stops Keep shared wrappers when siblings remain, but return unknown unless runtimeControl already ended the root. Adopted PIDs without a start marker no longer tree-kill by tracked PID alone. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(runner): require inactive binding before sibling-kept archive Siblings alone must not flip StopSession to stopped on retry, while an acknowledged inactive Codex registry binding remains stop proof. Handle PID- markers before any tracked kill so reuse cannot nuke strangers. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(runner): distinguish orphan reap from unknown keep-wrapper base Pass unknown into finishWithOrphanSweep so a successful argv reap stays stopped instead of being overwritten by an absent registry binding. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(runner): honor orphan reap when adopted PID lacks a start marker finishWithOrphanSweep with an unknown base lets a successful argv kill report stopped instead of forcing unknown after the marker probe fails. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(hub): confirm archive via metadata.hostPid tombstone When StopSession(sessionId) is unknown and KillSession supplies no confirmable pid+marker, probe PID-<metadata.hostPid>. already_gone allows archive; still_alive/unknown still refuse (#1911 Peer #1820). Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli): escalate Windows taskkill to /F like SIGTERM→SIGKILL Soft taskkill /T is refused on win32 console trees; orphanReap left orphans still_alive. waitForProcessToDie now force-escalates on both platforms so cross-platform reaping matches. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli): widen process.kill mock signal to string | number Match @types/node process.kill(pid, signal?: string | number) so the Windows taskkill escalation mocks typecheck on tip 54f141368. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli): tasklist liveness + immediate /F after soft taskkill refuse Windows orphan reap still failed in 5ms on Teemo: soft taskkill refused but escalate waited on an unreliable process.kill(0) probe. Prefer tasklist for win32 liveness and escalate to /F as soon as the PID is still live after soft kill. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli): coerce win32 ps-list PIDs and escalate soft taskkill to /F String PIDs from ps-list made Number.isFinite fail so orphan kill returned still_alive without calling taskkill. Coerce PIDs, prefer tasklist liveness, and escalate soft taskkill to /F like SIGTERM→SIGKILL. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli): Windows orphan scan via CIM CommandLine + taskkill /F escalate ps-list/fastlist is missing CommandLine and often missing from single-exe bundles, so Teemo orphan reap returned scan_failed (logged as live PIDs). List via Win32_Process CIM, coerce PIDs, escalate soft taskkill to /F. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli): grace-wait after successful soft taskkill before /F Escalate immediately only when soft taskkill is refused. When soft succeeds but the PID is still draining, honor the existing grace period before force (Codex Major on #1911). Co-authored-by: Cursor <cursoragent@cursor.com> * fix(runner): re-verify start marker before orphan argv tree-kill Capture getProcessStartMarker after the argv match list, re-check immediately before killTree, and skip on mismatch/null-while-alive so PID reuse cannot tree-kill an unrelated process on the orphan fallback. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(hub): preallocate HAPI id before fresh machine spawn Fresh POST /machines/:id/spawn omitted existingSessionId, so argv never got a reap stamp. Preallocate the hub row (fork/clear pattern) and pass that id so a runner restart before the first webhook remains reapeable. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli,hub): close three post-prealloc unreapable paths Pass --hapi-session-id into Claude/cursor/kimi/copilot bootstrap so the preallocated hub row is used. Treat null marker re-check while alive as still_alive (not stopped). Keep the preallocated stub when spawn-fail StopSession cannot confirm the child is gone. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli): adopt-stub via reservedSessionId, not existingSessionId `--hapi-session-id` binds a fresh create bootstrap to a hub-preallocated UUID (getOrCreateSession id) without taking the reopen path. Non-UUID stamps stay reap-only to match hub create schema. Restores integration spawn invariants broken by mapping hapi→existingSessionId. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli,hub): close Opus cold-read B1–B3 for #1911 B1: capture start markers from the same process snapshot as argv match (POSIX ps lstart / Win32 CreationDate); re-check before kill. Continue other orphans when one target is unconfirmed. B2: Windows killProcessTreeByPid verifies every collected descendant, not root-only — taskkill /T exit 0 is signalled, not gone. B3: runner sets childStarted:false on pre-exec rejection; hub deletes the preallocated stub without StopSession. Post-exec ambiguity still keeps the row until stop confirms. Terminal archives no longer refuse when no runner is connected. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli,hub): explicit adopt for hub-preallocated session stubs Fresh machine spawn preallocates with tag machine-spawn:<uuid>; CLI create with a different tag and the same id 409'd on identity conflict. Add adoptPreallocatedSession (hub-internal stub check) and POST /cli/sessions { adopt: true, id } so create binds the reserved row and overwrites tag/metadata without guessing the hub tag. Tests fail-first on the real prealloc→CLI create path. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli): preserve snapshot markers + unify Windows CIM start markers Runner stopSession was still on findOrphans (PID-only), discarding same-snapshot markers and reopening the PID-reuse kill window. Switch both call sites to findTargets. Force UTC ISO 'o' CreationDate for list and single-PID probe so ConvertTo-Json cannot disagree with getProcessStartMarker on WinPS 5.1. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli): exercise production orphan sweep wiring in tests Extract findStopSessionOrphanTargets as the stopSession discovery path both run.ts sweeps call. Add tests that drive that helper end-to-end (not an injected findOrphans seam), assert run.ts source wiring, and cover CIM-list ISO vs probe ISO agreement plus the broken WinPS shape mismatch. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(hub,cli): never delete live spawn on RPC timeout; release stub tags Ambiguous machine-spawn failures (timeout / unset childStarted) no longer call stopRunnerSession or deleteSession — that path killed healthy late boots and CASCADE-wiped transcripts. Pre-exec delete still requires a real machine-spawn stub (store tag check). Metadata updates release the stub tag so reopen flavors are not permanently adoptable; adopt refuses archived stubs. POSIX pgrep ENOENT fails closed instead of root-only verify. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli): parseable Windows tree scan + fail closed on CIM failure Space-joined PowerShell for collectWindowsProcessTree was a WinPS parse error ($seen=@{} $bfs=@()), so every scan fell open to [root] and B2 false-stopped again. Join with newlines; return scan_failed on error/empty; killProcessTreeByPid refuses root-only verify. Assert on generated command text (Teemo-confirmed) so mocks cannot hide the next malformed script. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli,hub): reopen stamps existing-session-id; scan-fail signals root Opus Critical on #1911: runner always stamps --existing-session-id for known hub ids so Claude/Kimi/Copilot reopen does not hit adopt-stub 409. On pgrep/CIM tree-scan failure still signal the verified root but never claim stopped. Archive when both KillSession and machine RPC are missing. Empty Win process-list stdout and strict Codex registry reads fail closed. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli,hub): OK: sentinel for Windows tree scan; adopt CAS CIM failure under SilentlyContinue printed root-only with exit 0 — byte-identical to a healthy childless tree, so scan_failed never fired and killProcessTreeByPid false-stopped (#1911 Overseer B1). Require ErrorActionPreference=Stop, trap exit 1, and an OK: success sentinel; parse anything else as scan_failed. Same Stop/trap on the machine-wide list command. Execute the generated scripts under Docker pwsh in CI when the image is present. Treat pgrep status=null as scan_failed; coarsen Windows tasklist poll; wrap adoptPreallocatedSession in a transaction with stub-tag CAS. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli,hub): reservedSessionId for fresh prealloc; signal Win survivors Fresh machine-spawn stubs must not stamp --existing-session-id — that sends Codex down reopen without a thread binding (#1911 bot Major) and re-broke local HTTP non-UUID hints as reopen (round-4). Hub passes prealloc as reservedSessionId; buildCliArgs stamps --hapi-session-id for reserved/sessionId and --existing-session-id only for live reopen. Codex parses --hapi-session-id and adopts via bootstrapSession. Windows tree kill now taskkills surviving pre-scan PIDs after /T misses broken intermediate links. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli,hub): fail-loud Docker pwsh CI; scope Codex registry; stub leak CI skipped all four windowsProcessTree.pwsh tests on 48bb (image inspect without pull). Pull the image in test.yml and throw under CI when missing. Scope strict runtime-registry failure to Codex stop contexts so one bad JSON cannot block archive machine-wide. Stamp childStarted:false on machine RPC outside_workspace_roots. Add Codex reservedSessionId prealloc and runSharedCodex regression coverage for the #1911 Critical. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli,hub): single-use reservedSessionId; skip UUID HTTP hints Consume the machine-spawn reservation on first prepare() so a second Codex root/fork cannot re-adopt a consumed stub (409). Stop stamping UUID local-HTTP sessionId as --hapi-session-id (adopt hard-fail). Update stale hub spawn mocks for the reservedSessionId arity. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(hub): archive never-started machine-spawn stubs on unknown Ambiguous keep-stubs (startedBy=runner, no hostPid, still machine-spawn tagged) made StopSession return unknown forever while the runner was online, so archive threw permanently. Treat unknown + prealloc stub + no hostPid as already_gone. Also reject spawn success when the runner reports a different id than the preallocated stub. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(hub,cli): refuse dual-RPC-miss archive; generation-check Win survivors Both KillSession and StopSession missing is not proof a detached CLI exited — keep the row unconfirmed until the runner reconnects. Windows tree kill now re-checks process-start markers before individually signalling survivors so PID reuse is not killed. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli): refuse archived reopen in bootstrapExistingSession Hub-archived rows could be resurrected when a late-booting child hit --existing-session-id: buildSessionMetadata stamped lifecycleState=running and updateMetadata overwrote the archive. Adopt already rejected archived stubs; reopen now matches. Intentional revive still goes through hub reopenSession first. Correct stub-hatch comment (no post-archive argv sweep). Co-authored-by: Cursor <cursoragent@cursor.com> * fix(hub,cli): reject un-archive in updateSessionMetadata CAS Hub-side transactional guard (Discovery #1911 M1): refuse metadata writes that clear archived lifecycle unless allowUnarchive (reopen clear). Client stops infinite updateMetadata backoff when mismatch applies archived. Clear archive immediately before spawn so Pi/PTY attempt snapshots stay valid. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(hub,cli): success+preserve hub archive; M3 reservedSessionId split Replace version-mismatch refuse (livelock in CAS backoff) with success + merge-preserve of hub-archived fields unless allowUnarchive. Ack path routes through noteHubArchived so the CLI exits (criterion 6). Clear- before-spawn remains the authorized revive. Thread --hapi-session-id to reservedSessionId for shared-parser flavors (adopt≠reopen). Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli): fail-closed orphan ps signal; PID- registry; spawn timeouts Overseer B1: signalled ps (status null) is scan_failed, not empty table. B2: unreadable registry on PID-* stops returns null (no soft []). B3: 10s timeout on process/orphanReap spawn.sync. Narrow bootstrap archived belt to archivedBy=hub to match hub merge-preserve. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli,hub): hubArchived latch; PTY archiveSnapshot durability Criterion 6: noteHubArchived latches hubArchived so late registerKillSessionHandler registrations still EXIT (EventEmitter does not replay). Mirror Pi: persist archiveSnapshot on ptyResumeAttempt across clear-before-spawn and quarantine, restore on failed-resume cleanup (#1911 Discovery Majors). Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli): Codex hub-archived exit; AC6 latch test production order C2: SharedCodexRoot registers hub-archived (and latch check) beside its private KillSession — Codex never called registerKillSessionHandler. C1: keep latch+subscribe; test write-then-register order Discovery proved. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli): type Codex hub-archived end mock for tsc vi.fn(async () => {}) inferred zero-arg calls; RootHost['end'] signature unblocks end.mock.calls[0][0] (TS2493). Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli): catch hub-archived Codex host.end rejection void this.host.end() discarded archive-RPC failures as unhandled rejections; Codex has no unhandledRejection handler, so Node can exit the shared wrapper and kill sibling roots mid-turn. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com>	2026-09-25 09:15:08 +01:00
Junmo KimandGitHub	4603c883a0	fix(cli): identify the Windows runner process through CIM (#1755) ... * fix(cli): identify the Windows runner process through CIM isHapiRunnerProcess() checks that the PID persisted in runner state still belongs to a HAPI runner. On Windows it read the command line through wmic and returned true for any live PID when that spawn failed. wmic is absent on current Windows 11 builds, so that branch is the only one those hosts take and the check degrades to a liveness test. When a reboot hands the stale runner PID to another process, runner start adopts it and exits without starting a runner. Query the command line through PowerShell Get-CimInstance Win32_Process first and keep wmic as the fallback, matching getProcessStartMarker() in this file. A probe that succeeds without reporting a command line falls through to the next one, and an unreadable identity still preserves the live state, so a real runner is never discarded on a probe that came back empty. * fix(cli): keep an unverifiable runner pid out of destructive paths The identity check answered a boolean, so "alive but unidentifiable" had to collapse into one of the two verdicts. It collapsed into "this is the runner", and runner start acts on that by calling stopRunner(), which force kills the persisted pid once the HTTP stop times out. A reused pid that no probe can read was therefore still reachable by a kill. Report the identity as runner, foreign, unknown or dead instead. Only a confirmed runner is reported as running. An unknown pid is neither signalled nor cleaned up, because clearing the state would also drop the lock that keeps a second runner from starting, and the pid may still be a healthy runner behind a probe that failed transiently. The posix branch had the same fail-open through isProcessAlive and now takes the shared unknown path. * fix(cli): gate the runner force kill on a confirmed identity Two paths reach stopRunner() with a boolean that cannot carry "unverified". runner start stops an existing runner, and the spawned start-sync child reads the same answer through isRunnerRunningCurrentlyInstalledHappyVersion(), where a false result is read as a version mismatch and also calls stopRunner(). Propagating the unknown state to each caller would leave the next path open, so check the identity where the process is actually signalled: only a confirmed runner is force killed. Also stop reading WMIC's column header as a command line. wmic get CommandLine prints the header even when the property is empty or unreadable, so trimmed stdout was never empty on success and an unreadable process was classified as foreign, which cleared the state and the lock instead of preserving them. * fix(cli): gate the whole runner stop on a confirmed identity The identity check sat in front of the force kill, but stopRunner() first sends an unauthenticated POST /stop to the port recorded in runner state. An http request is a signal too: if the pid and its port were both reused, an unrelated local service receives that request before the check runs. Move the check to the start of stopRunner() so a pid that is not a confirmed runner is never contacted at all. The later force-kill check is redundant once the whole sequence is gated, so it goes away.	2026-09-06 14:19:14 +08:00
KorenKritaandGitHub	abf9cb02a5	fix(pi): resume archived sessions safely (#1308) ... * fix(pi): resume archived sessions safely * fix(pi): harden native resume startup * fix(pi): harden resume termination evidence * fix(runner): persist resume process evidence * fix(runner): track resume process generations * fix(runner): verify full session tree shutdown * fix(pi): block pre-mapping resume dedup	2026-08-02 21:15:52 +08:00
SSU-WEI HUANGandGitHub	c87720ab4d	fix(cli): load extra headers from settings (#1041) ... * test: reproduce issue #786 * fix: load extra headers from settings (closes #786) * test: cover extra header precedence and redaction * fix: redact persisted extra headers in diagnostics * test: cover runner extra header identity * fix: restart runner when extra headers change	2026-07-16 12:27:50 +08:00
KorenKritaandGitHub	b3add07ad7	fix: detect stale PID in runner state after abnormal shutdown (#931) ... * fix: verify PID belongs to hapi before treating runner as alive After OS upgrade, stale runner.state.json PID can be reused by unrelated processes. The old kill(pid, 0) check passes for any process, causing start-sync to loop with 'Runner already running' indefinitely. Now uses ps/wmic to confirm the process command line contains 'hapi' before considering the runner alive. Falls back to alive-only check if ps/wmic fails. * fix: precise runner process detection and wmic fallback * fix: add fallback for ps failure in non-Windows branch	2026-06-18 10:11:02 +08:00
3 people	a6176014fd	fix(runner): self-restart resilience under systemd / external process supervision (#814) ... * feat(runner): HAPI_DISABLE_VERSION_HANDOFF opt-out for mtime self-restart The heartbeat in cli/src/runner/run.ts triggers spawnHappyCLI(['runner','start']) + process.exit(0) when getInstalledCliMtimeMs() differs from startedWithCliMtimeMs. The same mtime guard fires in controlClient.isRunnerRunningCurrentlyInstalledHappyVersion when a fresh CLI invocation inspects the live runner. For operators who own process supervision (systemd, tmux, custom rebuild pipelines, etc.), source-file mtimes shift for reasons unrelated to npm upgrades. The clean exit defeats Restart=on-failure under systemd and leaves the machine offline. Setting HAPI_DISABLE_VERSION_HANDOFF=1 in the runner's environment now skips both checks while keeping the rest of the heartbeat (session pruning, state file persistence) intact. Default behavior is unchanged for npm consumers. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(runner): preserve original argv across self-restart and verify handoff The mtime-driven self-restart in cli/src/runner/run.ts spawned `hapi runner start` with no arguments, then process.exit(0)'d unconditionally after a 10s sleep. Two failure modes: 1. The forwarded `runner start-sync` lost the operator's --workspace-root flags (anything passed at the original invocation). Browse + spawn silently degraded to "no workspace roots". 2. If the replacement runner failed to come up at all (build was mid-flight, binary missing, etc.) the original runner still exited cleanly. Under systemd Restart=on-failure that means no runner is brought back, and the machine drops off the hub until manual intervention. Changes: - persistence.ts: add startedWithArgv?: string[] to RunnerLocallyPersistedState - run.ts: snapshot process.argv.slice(2) at startup, persist it on initial state write and on every heartbeat, replay it as the new runner's argv (default to ['runner','start-sync'] when nothing was captured) - controlClient.ts: new waitForRunnerHandoff(oldPid, {timeoutMs}) polls runner.state.json for a different live PID - run.ts: only clearInterval + process.exit(0) when handoff is confirmed. On spawn failure or 30s timeout, refresh the mtime baseline (so we don't respawn-loop on the same drift) and stay alive so the machine keeps serving. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(runner): address Codex review findings on #814 Two Major correctness fixes flagged by upstream Codex review on PR #814: (1) Stale-mtime poisoning on failed handoff (run.ts:854,867) The previous failure paths assigned startedWithCliMtimeMs = installedCliMtimeMs which the next heartbeat persisted to runner.state.json. Downstream isRunnerRunningCurrentlyInstalledHappyVersion() then reported the still-stale runner as current, masking the failure until the *next* genuine mtime change. Symptom: an mtime change that briefly failed to hand off would be silently forgotten. Fix: leave startedWithCliMtimeMs immutable. Gate handoff entry on a new nextHandoffAttemptAt timestamp; failure paths bump it by HANDOFF_RETRY_BACKOFF_MS (5 min) via deferHandoffRetry(). The heartbeat continues to write the honest "still on the old code" mtime, and the runner naturally re-attempts after the cooldown. (2) HAPI_DISABLE_VERSION_HANDOFF not honored by live runner (controlClient.ts:192, persistence.ts) The env var was only checked in the invoking CLI process. Under the documented systemd use case the env is set on the service unit but NOT on the operator's interactive shell - so a shell `hapi runner start` would still treat mtime drift as stale and kill the supervised runner during a rebuild. The exact regression this layer was built to prevent. Fix: capture HAPI_DISABLE_VERSION_HANDOFF at runner start time into state.startedWithVersionHandoffDisabled, persisted via the heartbeat. The controlClient mtime check now OR's the live env var with the persisted snapshot, so any caller honours the running runner's opt-out regardless of their own environment. Tests: cli typecheck clean; 14/14 runner unit tests pass. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(runner): address Codex #814 [Major] argv-capture + handoff race Two additional Major findings on the runner self-restart layer that were not addressed in a49fc57: 1. run.ts:672 - process.argv.slice(2) returns ['start-sync', ...] in compiled binary mode (raw argv is [hapi, runner, start-sync, ...]), so the handoff spawned `hapi start-sync ...` which resolveCommand treats as an unknown top-level and falls back to Claude. Replaced with getCliArgs() (the project's canonical argv normalizer) plus a defensive guard that falls back to ['runner', 'start-sync'] if the captured argv does not begin with 'runner'. 2. run.ts:892 - waitForRunnerHandoff did not actually keep the old runner alive. The child's startRunner() unconditionally called stopRunner() before acquiring the lock or writing its own state, so the parent's /stop handler resolved shutdown and exited BEFORE the child committed. If the child then failed (lock contention, auth error, anything between stopRunner and writeRunnerState), the machine went offline with no runner at all. New handoff protocol: - Parent sets HAPI_RUNNER_HANDOFF_FROM_PID=<pid> on the spawned child's env, then releases the lock BEFORE entering waitForRunnerHandoff (breaks the parent-holds-lock / child-needs-lock-to-write-state deadlock). - On wait-timeout the parent re-acquires the lock (long-retry, 30s) and defers retry; if re-acquire fails (third party took the lock) the parent exits cleanly so it does not stay alive without the lock invariant. - Child detects the env signal; if state.pid matches and that pid is alive, this is an authorized handoff: skip stopRunner(), skip the version-match early-exit, and acquire the lock with a longer retry window (60 attempts x 500ms) so it waits through the parent's asynchronous release. CLI typecheck clean. 14/14 runner unit tests still pass. The wider 46/664 failures in the CLI suite are pre-existing in this branch (unrelated: AppServerEventConverter, cursorEventConverter, hook server, Query) - baseline before this commit has 42+; my changes do not regress them. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com>	2026-06-10 13:59:27 +08:00
weishu	7d5a5ee919	chore: remove obsolete spawn leftovers	2026-05-21 14:25:10 +08:00
ROOOOandGitHub	f967bd9928	fix(runner): restart when hub identity changes (#303) ... * fix(runner): restart when hub identity changes * fix(runner): fail closed on missing identity	2026-03-18 10:43:36 +08:00
weishu	0228146b99	refactor: rename daemon to runner throughout codebase	2026-01-19 11:12:48 +08:00