mirror of
https://github.com/wu736139669/hapi.git
synced 2026-11-07 01:21:37 +00:00
* feat(cli): report the CLI process start marker in session metadata The session webhook only names hostPid. A PID alone does not tell the runner which process generation reported it: once the CLI exits, the OS can hand the same PID to any other process, including another runner-spawned CLI. Add metadata.hostStartMarker, the reporting process's own start marker (getProcessStartMarker(process.pid), the same probe the runner uses for its resume records and the stop-session generation check). buildSessionMetadata is the only place that sets hostPid, so every bootstrap path (fresh, existing, lazy, shared Codex) reports it. The runner uses it to decide whether an untracked runner webhook may be adopted (decideUntrackedRunnerWebhook). * fix(runner): adopt untracked runner webhooks only from the reporting process generation Since #1911 the runner adopts an untracked `startedBy: 'runner'` webhook (runner restart before the webhook; Claude has no HAPI id on argv) and persists getProcessStartMarker(pid) of whatever currently owns hostPid, so that a later stopSession can tree-kill that generation. If the webhook is late - the reporting CLI has exited and the OS has already reused its PID - the persisted marker belongs to a foreign process, possibly another runner-spawned CLI. The generation check in stopSession then passes and the foreign process tree is killed. The webhook now has to prove the generation itself: the CLI reports its own marker as metadata.hostStartMarker and decideUntrackedRunnerWebhook adopts only when it matches the live PID. A marker that names another generation, or a failed probe, is ignored without killing anything, also on the webhook-timeout path: a proven mismatch means the PID is not the ghost. A webhook without a marker (older CLI) is left untracked instead of adopted blindly; the timeout kill for those CLIs is unchanged. The integration test drives a real runner over its control server: a webhook carrying the live marker is adopted, listed and stoppable; a webhook with a stale or missing marker leaves the PID untracked and alive, and stop-session reports unknown. Follow-up to #1911 (#1910). * fix(cli): probe the process start marker only for runner-started sessions buildSessionMetadata probed getProcessStartMarker(process.pid) on every bootstrap, terminal starts included. The runner reads metadata.hostStartMarker only in the untracked `startedBy: 'runner'` path (decideUntrackedRunnerWebhook); a terminal webhook never consumes it, and neither do hub or web. The probe is a synchronous spawn: `ps` on POSIX, `tasklist` plus a PowerShell Get-CimInstance call on Windows, each with a 10 s timeout. A terminal `hapi` start on Windows therefore blocked on a PowerShell cold start before the session row existed, for a value nobody reads. With shared Codex the probe also ran per /new and /fork on the event loop shared by all sibling roots. Set hostStartMarker only when startedBy is 'runner'; that is the one case the runner evaluates. This narrows the earlier commit: every bootstrap path reports the marker for runner-started sessions, not for terminal ones. * fix(codex): keep hostStartMarker in the shared runtime's runner webhook The shared Codex runtime bootstraps its roots with reportStarted: false and reports them to the runner itself, from root.session.getMetadata(). That is the hub's view of the metadata: the hub parses stored metadata through MetadataSchema, so a hub that predates hostStartMarker strips the field and the webhook arrives without it. The hub's update-metadata merge never returns it either, so the loss lasts for the root's lifetime. With the generation check, an untracked runner-started webhook without a marker is ignored instead of adopted. A runner restart (for example the version handoff of a single-host upgrade, where the hub process keeps running the old code) followed by /new or /fork in a long-lived shared root therefore left the root unlisted and untracked, the very situation the adoption from #1911 addresses. The Claude and lazy bootstrap paths are unaffected: they report the local metadata. Take the marker from the local bootstrap metadata instead. The opt-in runtime integration test models the stripped hub view in its session mock and asserts the reported payload still carries the marker. The mock also gains the `on` subscription that SharedCodexRoot requires since #1911, without which the suite's bind test could not run. * test(runner): run the late-webhook suite in the serial integration project lateRunnerWebhook.integration.test.ts starts a real runner and a detached child process per case, but the default vitest project picked it up and `bun run test` ran it in a parallel worker. Tests that spawn runner process trees belong to the serial integration project (vitest.integration.config.ts, CI job `integration`), as the config comments and .github/workflows/test.yml require. Include the file in the integration project, exclude it from the default one, and remove the temporary HAPI_HOME of each case, which was left behind in the temp directory. --------- Co-authored-by: Benedict Groppe <195190800+BGGBTAC@users.noreply.github.com>
148 lines
6.9 KiB
TypeScript
148 lines
6.9 KiB
TypeScript
import { describe, expect, it } from 'vitest'
|
|
import { spawn, type ChildProcess } from 'node:child_process'
|
|
import { randomUUID } from 'node:crypto'
|
|
import { mkdtemp, readFile, rm } from 'node:fs/promises'
|
|
import { tmpdir } from 'node:os'
|
|
import { join, resolve } from 'node:path'
|
|
import { getProcessStartMarker } from '@/utils/process'
|
|
import { TEST_OWNED_MARKER_KEY } from '@/test/integrationEnv'
|
|
|
|
/**
|
|
* Drives a real runner over its control server with a synthetic child that
|
|
* stands in for a CLI whose webhook arrives while the runner has no tracking
|
|
* entry for its PID (runner restart, or a webhook that outlived its sender).
|
|
* Adoption persists the PID's start marker for the stopSession generation
|
|
* check, so the webhook must prove it comes from the process that owns the
|
|
* PID right now (decideUntrackedRunnerWebhook). Start markers come from
|
|
* `ps`, so the suite is POSIX only, like the shared Codex full stack test.
|
|
*/
|
|
|
|
type RunnerState = { pid: number; httpPort: number }
|
|
type ResumeRecord = { requestedSessionId: string; confirmedSessionId?: string; pid: number; processStartMarker: string }
|
|
|
|
async function until<T>(probe: () => Promise<T | undefined>, label: string, timeoutMs = 20_000): Promise<T> {
|
|
const deadline = Date.now() + timeoutMs
|
|
while (Date.now() < deadline) {
|
|
const value = await probe()
|
|
if (value !== undefined) return value
|
|
await new Promise(resolve => setTimeout(resolve, 100))
|
|
}
|
|
throw new Error(`Timed out: ${label}`)
|
|
}
|
|
|
|
function exited(child: ChildProcess): boolean {
|
|
return child.exitCode !== null || child.signalCode !== null
|
|
}
|
|
|
|
async function stop(child: ChildProcess | undefined): Promise<void> {
|
|
if (!child || exited(child)) return
|
|
child.kill('SIGTERM')
|
|
try {
|
|
await until(async () => exited(child) ? true : undefined, `PID ${child.pid} shutdown`, 5_000)
|
|
} catch {
|
|
child.kill('SIGKILL')
|
|
}
|
|
}
|
|
|
|
/** Every process carries the run marker so the globalSetup sweep reaps leaks. */
|
|
function testEnv(overrides: NodeJS.ProcessEnv = {}): NodeJS.ProcessEnv {
|
|
return { ...process.env, [TEST_OWNED_MARKER_KEY]: process.env.HAPI_HOME, ...overrides }
|
|
}
|
|
|
|
/** Long-lived stand-in for a CLI process; its own group so tree kills stay local. */
|
|
function spawnIdleChild(): ChildProcess {
|
|
const child = spawn(process.env.HAPI_BUN_EXEC!, ['--eval', 'setInterval(() => {}, 1000)'], {
|
|
env: testEnv(), stdio: 'ignore', detached: true,
|
|
})
|
|
expect(child.pid).toBeDefined()
|
|
return child
|
|
}
|
|
|
|
/** Isolated HAPI_HOME: own state file, lock and resume records; same temp hub. */
|
|
function startRunner(home: string): ChildProcess {
|
|
return spawn(process.env.HAPI_BUN_EXEC!, ['--cwd', resolve('.'), resolve('src/index.ts'), 'runner', 'start-sync'], {
|
|
env: testEnv({ HAPI_HOME: home, HAPI_DISABLE_VERSION_HANDOFF: '1' }), stdio: 'ignore',
|
|
})
|
|
}
|
|
|
|
async function runnerReady(home: string, runner: ChildProcess): Promise<RunnerState> {
|
|
return until(async () => {
|
|
try {
|
|
const state = JSON.parse(await readFile(join(home, 'runner.state.json'), 'utf8')) as Partial<RunnerState>
|
|
return state.pid === runner.pid && state.httpPort ? state as RunnerState : undefined
|
|
} catch {
|
|
return undefined
|
|
}
|
|
}, 'isolated runner ready')
|
|
}
|
|
|
|
function control(state: RunnerState) {
|
|
return async (path: string, body: unknown = {}): Promise<Record<string, unknown>> => {
|
|
const response = await fetch(`http://127.0.0.1:${state.httpPort}${path}`, {
|
|
method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify(body),
|
|
})
|
|
expect(response.status).toBe(200)
|
|
return await response.json() as Record<string, unknown>
|
|
}
|
|
}
|
|
|
|
async function readResumeRecords(home: string): Promise<ResumeRecord[]> {
|
|
return JSON.parse(await readFile(join(home, 'runner.state.json.resume-processes.json'), 'utf8')) as ResumeRecord[]
|
|
}
|
|
|
|
describe.skipIf(process.platform === 'win32')('untracked runner webhook adoption', () => {
|
|
it('adopts a webhook that carries the live process start marker, so the child is listed and stoppable', async () => {
|
|
const sessionId = randomUUID()
|
|
const home = await mkdtemp(join(tmpdir(), 'hapi-late-webhook-adopt-'))
|
|
// A child of the previous runner generation: nothing on disk names it.
|
|
const child = spawnIdleChild()
|
|
const runner = startRunner(home)
|
|
try {
|
|
const post = control(await runnerReady(home, runner))
|
|
const hostStartMarker = getProcessStartMarker(child.pid!)
|
|
expect(hostStartMarker).not.toBeNull()
|
|
await post('/session-started', { sessionId, metadata: { hostPid: child.pid, hostStartMarker, startedBy: 'runner' } })
|
|
expect(exited(child)).toBe(false)
|
|
expect(await post('/list')).toMatchObject({ children: [{ happySessionId: sessionId, pid: child.pid, startedBy: 'runner' }] })
|
|
expect(await readResumeRecords(home)).toMatchObject([
|
|
{ pid: child.pid, requestedSessionId: sessionId, confirmedSessionId: sessionId, processStartMarker: hostStartMarker },
|
|
])
|
|
expect(await post('/stop-session', { sessionId })).toMatchObject({ status: 'stopped' })
|
|
await until(async () => exited(child) ? true : undefined, 'adopted child stopped')
|
|
} finally {
|
|
await stop(runner)
|
|
await stop(child)
|
|
await rm(home, { recursive: true, force: true })
|
|
}
|
|
}, 30_000)
|
|
|
|
it('leaves a webhook alone that cannot prove the process generation, and never kills the PID', async () => {
|
|
const sessionId = randomUUID()
|
|
const home = await mkdtemp(join(tmpdir(), 'hapi-late-webhook-reused-pid-'))
|
|
// Stands in for the process the OS handed the PID to after the
|
|
// reporting CLI exited: the webhook names a generation that is gone,
|
|
// or (older CLI) no generation at all.
|
|
const bystander = spawnIdleChild()
|
|
const runner = startRunner(home)
|
|
try {
|
|
const post = control(await runnerReady(home, runner))
|
|
for (const metadata of [
|
|
{ hostPid: bystander.pid, hostStartMarker: 'Thu Jan 1 00:00:00 1970', startedBy: 'runner' },
|
|
{ hostPid: bystander.pid, startedBy: 'runner' },
|
|
]) {
|
|
await post('/session-started', { sessionId, metadata })
|
|
expect(await post('/list')).toMatchObject({ children: [] })
|
|
expect(await readResumeRecords(home)).toEqual([])
|
|
expect(await post('/stop-session', { sessionId })).toMatchObject({ status: 'unknown' })
|
|
// Killing is escalated asynchronously; give a wrongful kill time to land.
|
|
await new Promise(resolve => setTimeout(resolve, 300))
|
|
expect(exited(bystander)).toBe(false)
|
|
}
|
|
} finally {
|
|
await stop(runner)
|
|
await stop(bystander)
|
|
await rm(home, { recursive: true, force: true })
|
|
}
|
|
}, 30_000)
|
|
})
|