feat(cursor): invisible sync-on-open migrator from legacy stream-json to ACP (#844)

* feat(cursor): invisible sync-on-open migrator from legacy stream-json to ACP

Closes #824

When the operator reopens a legacy stream-json Cursor session in HAPI,
the hub now transparently transplants its `~/.cursor/chats/<wsh>/<uuid>/store.db`
into `~/.cursor/acp-sessions/<uuid>/`, verifies it loads via `agent acp`,
flips `metadata.cursorSessionProtocol = 'acp'`, and removes the legacy
source - all before `resumeSession` returns. Subsequent opens are pure ACP.

The primary justification is safety, not feature parity. #784 (`cursor-agent`
fabricates `Questions skipped by the user` responses in legacy stream-json
mode) still fires regularly in dogfood despite #801's mitigation: the agent
ships destructive side effects against fabricated consent. Migration to ACP
closes the protocol-level door because the `AskQuestion` tool does not exist
on the ACP side, so there is nothing to fabricate.

working. That tradeoff was reasonable at the time. The accumulated #784
evidence makes legacy sessions actively unsafe; this PR makes the upgrade
path invisible enough that users stop avoiding it.

A pre-PR spike established that legacy and ACP `store.db` files use the
identical SQLite schema; only the directory layout differs. The migrator
therefore:

1. Sanity-checks the source store and pre-flips state (`session.active`,
   `lifecycleState`, on-disk presence, target collision)
2. Optionally archives a stale-running row (`forceArchiveRunning: true` is
   the default for the auto-migrate path because the caller already
   verified `session.active === false`)
3. Atomically creates `~/.cursor/acp-sessions/<uuid>/` with mode `0o700`
4. Copies `store.db` and chmods to `0o600` (multi-user-host hardening)
5. Writes a minimal `meta.json` sidecar (`schemaVersion`, `cwd`, optional
   `title`) with mode `0o600`
6. Spawns `agent acp` under HAPI_HOME isolation and verifies the session
   loads via `session/load`. On long histories the verify also drives a
   trivial single-turn prompt; on short ones load-only is enough
7. Flips `cursorSessionProtocol = 'acp'` AND clears the
   `cursorMigrationState` banner flag in a SINGLE metadata write
8. Removes the legacy source store (only after verify succeeded and the
   protocol flip committed). The legacy `~/.cursor/chats` parent dir is
   left as-is

Every failure leaves the legacy state intact. No `rm` fires without a
verify success AND a committed protocol flip.

The transplant takes 15-20s on long histories (copy a multi-hundred-MB
store, spawn `agent acp`, replay thousands of notifications, tear down
the probe). Without a progress indicator the wait reads as "broken" to a
fresh reviewer. A minimal banner ships alongside the migrator:

- Hub sets `metadata.cursorMigrationState = 'in_progress'` BEFORE the
  long-running transplant. The session-cache refresh emits the existing
  `session-updated` SSE event (no new event type), so the web client
  picks it up in milliseconds. No client-side polling needed.
- Hub clears the flag in the SAME metadata write that flips
  `cursorSessionProtocol` to `'acp'` on success, so the banner disappears
  in the same render tick the chat re-renders as ACP - no flicker window.
- Hub clears the flag explicitly in the auto-migrate helper's `finally`
  on failure/exception, so the banner never gets stuck if migration
  falls back to the legacy launcher.
- Web renders an accessible (role=status, aria-live=polite) banner with
  an indeterminate spinner. Deliberately no fake percentage - we do not
  have phase data and a fake progress bar would lie.

This PR is intentionally sequenced AFTER swear01's three ACP mop-up PRs
(merged today as ad038bbf, 8094b500, fa363c2f), all of which are
prerequisite for safe concurrent ACP launches.

The verify probe spawns `agent acp` directly via `AcpVerifyProbe` under
HAPI_HOME isolation (the migrator overrides `HOME` to a temp dir for the
verify pass), so it never touches `<real-HAPI_HOME>/locks/agent-acp-active/`
at all. Per swear01's #835 design note, the post-flip ACP launcher claims
the lock through the standard `registerActiveAcpTransport` entry and
behaves like any other concurrent ACP start. The migrator itself never
writes `pid` or `count` files directly.

The auto-migrate path is gated by `HAPI_CURSOR_LEGACY_AUTO_MIGRATE`. Set
to `0`, `false`, `no`, or `off` to suppress it entirely (legacy sessions
keep running through the existing stream-json launcher). Default is on.

A REST endpoint at `POST /api/sessions/:id/migrate-to-acp` allows
explicit migration of a single session outside the sync-on-open path
(e.g. for a specific cold archived session a user wants to re-engage).
Bulk migration surfaces (CLI subcommand, web button, bulk REST endpoint,
candidate-listing API) were deliberately stripped per reviewer feedback;
per-session sync-on-open + this escape hatch are the only two paths.

- 343 hub unit tests (4 new for the migration banner flag transitions,
  53 for the migrator core, 32 for the verify probe, 15 for the auto-
  migrate helper guard matrix, plus existing suites)
- 3 integration tests against a real `agent acp` (skipped by default
  unless `HAPI_CURSOR_LEGACY_MIGRATOR_INTEGRATION=1`)
- 10 web unit tests for the banner component (visibility paths + a11y)
- Typecheck clean across cli, web, hub

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(cursor): address upstream Codex review findings on #844 (2 majors)

Finding 1 (Major) — verify probe `agentLookupHome` ignored `metadata.homeDir`
in service-account hub deployments. `migrateOne` resolved the legacy store
under `metadata.homeDir` (the recorded session-owner home) but the default
createProbe factory still set `agentLookupHome` from `this.deps.homeDir()`
(the hub user's home). On a service-account hub, the store lookup
succeeded but `agent acp` discovery fell back to the hub user's
`~/.local/bin`, so verify silently failed and sync-on-open quietly fell
back to legacy.

Fix:
- Widen `CursorLegacyMigratorDeps.createProbe` signature from
  `(env) => AcpVerifyProbe` to `(env, agentLookupHome) => AcpVerifyProbe`
- Default factory uses the passed `agentLookupHome`
- `verifyInTempHome` threads `opts.sourceHome` (already the resolved
  session-owner home) through as the 2nd arg

2 new regression tests pin the contract:
- service-account case (metadata.homeDir != deps.homeDir()): captured
  agentLookupHome MUST equal metadata.homeDir
- legacy session record (no metadata.homeDir): falls back to
  deps.homeDir() correctly

Finding 2 (Major) — `bun.lock` win32-x64 pinned to 0.20.0 while
`cli/package.json` required 0.20.1 (rebase artifact from the v0.20.0 →
v0.20.1 release commit landing in upstream/main between the original
spike and the rebase). Frozen-install Windows users would either get
the wrong native binary or have the lock rejected.

Fix: regenerated bun.lock so the entry resolves
`@twsxtd/hapi-win32-x64@0.20.1`. `bun install --frozen-lockfile` now
passes clean.

Test budget:
- 391 hub unit tests pass (2 new for createProbe agentLookupHome
  contract, +0 regressions)
- Typecheck clean across cli + web + hub
- `bun install --frozen-lockfile` clean

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
HeavyGee
2026-06-10 13:57:55 +08:00
committed by GitHub
co-authored by Cursor
parent ddf3a5545b
commit 55d1bbb7bd
21 changed files with 4539 additions and 7 deletions
+338 -3
View File
@@ -8,7 +8,7 @@
*/
import { isKnownFlavor, type LocalResumeTarget, type ResumableSession } from '@hapi/protocol'
import type { SlashCommandsResponse } from '@hapi/protocol/apiTypes'
import type { CursorMigrateOutcome, CursorMigrateToAcpRequest, SlashCommandsResponse } from '@hapi/protocol/apiTypes'
import type { AgentFlavor, CodexCollaborationMode, DecryptedMessage, PermissionMode, Session, SyncEvent } from '@hapi/protocol/types'
import { unwrapRoleWrappedRecordEnvelope } from '@hapi/protocol/messages'
import type { Server } from 'socket.io'
@@ -16,6 +16,8 @@ import type { Store, CancelQueuedMessageResult } from '../store'
import type { HapiSessionExportResult } from '@hapi/protocol/sessionExport'
import type { RpcRegistry } from '../socket/rpcRegistry'
import type { SSEManager } from '../sse/sseManager'
import { CursorLegacyMigrator, type CursorLegacyMigratorOptions } from '../cursor/cursorLegacyMigrator'
import { EventPublisher, type SyncEventListener } from './eventPublisher'
import { MachineCache, type Machine } from './machineCache'
import { MessageService } from './messageService'
@@ -430,6 +432,154 @@ export class SyncEngine {
this.handleSessionEnd({ sid: sessionId, time: Date.now() })
}
/**
* Apply the post-migration metadata flip in hapi.db:
* - metadata.cursorSessionProtocol = 'acp'
* - session.model = lastUsedModel (if provided)
*
* Returns 'success' on a clean write, 'version-mismatch' if the metadata
* version moved underneath us (caller retries) or 'not-found' if the row
* is gone.
*
* Used by CursorLegacyMigrator after the on-disk transplant + verify
* succeeds. Kept on the engine (not on the migrator) so that all hapi.db
* writes funnel through the existing cache-refresh path.
*/
flipCursorSessionProtocolToAcp(
sessionId: string,
namespace: string,
lastUsedModel: string | null
): { result: 'success' | 'version-mismatch' | 'not-found' | 'session-active' } {
for (let attempt = 0; attempt < 2; attempt += 1) {
const latest = this.sessionCache.getSessionByNamespace(sessionId, namespace)
?? this.sessionCache.refreshSession(sessionId)
if (!latest?.metadata) {
return { result: 'not-found' }
}
// Combined SSE-event payload contract (UX A++): clear the
// `cursorMigrationState='in_progress'` flag in the SAME metadata
// write that flips `cursorSessionProtocol` to 'acp'. The web
// banner keys off `cursorMigrationState`, so a single SSE
// session-updated event swaps both atomically — banner gone,
// protocol flipped — preventing a flicker window where the
// banner has already disappeared but the chat hasn't re-rendered
// to the ACP transport yet.
const carriedMigrationState = latest.metadata.cursorMigrationState
// Atomic active-check inside the same synchronous flip op so
// that a resume cannot land between the migrator's recheck
// and the actual DB update. Bun is single-threaded — once
// we've read `latest` and the row is inactive, no other JS
// can mutate active=true until this method returns. Codex
// review #34 P1 v2: the migrator's recheck is best-effort;
// this is the authoritative gate.
//
// Codex review #34 P2 v5: only block on `active === true`,
// NOT on lifecycleState === 'running'. After a force-archive
// flow archiveSession() synchronously sets active=false but
// the cleanup metadata write that flips lifecycleState
// 'running' → 'archived' may still be in-flight, and that
// is OUR archive completing, not a resume race. The active
// flag is the authoritative live-runner signal.
if (latest.active === true) {
return { result: 'session-active' }
}
// Codex review #34 P2 v7: ALSO clear a stale lifecycleState
// value if it still says 'running'. The migrator now skips
// archiveSession() for stale-running rows (active=false but
// lifecycle=running with --force-archive-running) because
// there's no live runner to archive. Without this fixup,
// successfully migrated stale rows would retain lifecycle=
// running forever and any downstream code that filters by
// lifecycleState (not the cache active flag) would keep
// treating archived ACP sessions as live.
const oldLifecycle = typeof latest.metadata.lifecycleState === 'string' ? latest.metadata.lifecycleState : undefined
const nextMetadata: typeof latest.metadata = {
...latest.metadata,
cursorSessionProtocol: 'acp' as const,
...(oldLifecycle === 'running' ? { lifecycleState: 'archived' as const } : {})
}
// Drop the migration-in-progress flag in the same write (see
// header comment). Safe whether or not it was set.
if (carriedMigrationState !== undefined) {
delete nextMetadata.cursorMigrationState
}
const result = this.store.sessions.updateSessionMetadata(
sessionId,
nextMetadata,
latest.metadataVersion,
namespace,
{ touchUpdatedAt: false }
)
if (result.result === 'version-mismatch') {
this.sessionCache.refreshSession(sessionId)
continue
}
if (result.result !== 'success') {
return { result: 'not-found' }
}
this.sessionCache.refreshSession(sessionId)
if (lastUsedModel && lastUsedModel.trim().length > 0) {
this.store.sessions.setSessionModel(sessionId, lastUsedModel.trim(), namespace, { touchUpdatedAt: false })
this.sessionCache.refreshSession(sessionId)
}
return { result: 'success' }
}
return { result: 'version-mismatch' }
}
/**
* Migrate a single legacy cursor session to ACP. Hub-side; runs on the
* operator's machine (the hub host); see tiann/hapi#824 design.
* Returns a structured outcome (ok or refusal); does not throw.
*/
async migrateLegacyCursorSession(
sessionId: string,
namespace: string,
request: CursorMigrateToAcpRequest
): Promise<CursorMigrateOutcome> {
const session = this.sessionCache.getSessionByNamespace(sessionId, namespace)
?? this.sessionCache.refreshSession(sessionId)
if (!session) {
return { ok: false, sessionId, reason: 'internal_error', message: 'session not found in namespace', durationMs: 0 }
}
const migrator = this.buildMigratorForRequest(request)
return migrator.migrateOne(session, {
keepSource: request.keepSource,
forceArchiveRunning: request.forceArchiveRunning,
skipVerify: request.skipVerify
})
}
private buildMigratorForRequest(_request: CursorMigrateToAcpRequest): CursorLegacyMigrator {
const migratorOpts: CursorLegacyMigratorOptions = {}
return new CursorLegacyMigrator(migratorOpts, {
archiveSession: async (sessionId) => {
await this.archiveSession(sessionId)
},
// NOTE: no awaitSessionInactive injection — handleSessionEnd()
// synchronously sets cache.active=false inside archiveSession,
// so any cache-based poll would return immediately and provide
// false reassurance. The migrator now relies on
// awaitLockRelease's minimum-dwell + SQLite busy-probe +
// size-stability combination instead. Codex review #34 P1 v3.
getCurrentSession: (sessionId, namespace) => {
const s = this.sessionCache.getSessionByNamespace(sessionId, namespace)
if (!s) return null
return {
active: s.active === true,
lifecycleState: typeof s.metadata?.lifecycleState === 'string' ? s.metadata.lifecycleState : undefined,
cursorSessionProtocol: typeof s.metadata?.cursorSessionProtocol === 'string' ? s.metadata.cursorSessionProtocol : undefined
}
},
updateSessionAfterMigrate: (sessionId, namespace, lastUsedModel) => {
const result = this.flipCursorSessionProtocolToAcp(sessionId, namespace, lastUsedModel)
if (result.result === 'success') return { ok: true }
if (result.result === 'session-active') return { ok: false, reason: 'session_active' as const }
return { ok: false, reason: 'version_mismatch_or_missing' as const }
}
})
}
async switchSession(sessionId: string, to: 'remote' | 'local'): Promise<void> {
await this.rpcGateway.switchSession(sessionId, to)
}
@@ -633,6 +783,183 @@ export class SyncEngine {
return undefined
}
/**
* tiann/hapi#824 — sync-on-open auto-migration. Returns the (possibly
* refreshed-from-cache) session. If the session is a legacy stream-json
* Cursor session AND the env flag is on, attempts a transplant migration
* synchronously before the caller spawns the runner.
*
* The migrator's verify probe runs in an isolated HAPI_HOME (see
* verifyInTempHome), so this method is safe to call even when other ACP
* transports are alive on the host: per tiann/hapi#832, two `agent acp`
* processes coexist on the same host without conflict, and swear01's
* tiann/hapi#835 refactors the agent-acp-active lock into a cross-process
* refcount that explicitly supports this. We rely on #835 landing before
* this PR — see manifest layer ordering and the dependency note in
* PR #34's body.
*
* Failure modes are all soft — the session is returned unchanged and the
* caller proceeds with the legacy launcher.
*/
private async maybeAutoMigrateLegacyCursorSession(session: Session, namespace: string): Promise<Session> {
const md = session.metadata
const flagRaw = process.env.HAPI_CURSOR_LEGACY_AUTO_MIGRATE?.trim().toLowerCase() ?? ''
console.info('[auto-migrate] considering', {
sessionId: session.id,
flavor: md?.flavor ?? null,
proto: md?.cursorSessionProtocol ?? null,
hasCursorId: typeof md?.cursorSessionId === 'string' && md.cursorSessionId.length > 0,
envFlag: flagRaw === '' ? '(unset; default on)' : flagRaw
})
if (flagRaw === '0' || flagRaw === 'false' || flagRaw === 'no' || flagRaw === 'off') {
console.info('[auto-migrate] skipped: env flag disabled', { sessionId: session.id })
return session
}
if (!md || md.flavor !== 'cursor') {
console.info('[auto-migrate] skipped: not a cursor session', { sessionId: session.id, flavor: md?.flavor ?? null })
return session
}
if (md.cursorSessionProtocol === 'acp') {
console.info('[auto-migrate] skipped: already ACP', { sessionId: session.id })
return session
}
if (typeof md.cursorSessionId !== 'string' || md.cursorSessionId.length === 0) {
console.info('[auto-migrate] skipped: no cursorSessionId', { sessionId: session.id })
return session
}
console.info('[auto-migrate] starting transplant', { sessionId: session.id, cursorSessionId: md.cursorSessionId })
// UX A++: surface the migration to the user via a banner in the
// web UI. We set `cursorMigrationState='in_progress'` on the row
// BEFORE the long-running transplant; the sessionCache.refresh()
// call emits a `session-updated` SSE event the web client uses to
// render the banner. The flag is cleared on success by the same
// metadata write that flips cursorSessionProtocol to 'acp' (see
// flipCursorSessionProtocolToAcp) so the banner disappears in the
// same render tick the chat re-renders as ACP — no flicker. On
// failure we clear the flag explicitly in the catch path below.
const flagSet = this.setCursorMigrationStateInProgress(session.id, namespace)
let bannerCleanupNeeded = flagSet
try {
const migrator = this.buildMigratorForRequest({})
// Codex #34 P2 (round 13): for inactive rows whose metadata
// still reads `lifecycleState === 'running'` (e.g. orphaned by
// a hub crash where the lifecycle transition didn't land), the
// migrator's preflight refuses with `running_refused` unless
// `forceArchiveRunning` is true. resumeSession's caller-side
// guard already ensured `session.active === false` (see the
// early-return above), so we know there's no runner to yank;
// the `running` lifecycle is stale metadata, not a live agent.
// This is exactly the stale-row case the sync-on-open path is
// meant to clean up — refusing here would defeat the whole
// point and silently fall back to the legacy launcher forever.
const outcome = await migrator.migrateOne(session, { forceArchiveRunning: true })
if (outcome.ok) {
console.info('[auto-migrate] success', {
sessionId: session.id,
cursorSessionId: md.cursorSessionId,
acpSessionId: outcome.acpSessionId,
durationMs: outcome.durationMs,
sourceRemoved: outcome.sourceRemoved,
replayNotifications: outcome.replayNotifications,
lastUsedModelPreserved: outcome.lastUsedModelPreserved
})
// Successful migration already cleared the flag atomically
// in flipCursorSessionProtocolToAcp; skip the cleanup write.
bannerCleanupNeeded = false
const refreshed = this.sessionCache.getSessionByNamespace(session.id, namespace)
if (refreshed) return refreshed
return session
}
// Soft fail — log and let the legacy launcher handle it.
console.info('[auto-migrate] legacy cursor session left as stream-json', {
sessionId: session.id,
reason: outcome.reason,
message: outcome.message
})
} catch (err) {
console.warn('[auto-migrate] unexpected error; falling back to legacy launcher', {
sessionId: session.id,
err: err instanceof Error ? err.message : String(err)
})
} finally {
// Failure or exception path: clear the in-progress banner flag
// so the user isn't left with a permanent "Upgrading..." banner
// even though we silently fell back to the legacy launcher.
if (bannerCleanupNeeded) {
this.clearCursorMigrationState(session.id, namespace)
}
}
return session
}
/**
* Set `metadata.cursorMigrationState='in_progress'` on the session row
* with a single retry on version-mismatch. Returns true if the flag was
* persisted (so the caller knows the finally-cleanup is required), false
* if the write failed entirely — in which case the banner never appeared
* and there's nothing to clean up. UX A++ helper for the auto-migrate
* banner; see maybeAutoMigrateLegacyCursorSession.
*/
private setCursorMigrationStateInProgress(sessionId: string, namespace: string): boolean {
for (let attempt = 0; attempt < 2; attempt += 1) {
const latest = this.sessionCache.getSessionByNamespace(sessionId, namespace)
?? this.sessionCache.refreshSession(sessionId)
if (!latest?.metadata) return false
if (latest.metadata.cursorMigrationState === 'in_progress') return true
const nextMetadata = { ...latest.metadata, cursorMigrationState: 'in_progress' as const }
const result = this.store.sessions.updateSessionMetadata(
sessionId,
nextMetadata,
latest.metadataVersion,
namespace,
{ touchUpdatedAt: false }
)
if (result.result === 'success') {
this.sessionCache.refreshSession(sessionId)
return true
}
if (result.result === 'version-mismatch') {
this.sessionCache.refreshSession(sessionId)
continue
}
return false
}
return false
}
/**
* Clear `metadata.cursorMigrationState` (failure / exception cleanup).
* Idempotent; safe to call when the flag was never set. UX A++ helper.
*/
private clearCursorMigrationState(sessionId: string, namespace: string): void {
for (let attempt = 0; attempt < 2; attempt += 1) {
const latest = this.sessionCache.getSessionByNamespace(sessionId, namespace)
?? this.sessionCache.refreshSession(sessionId)
if (!latest?.metadata) return
if (latest.metadata.cursorMigrationState === undefined) return
const nextMetadata: typeof latest.metadata = { ...latest.metadata }
delete nextMetadata.cursorMigrationState
const result = this.store.sessions.updateSessionMetadata(
sessionId,
nextMetadata,
latest.metadataVersion,
namespace,
{ touchUpdatedAt: false }
)
if (result.result === 'success') {
this.sessionCache.refreshSession(sessionId)
return
}
if (result.result === 'version-mismatch') {
this.sessionCache.refreshSession(sessionId)
continue
}
return
}
}
/** Inactive session with directory path but no agent thread and no prior user turn. */
private canFreshSpawnNeverStartedSession(session: Session, sessionId: string, namespace: string): boolean {
const metadata = session.metadata
@@ -655,11 +982,19 @@ export class SyncEngine {
}
}
const session = access.session
if (session.active) {
const initialSession = access.session
if (initialSession.active) {
return { type: 'success', sessionId: access.sessionId }
}
// tiann/hapi#824 — invisible, automatic, per-session ACP migration on
// first open. If this is a legacy stream-json Cursor session and we
// can safely migrate it right now (no other agent acp transport
// would block the post-migration ACP launcher), run the transplant
// synchronously before resuming. The user sees the regular session
// loading state for ~35s longer; the session opens as ACP.
const session = await this.maybeAutoMigrateLegacyCursorSession(initialSession, namespace)
const targetResult = this.resolveLocalResumeTarget(access.sessionId, namespace)
let flavor: AgentFlavor
let resumeToken: string | undefined