mirror of
https://github.com/wu736139669/hapi.git
synced 2026-08-05 06:24:37 +00:00
8e34e7599b
* perf(hub,web): emit structured patches for session todos/teamState/metadata/agentState writes (#895, closes second half of #884) Today the four CLI handlers in `sessionHandlers.ts` that write session-scoped state (TodoWrite messages -> setSessionTodos; team-state deltas -> setSessionTeamState; update-metadata RPC; update-state RPC) emit `session-updated` events with no `data` payload. `syncEngine.handleRealtimeEvent` intercepts each one, re-reads the row from SQLite, and broadcasts the entire ~5KB Session via SSE. That works (the web client's `isSessionRecord` shortcut keeps the cache patched), but it costs a DB read and a full-payload SSE fan-out per write, and any failure mode that drops the broadcast data falls through to `useSSE.ts:509-512` and triggers per-session REST refetches - the storm vector documented in #884. This is the architectural follow-up to #885. #885 added `staleTime` on the detail query (eliminates focus / mount refetches inside a 30s window). This PR removes the structural reason these four writes touch the REST path at all. `SessionPatchSchema` learns four optional structured fields: - `todos` (array) - `teamState` (object) - `metadata` (versioned `{ version, value }` wrapper) - `agentState` (versioned `{ version, value }` wrapper) `.strict()` preserved so unknown keys still throw. The versioned wrappers mirror the existing socket.io `update-session` broadcast at lines 211 / 259 so metadata and agentState always travel as an atomic (version, value) pair - caches need the version to reject stale patches. Each of the four emit-sites now carries a structured `data` payload with the delta it just wrote. `syncEngine.handleRealtimeEvent` for `session-updated` events with non-empty patch data: applies the patch to the in-memory Session in place via the new `sessionCache.applySessionPatch`, then forwards the event as-is. Empty patches, no-data events, and patches against uncached sessions all fall back to the legacy `refreshSession` path so behavior for other emitters (e.g. `cursor/codexDesktop.ts`) is unchanged. Dedup hook against agent-session-id changes preserved on the fast path. `patchSessionDetail` is no longer a blanket spread - it enumerates each field explicitly so the versioned metadata / agentState patches can be unwrapped into the Session's flat (metadata, metadataVersion) and (agentState, agentStateVersion) pairs. Spreading the patch wholesale would have written a `{ version, value }` object into `session.metadata` and corrupted the cache. `patchSessionSummary` recomputes the touched derivations - `todoProgress` from todos, `pendingRequestsCount` / `pendingRequestKinds` from agentState, SessionSummaryMetadata from metadata - via three new pure helpers exposed from `shared/src/sessionSummary.ts` (`computeTodoProgress`, `computePendingRequestKinds`, `toSessionSummaryMetadata`). `toSessionSummary` is refactored to use these helpers - identical output, single source of truth. - shared: `SessionPatchSchema` parses each new patch shape, stays strict, rejects empty metadata without `version`, rejects full Session payloads (those go through `isSessionRecord`). - shared: summary derivation helpers covered against bare AgentState / Metadata inputs (the shape the SSE patch path provides). - hub: each emit-site asserted to carry the expected structured payload. - hub: `applySessionPatch` unit-tests cover todos / metadata / agentState application, empty-patch rejection (forces caller back to refreshSession), cross-namespace guard, and missing-session fallback. Empirical wire round-trip verifies each patch shape survives `JSON.stringify` intact and routes the web client through `getSessionPatch` (non-empty result) instead of the REST invalidation fallback. Per #884 expectation: with this fix on top of #885, idle GET /api/sessions/<id> rate is expected to drop to near-zero on the reporter's 100+ session install. Operator (heavygee) will attach the live-measured before / after to the PR post-merge. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(hub): snapshot metadata reference before applySessionPatch mutation so dedup-on-id-change fires The structured-patch fast path added in 05147a6a (closes #884 second half) broke the dedup-on-metadata-change trigger in handleRealtimeEvent. Root cause: applySessionPatch MUTATES the cached Session in place (reassigns session.metadata = patch.metadata.value). The dedup check compares before vs after agent session IDs, but `before = getSession(id)` and `after = getSession(id)` returned the SAME object reference, so before.metadata had already been overwritten by the time the check ran. hasSameAgentSessionIds always returned true and dedup silently never fired on the fast path. The legacy refreshSession path got dedup for free because it REPLACES the cache map entry with a new Session object, leaving the pre-refresh reference intact for the comparator. Fix: capture beforeMetadata before applySessionPatch runs; use it for both branches so the comparison contract is identical. Adds syncEngineHandleRealtimeEvent.test.ts with three regression guards: - structured metadata patch with changed cursorSessionId fires dedup - todos-only patch does NOT fire dedup (no false positives) - legacy refresh path (no patch data) still fires dedup Co-authored-by: Cursor <cursoragent@cursor.com> * fix(schemas): reorder SessionPatchSchema fields so soup-merge with codex-usage layer conflicts cleanly Pure reorder (no semantic change). feat/codex-usage-indicator-rebased adds a flat `metadata: MetadataSchema.nullable().optional()` + `metadataVersion` to SessionPatchSchema in the same line range upstream/main has the model/ modelReasoningEffort fields. My branch added the versioned `metadata` field at the END of the object, so git 3-way merge silently auto-merged both, producing an invalid object literal with duplicate `metadata` keys. By placing my `metadata` / `agentState` / `todos` / `teamState` insertions in the SAME line range codex inserts (between updatedAt and model), git now raises an explicit CONFLICT during the soup merge, which can be resolved correctly once and replayed by rerere. No behavior change on a clean upstream/main merge. Pure cosmetic; no test or runtime impact. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sse): propagate TeamDelete clear through structured patch path Closes PR #897 Major review (HAPI Bot, 2026-06-13): TeamDelete events drove `applyTeamStateDelta` to return `null`, but the emit-site coalesced that to `undefined`. JSON serialization then dropped the key, the hub cache skipped its assignment branch (`patch.teamState !== undefined` was false), and the web client saw an empty patch and fell back to REST invalidation — exactly the storm path this PR was supposed to close. Sidebar / NotificationHub / dedup all served stale team state until the next full refresh. Fix in four coordinated places (wire ↔ cache contract): - shared/src/schemas.ts: `teamState: TeamStateSchema.nullable().optional()` so `null` is a valid wire shape meaning "cleared". Comment documents the discriminator contract for consumers. - hub/src/socket/handlers/cli/sessionHandlers.ts: drop the `?? undefined` coalesce so `null` survives JSON serialization. - hub/src/sync/sessionCache.ts (applySessionPatch): use `Object.prototype.hasOwnProperty.call(patch, 'teamState')` to discriminate "field absent" from "field is null", then map null → undefined to match the cached `Session.teamState` type. - web/src/hooks/useSSE.ts (patchSessionDetail): same hasOwnProperty discriminator + null → undefined mapping. Regression tests: - schemas.sessionPatch.test.ts: `{ teamState: null }` parses successfully (locks the wire contract). - sessionCache.applySessionPatch.test.ts: TeamDelete clears cached teamState; todos-only patch leaves teamState untouched (guards the hasOwnProperty branch against a regression back to `!== undefined`). Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sse): version-gate metadata/agentState patches against cache regression Closes PR #897 follow-up Major review (HAPI Bot, 2026-06-16): the new structured SSE patch path unwraps versioned metadata/agentState fields without checking the cached metadataVersion/agentStateVersion. SSE reconnects + the existing per-query invalidation can leave a detail cache repopulated by a fresh REST refetch BEFORE a buffered older patch replays. Without the gate the older patch overwrites the newer cache, regressing resume / session-id / pending-requests state. Mirrors the hub-side CLI room handler contract (`incoming.version > currentVersion`, `web/src/hooks/useSSE.ts`): - `patchSessionDetail`: gate metadata/agentState assignment behind `isNewerVersionedPatch(patch.version, nextSession.<field>Version)`. The pre-patch version is captured by `{ ...previous.session }` so the comparison is against the cache-at-write-time. - `patchSessionSummary`: read the detail cache (via queryClient) for the canonical metadataVersion / agentStateVersion. Use `>=` (not `>`) because the callsite runs `patchSessionDetail` first — when detail accepts a newer patch the cache already holds the new version, so matching `>=` keeps summary aligned with detail's acceptance; when detail rejects, `>=` aligns summary with detail's rejection. - Exported `isNewerVersionedPatch(patchVersion, currentVersion)` as a pure helper so the rule is unit-testable in isolation. - Test: `useSSE.test.ts` pins the 4 cases (newer ✓ / older ✗ / same-version ✗ / first-write currentVersion=0 ✓). Hub-side `applySessionPatch` does NOT need the same gate: in-process events from `handleUpdateMetadata` / `handleUpdateState` are emitted only AFTER the optimistic-concurrency check at the store layer succeeds, and `syncEngine.handleRealtimeEvent` consumes them synchronously in order. The vulnerability is the SSE reconnect/replay window on the web client. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sse): include updatedAt in structured patches + pendingRequests summary Closes PR #897 post-rebase bot review (HAPI Bot, 2026-06-18): Major — structured patches dropped session.updatedAt. TodoWrite, teamState, metadata, and agentState DB writes all touch sessions.updated_at, but the fast path forwarded only field deltas. Hub/web caches and session list ordering stayed stale until a full refresh. All four emit-sites in sessionHandlers now reload the stored row after a successful write and include updatedAt in the SSE patch payload (applySessionPatch already applies it via Math.max). Minor — agentState summary patches updated pendingRequestsCount/kinds but left pendingRequests stale, so SessionAttentionIndicator tooltips showed old request tools after an SSE patch. patchSessionSummary now uses computePendingRequestsCount + computePendingRequests alongside the existing kinds helper. Tests: sessionHandlers.test.ts asserts updatedAt on todos/metadata/agentState patches. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(web,hub): apply serviceTier in structured session patch path Closes PR #897 bot Minor (2026-06-18): field-by-field patchSessionDetail stopped copying serviceTier after the spread refactor, so Codex Fast/ Standard could show stale tier until a full refetch. Mirror nullable hasOwnProperty handling in patchSessionDetail and hub applySessionPatch. Co-authored-by: Cursor <cursoragent@cursor.com> * test(hub): allow same-ms updatedAt on structured patch emit asserts Date.now() resolution makes create+update land on the same millisecond in unit tests; the store still touches updated_at. Use >= so CI is not flaky. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(web): refuse versioned summary SSE patches without detail version source When session detail is not cached, defaulting metadata/agentState versions to 0 let stale buffered patches overwrite a freshly refetched list and suppress list invalidation. Bail out so the list refetches instead. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(web): keep updatedAt monotonic when applying SSE session patches Stale versioned metadata/agentState replays can still carry an older updatedAt. Use Math.max on detail and summary paths so rejected replays cannot rewind list/detail clocks while patched=true suppresses invalidation. Co-authored-by: Cursor <cursoragent@cursor.com> * chore: retrigger Codex PR review after infra stream failure Prior pr-review run died on reconnect (stream closed before response.completed); no code findings. Empty commit to re-fire pull_request_target. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(web): compare all summary metadata fields in keep-alive skip isRenderIrrelevantPatch omitted path/machineId/flavor/worktree, so a same-ms metadata patch could be dropped while summaryPatched stayed true and list invalidation never repaired grouping/icon/path. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sse): version-wrap todos/teamState patches for dual-SSE races Global + session EventSources can deliver out of order. Carry store todos_updated_at / team_state_updated_at as patch versions, gate web applies, and tighten keep-alive skip compares (metadata + request tool/kind). Co-authored-by: Cursor <cursoragent@cursor.com> * fix(web): put SSE version watermarks on SessionSummary Requiring a detail query to apply versioned list patches forced O(N) /sessions invalidation on every global SSE write. Gate against summary watermarks instead; skip no-op detail clones on duplicate deliveries. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(hub): ratchet todosUpdatedAt on rewind rebuild replaceSessionTodos was stamping the remaining TodoWrite's older createdAt, so a lagged pre-rewind structured SSE patch could resurrect deleted todos. Advance the watermark on force-replace instead. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(web): apply copilotAgentMode in structured detail SSE patches Field-by-field detail mapper dropped the new Copilot keep-alive field, so detailPatched suppressed invalidation and SessionChat kept a stale mode. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Debian <heavygee@oos-linux.in.lockhouse>
hapi-hub
Telegram bot + HTTP API + realtime updates for hapi hub.
What it does
- Telegram bot for notifications and the Mini App entrypoint.
- HTTP API for sessions, messages, permissions, machines, and files.
- Server-Sent Events stream for live updates in the web app.
- Socket.IO channel for CLI connections.
- Serves the web app from
web/distor embedded assets in the single binary. - Persists state in SQLite.
Configuration
See src/configuration.ts for all options.
Required
CLI_API_TOKEN- Base shared secret used by CLI and web login. Clients append:<namespace>for isolation. Auto-generated on first run if not set.
Optional (Telegram)
TELEGRAM_BOT_TOKEN- Token from @BotFather.HAPI_PUBLIC_URL- Public HTTPS URL for Telegram Mini App access. Also used to derive default CORS origins for the web app.
Optional (Voice)
ELEVENLABS_API_KEY- ElevenLabs API key for voice assistant.ELEVENLABS_AGENT_ID- Custom ElevenLabs agent ID (auto-created if not set).OPENAI_API_KEY- OpenAI dictation (gpt-transcribe/gpt-live-transcribe).DEEPGRAM_API_KEY- Deepgram dictation (nova-3, standard and realtime).GROQ_API_KEY- Groq dictation (whisper-large-v3).TRANSCRIPTION_BASE_URLandTRANSCRIPTION_MODEL- OpenAI-compatible/local transcription endpoint and model.TRANSCRIPTION_API_KEY- Optional bearer token for the OpenAI-compatible endpoint.
Optional
HAPI_LISTEN_HOST- HTTP bind address (default: 127.0.0.1).HAPI_LISTEN_PORT- HTTP port (default: 3006).CORS_ORIGINS- Comma-separated origins, or*.HAPI_HOME- Data directory (default: ~/.hapi).DB_PATH- SQLite database path (default: HAPI_HOME/hapi.db).TELEGRAM_NOTIFICATION- Enable/disable Telegram notifications (default: true).HAPI_RELAY_API- Relay API domain (default: relay.hapi.run).HAPI_RELAY_AUTH- Explicit relay auth key. By default the hub obtains and persists an individually revocable key from the relay. A persisted key rejected with HTTP 403 is discarded and reissued once; an explicitly configured environment key must be updated manually.HAPI_RELAY_FORCE_TCP- Force TCP relay mode (true/1).VAPID_SUBJECT- Contact email/URL for Web Push.
Running
Binary (single executable):
export TELEGRAM_BOT_TOKEN="..."
export CLI_API_TOKEN="shared-secret"
export HAPI_PUBLIC_URL="https://your-domain.example"
hapi hub
hapi server remains supported as an alias.
If you only need web + CLI, you can omit TELEGRAM_BOT_TOKEN.
To enable Telegram, set TELEGRAM_BOT_TOKEN and HAPI_PUBLIC_URL, start the hub, open /app
in the bot chat, and bind the Mini App with CLI_API_TOKEN:<namespace> when prompted.
From source:
bun install
bun run dev:hub
HTTP API
See src/web/routes/ for all endpoints.
Authentication (src/web/routes/auth.ts)
POST /api/auth- Get JWT token (Telegram initData orCLI_API_TOKEN[:namespace]).POST /api/bind- Bind a Telegram account using initData +CLI_API_TOKEN:<namespace>.
Sessions (src/web/routes/sessions.ts)
GET /api/sessions- List all sessions.GET /api/sessions/:id- Get session details.POST /api/sessions/:id/abort- Abort session.POST /api/sessions/:id/switch- Switch session to remote mode.POST /api/sessions/:id/resume- Resume inactive session.POST /api/sessions/:id/upload- Upload file (base64, max 50MB).POST /api/sessions/:id/upload/delete- Delete uploaded file.POST /api/sessions/:id/archive- Archive active session.PATCH /api/sessions/:id- Rename session.DELETE /api/sessions/:id- Delete inactive session.GET /api/sessions/:id/slash-commands- List slash commands.GET /api/sessions/:id/skills- List skills.POST /api/sessions/:id/permission-mode- Set permission mode.POST /api/sessions/:id/model- Set model preference.POST /api/sessions/:id/effort- Set Claude effort preference.
Messages (src/web/routes/messages.ts)
GET /api/sessions/:id/messages- Get messages (paginated).POST /api/sessions/:id/messages- Send message.
Permissions (src/web/routes/permissions.ts)
POST /api/sessions/:id/permissions/:requestId/approve- Approve permission.POST /api/sessions/:id/permissions/:requestId/deny- Deny permission.
Machines (src/web/routes/machines.ts)
GET /api/machines- List online machines.POST /api/machines/:id/spawn- Spawn new session on machine.POST /api/machines/:id/paths/exists- Check if path exists.
Usage (src/web/routes/usage.ts)
GET /api/usage/summary- Get cache-aware token usage for the owner namespace (range=7d|30d|all).
Git/Files (src/web/routes/git.ts)
GET /api/sessions/:id/git-status- Git status.GET /api/sessions/:id/git-diff-numstat- Diff summary.GET /api/sessions/:id/git-diff-file- File-specific diff.GET /api/sessions/:id/file- Read file content.GET /api/sessions/:id/files- File search with ripgrep.
Events (src/web/routes/events.ts)
GET /api/events- SSE stream for live updates.POST /api/visibility- Report client visibility state.
Voice (src/web/routes/voice.ts)
POST /api/voice/token- Get ElevenLabs conversation token.GET /api/voice/transcription/providers- List configured providers and supported modes.POST /api/voice/transcription- Transcribe a bounded recording.POST /api/voice/transcription/realtime-token- Mint a short-lived OpenAI, ElevenLabs, or Deepgram credential.
Push Notifications (src/web/routes/push.ts)
GET /api/push/vapid-public-key- Get VAPID public key.POST /api/push/subscribe- Subscribe to push notifications.DELETE /api/push/subscribe- Unsubscribe.
CLI (src/web/routes/cli.ts)
POST /cli/sessions- Create/load session.GET /cli/sessions/:id- Get session by ID.POST /cli/machines- Create/load machine.GET /cli/machines/:id- Get machine by ID.
Socket.IO
See src/socket/handlers/cli.ts for event handlers.
Namespace: /cli
Client events (CLI to hub)
message- Send message to session.update-metadata- Update session metadata.update-state- Update agent state.session-alive- Keep session active.session-ready- Cursor ACPsession/load(ornewSession) succeeded; hub defers merge/dedup until this arrives on reopen.session-end- Mark session ended.machine-alive- Keep machine online.rpc-register- Register RPC handler.rpc-unregister- Unregister RPC handler.
Terminal events (web to hub)
terminal:create- Open terminal for session.terminal:write- Send input.terminal:resize- Resize dimensions.terminal:close- Close terminal.
Hub events (hub to clients)
update- Broadcast session/message updates.rpc-request- Incoming RPC call.
See src/socket/rpcRegistry.ts for RPC routing.
Telegram Bot
See src/telegram/bot.ts for bot implementation.
Commands
/start- Welcome message with Mini App link./app- Open Mini App.
Features
- Permission request notifications with approve/deny buttons.
- Session ready notifications.
- Deep links to Mini App sessions.
See src/telegram/callbacks.ts for button handlers.
Core Logic
See src/sync/syncEngine.ts for the main session/message manager:
- In-memory session cache with versioning.
- Message pagination and retrieval.
- Permission approval/denial.
- RPC method routing via Socket.IO.
- Event publishing to SSE and Telegram.
- Git operations and file search.
- Activity tracking and timeouts.
Storage
See src/store/index.ts for SQLite persistence:
- Sessions with metadata and agent state.
- Messages with pagination support.
- Machines with runner state.
- Todo extraction from messages.
- Users table for Telegram bindings (includes namespace).
Message content is stored via src/store/contentCodec.ts: oversized strings
inside agent messages (giant tool output) are head+tail truncated at ingest,
and payloads ≥256 chars are zstd-compressed (TEXT = plaintext JSON, BLOB =
zstd). User prompts are never truncated — queued rows are delivered to the CLI
verbatim.
Maintenance scripts (run with the hub stopped before swapping files):
scripts/compact-db.ts— retroactively truncate + compress + VACUUM an existing DB into a new file (the source is only opened read-only).scripts/cleanup-sessions.ts— bulk-delete sessions by message count, path glob, or first-message pattern.
Source structure
src/web/- HTTP service and routes.src/socket/- Socket.IO setup and handlers.src/socket/handlers/cli/- Modular CLI handlers.src/telegram/- Telegram bot.src/sync/- Core session/message logic.src/store/- SQLite persistence.src/sse/- Server-Sent Events.src/config/- Configuration loading and generation.src/notifications/- Push and Telegram notifications.src/visibility/- Client visibility tracking.
Security model
Access is controlled by:
- Telegram initData verification plus bound Telegram users (bound via
CLI_API_TOKEN:<namespace>). CLI_API_TOKENbase secret for CLI and browser access (namespace is appended by clients).
Transport security depends on HTTPS in front of the hub.
Build for deployment
From the repo root:
bun run build:hub
bun run build:web
The hub build output is hub/dist/index.js, and the web assets are in web/dist.
Networking notes
- Telegram Mini Apps require HTTPS and a public URL. If the hub has no public IP, use Cloudflare Tunnel or Tailscale and set
HAPI_PUBLIC_URLto the HTTPS endpoint. - If the web app is hosted on a different origin, set
CORS_ORIGINS(orHAPI_PUBLIC_URL) to include that static host origin.
Standalone web hosting
The web UI can be hosted separately from the hub (for example on GitHub Pages or Cloudflare Pages):
- Build and deploy
web/distfrom the repo root. - Set
CORS_ORIGINS(orHAPI_PUBLIC_URL) to the static host origin. - Open the static site, click the Hub button on the login screen, and enter the hapi hub origin.
Leaving the hub override empty preserves the default same-origin behavior when the hub serves the web assets directly.