* feat(voice): voice personality, picker catalog, and prompt layer foundation
- voicePickerCatalog.ts: per-backend voice lists for Gemini and Qwen with
resolve helpers (resolveGeminiLiveVoice, resolveQwenRealtimeVoice)
- voicePersonality.ts: VoicePersonalityPreferences schema, presets, composed
system prompt with identity/character/response-length layers
- voicePromptLayers.ts: buildResolvedVoiceSystemPrompt, preset delivery snippets
- voiceSystemPromptParam.ts: hub-side base64url decode for ?systemPrompt=
- voicePickerPreferences.ts, voicePersonalitySession.ts: browser-side encode,
decode, and storage helpers
- useVoicePersonality: React hook for preferences persistence
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): preset delivery included when non-balanced preset selected; restore test typecheck
- isDefaultVoicePersonality: add preset check so warm/calm/direct presets
trigger the delivery snippet instead of being treated as default
- web/tsconfig.json: remove test file exclusion from typecheck (restoring
strict coverage of test code); fix resulting type error in mock declaration
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): include use_speaker_boost in ElevenLabs TTS override payload
The checkbox persisted the pref but ttsDiffersFromDefault and
buildElevenLabsTtsOverride both omitted it, so the setting was never
sent to the agent.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* test(voice): update speaker_boost test to assert it IS included in override
The previous test asserted use_speaker_boost was omitted; now it's
correctly included in the TTS payload.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): authorize use_speaker_boost in ElevenLabs override schema
Add use_speaker_boost to both the VoiceAgentConfig tts override type
and the buildVoiceAgentConfig() platform_settings so the field is
accepted by the ElevenLabs agent runtime.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): propagate full language code through composed prompt, not just zh
getDefaultVoiceSystemPrompt and resolveComposedVoiceSystemPrompt were
filtering language to zh-only before passing to composeVoiceAgentPrompt.
Now append buildVoiceLanguageBlock(language) after composition so French,
Spanish, Japanese etc. reach Gemini/Qwen sessions correctly.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): only append language block when language explicitly set
Building language block unconditionally when no language is given
caused getDefaultVoiceSystemPrompt() to diverge from VOICE_SYSTEM_PROMPT.
Only append the block when a code is explicitly provided.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): always include language block for Gemini/Qwen in composed prompt
When auto-detect is on (language=undefined), the composed prompt sent
via hub proxy was losing the language auto-detect instruction because
the block was only added when language was explicitly set.
Now: ElevenLabs skips the block (has its own language field); Gemini/Qwen
always include it — undefined produces the auto-detect block, an explicit
code produces the appropriate language instruction.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
---------
Co-authored-by: HAPI <noreply@hapi.run>
* feat(voice): pluggable voice backend with Gemini Live & Qwen Realtime
Rebased from Overbaker/hapi#401 onto current main. Adds a pluggable voice
backend architecture that extends the existing ElevenLabs integration:
- Gemini 2.5 Live (gemini-live): Google real-time audio via WebSocket
with full function calling (messageCodingAgent, processPermissionRequest)
- Qwen Realtime (qwen-realtime): Alibaba DashScope via hub WebSocket
proxy (browser cannot set Authorization header directly)
- VoiceBackendSession: dynamic backend selector with React.lazy loading,
gates voice button until backend module is registered
- Hub WS proxies: JWT-authenticated /api/voice/gemini-ws and
/api/voice/qwen-ws endpoints in Bun.serve, with message queueing during
upstream connect to prevent dropped setup frames
- AudioWorklet pipeline: inline Blob URL recorder, 24 kHz PCM player,
serial tool call execution, AudioContext created in user gesture for mobile
- Backend discovery: GET /voice/backend + POST /voice/gemini-token /
POST /voice/qwen-token hub routes; frontend auto-detects active backend
Merge notes:
- Rebased 135 upstream commits cleanly; HappyComposer keeps upstream's
configurable enter-behavior setting (supersedes hard-coded Ctrl+Enter)
- Converted gemini test files from bun:test to vitest (web package uses vitest)
- All 221 hub tests and 636 web tests pass; TypeScript clean
* fix(voice): restore user mic mute state after Gemini turn completes
turnComplete handler was unconditionally calling setMuted(false), which
re-enabled the mic track even when the user had manually muted. Now
restores to state.micMuted instead.
* fix(voice): remove hard-coded Chinese language from Gemini backend
buildGeminiLiveConfig was appending VOICE_CHINESE_LANGUAGE_BLOCK which
forced Gemini to always respond in Mandarin regardless of user locale.
Gemini now uses the neutral base prompt and responds in the language the
user speaks to it, consistent with the ElevenLabs behaviour.
* fix(voice): reset modelSpeaking in cleanup to unblock mic on restart
If the session closes while Gemini is mid-speech, cleanup() left
state.modelSpeaking=true. The next startSession() would then drop all
mic audio in sendAudioChunk() until a model turn eventually flipped
the flag — effectively deaf until page reload.
* fix(voice): guard stale close handlers in Gemini and Qwen sessions
ws.onclose operated on module-level state.ws, not the socket that fired
the event. A rapid stop/restart could cause the old socket's onclose to
call cleanup() after the new socket was assigned, tearing down the live
session. Guard with `if (state.ws !== ws) return` before cleanup.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): remove hard-coded Chinese language from Qwen backend
Matches the Gemini fix — both backends now use VOICE_SYSTEM_PROMPT
without the Chinese language block, giving consistent English-default
behaviour across all non-ElevenLabs backends.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* feat(voice): proactive/reactive toggle in voice settings
Adds a "Proactive voice" toggle (default: off = reactive) to the Voice
Assistant settings section.
Reactive (default): initial context and agent-ready events are fed
silently; the assistant waits for the user to speak first.
Proactive: original behaviour — Gemini/Qwen narrate context on connect
and speak unprompted when the agent finishes a task. ElevenLabs is also
affected via onReady sending a user message rather than a silent update.
Covers all three backends uniformly. localStorage key: hapi-voice-proactive.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): normalize WS close codes, drop barrel re-exports, fix SSE visibility
- hub/server.ts: add toClientCloseCode() to normalize reserved upstream
close codes (1005/1006/1015) to 1011 before forwarding to browser;
abnormal upstream drops (1006) would otherwise throw on clientWs.close()
and leave the browser socket open
- realtime/index.ts: remove static GeminiLiveVoiceSession and QwenVoiceSession
barrel exports; VoiceBackendSession lazy-imports both, so barrel re-exports
created static dependencies that defeated the intended code-split
- App.tsx: gate global useVisibilityReporter on !sessionEventSubscription so
the always-on SSE connection does not suppress native Web Push notifications
for sessions the user is not currently viewing
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): respect language setting in Gemini/Qwen; fix voice-start toggle label
- buildGeminiLiveConfig() now accepts optional language param; appends
VOICE_CHINESE_LANGUAGE_BLOCK only when language === 'zh'
- GeminiLiveVoiceSession passes config.language through
- QwenVoiceSession conditionally builds basePrompt from language setting
- Fixes silent no-op when user selects Chinese in voice settings on
Gemini/Qwen backends (was ElevenLabs-only)
- Rename voice-start toggle label to 'Start voice session with summary'
- Fix description: clarifies the choice is about session-open behaviour
(summary vs greeting), not ongoing narration
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): send greeting trigger in reactive mode for Gemini
Gemini Live has no built-in first-message like ElevenLabs agents do;
without an explicit turnComplete:true it sits silently. In reactive mode
(default, toggle off) now sends a greeting instruction after any silent
context feed so Gemini introduces itself and invites the user to speak.
Proactive mode is unchanged: the context summary is the opening speech.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): suppress Gemini self-identification and context leak in greeting
- VOICE_SYSTEM_PROMPT: explicit instruction never to call itself Gemini,
Google, or any underlying model/provider name — always HAPI
- Greeting trigger text: instruct to greet as HAPI only, suppress model
name and any reference to context/recent activity in the opening line
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): address code review findings — error handling, proxy, audio
Gemini + Qwen client:
- onerror now sets setupDone/sessionReady and nulls state.ws before
calling reject(), so the stale-close guard trips in onclose and
prevents a duplicate statusCallback('error') on WS failure
Gemini client:
- Proactive mode with no initialContext now falls through to the
greeting trigger instead of sitting silently
- Remove unused handleBargeIn callback (dead code)
Qwen client:
- Add input_audio_sample_rate: 16000 to session.update so PCM rate
is declared explicitly rather than relying on DashScope's default
Hub proxy:
- Remove no-op ternary in Gemini flush loop and message handler
(typeof x === 'string' ? x : x); use upstream.send(msg) directly
- Qwen onerror now calls upstreamMap.delete() before closing client,
eliminating the stale map entry window
- Align Qwen hub fallback model string with QWEN_REALTIME_MODEL
constant ('qwen3-omni-flash-realtime')
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): trailing-slash WS URL, Qwen session.update schema
hub/voice.ts:
- Replace string-concat WS URL construction with buildVoiceWsUrl() which
uses URL API to set protocol/pathname cleanly — fixes double-slash when
HAPI_PUBLIC_URL has a trailing slash (would silently skip the proxy route)
QwenVoiceSession.tsx:
- Wrap tool definitions in {type:'function', function:{...}} as required
by Qwen-Omni realtime schema — previous flat shape caused session.update
rejection before audio capture could start
- Use pcm16/pcm24 audio formats matching DashScope spec; remove
input_audio_sample_rate (encoded in format name)
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): await audio capture before setMuted; sanitize upstream close codes
GeminiLiveVoiceSession + QwenVoiceSession:
- startAudioCapture() is now async and awaits recorder.start() before
calling setMuted() — previously setMuted ran before getUserMedia resolved
so a session restarted while muted would open the mic anyway
- statusCallback('connected') now fires after audio is ready
- setMuted() called unconditionally (not just when true) to correctly
apply saved state in either direction
hub/src/web/server.ts:
- Both Gemini and Qwen close() handlers now pass the client code through
toClientCloseCode() before forwarding to upstream — prevents reserved
codes (e.g. 1006) from causing WebSocket.close() to throw and leave
the upstream session open until provider timeout
- Reason string capped at 123 bytes (WebSocket protocol limit)
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): wrap startAudioCapture in try/catch to propagate mic errors
An unhandled rejection inside the async onmessage callback does not
propagate to the outer startSession Promise — the UI hangs on
'connecting' and the provider socket stays partially open. Wrapping
the await in try/catch calls cleanup()/statusCallback('error')/reject()
so failures surface correctly.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): propagate backend discovery failure instead of silently falling back to ElevenLabs
fetchVoiceBackend no longer catches errors and defaults to 'elevenlabs' — any
network or server failure now throws so VoiceBackendSession can surface it via
onStatusChange('error', ...) rather than silently mounting the wrong backend.
VoiceBackendSession also resets backend state to null when api changes, so
a stale ElevenLabs registration from a prior discovery cannot persist into
a new session.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): throw on unrecognised backend value instead of silently falling back to ElevenLabs
Unknown backend strings (future values, typos) now throw rather than defaulting
to elevenlabs, closing the narrow remaining form of the original misrouting bug.
Also removes the unnecessary `as VoiceBackendResponse` cast.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): add Qwen greeting/proactive trigger; fix socket buffer for base64 uploads
Qwen session.updated handler now sends the same proactive summary or greeting
trigger that Gemini does — previously it started silently in both proactive and
reactive modes.
maxHttpBufferSize raised to 68 MiB to account for base64 expansion: 50 MiB
decoded files become ~66.7 MiB as base64 JSON, so the previous 55 MiB ceiling
would disconnect uploads above ~41 MiB before they reached the CLI.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): replace unsupported conversation.item.create with session.update for Qwen text
Qwen's realtime API only supports conversation.item.create for function_call_output.
Sending it with type:'message' for greetings/context was invalid and could fail
before the user spoke.
sendTextMessage and sendContextualUpdate now update session instructions via
session.update (accumulating context into the system prompt) and trigger
response.create only when a spoken reply is needed — matching Qwen's supported
client event surface.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): guard session.updated re-entry and reset config on session start
session.updated now returns early after the first ack — subsequent session.update
calls (instruction appends) also echo session.updated but must not re-trigger
audio capture or the greeting path.
currentSessionConfig is now reset to null at the top of startSession so a stale
config from a failed previous session cannot leak into the new one.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): assert wsUrl presence for Gemini proxy connections
Without this guard, a missing wsUrl in the hub token response would
silently attempt to connect directly to Google with "proxied" as the
API key — producing a confusing auth failure instead of a clear error.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): correct Qwen audio formats and default voice
DashScope realtime API accepts only 'pcm' for both input and output
audio formats. The pcm16/pcm24 values caused session.update rejection
before audio capture could start, leaving the Qwen backend unusable.
Also updates the default voice from Mia (not in the qwen3-omni-flash-
realtime voice list) to Cherry, which is documented as supported.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): close AudioContext on failed voice session start
Failed token fetch, microphone denial, or WebSocket error during
setup left state.playbackContext open. Each failure path now calls
cleanup() before throwing/rejecting, preventing AudioContext leaks
on mobile browsers with hard limits on concurrent contexts.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* chore: restore non-voice files to upstream/main state
Reverts changes to files that shouldn't differ from upstream:
- .gitignore: remove fork-only AGENTS.local.md entry
- web/src/App.tsx: restore dual-subscription SSE pattern (scope-aware)
- web/src/hooks/useSSE.ts: restore SSEScope/scope parameter
- web/src/hooks/useSSE.test.ts: restore (was accidentally deleted)
- web/src/lib/appSseSubscriptions.ts: restore (was accidentally deleted)
- web/src/lib/appSseSubscriptions.test.ts: restore (was accidentally deleted)
- hub/src/sync/syncEngine.ts: restore (off-topic change)
* fix(voice): harden Gemini and Qwen WS proxies against client abuse
Hub sends HAPI-owned Gemini setup on proxy connect and rejects client
setup frames. Qwen proxy always uses QWEN_REALTIME_MODEL instead of a
client query parameter. Shared buildGeminiLiveSetupMessage() keeps wire
format in one place.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(voice): harden Qwen proxy — hub-owned setup, client frame allowlist
Mirror the Gemini proxy security model for Qwen:
- Hub sends initial session.update (voice/tools/instructions) on upstream
connect so the browser cannot override config fields.
- Proxy message() now calls isQwenSafeClientFrame() and closes the
connection (1008) if a client session.update touches any field other
than 'instructions' (blocks tool/voice/modality overrides).
- QwenVoiceSession no longer sends session.update on session.created;
it waits for the hub-relayed session.updated and then sends only
instruction-only updates for context/proactive content.
- Language passed as query param (?language=zh) so hub builds the
correct Chinese system prompt without a client-supplied session.update.
- buildQwenSessionUpdateMessage() and isQwenSafeClientFrame() added to
@hapi/protocol/voice; 9 new unit tests cover filter edge cases.
* fix(voice): respect Qwen session.created→session.update protocol ordering
DashScope requires session.update to be sent AFTER session.created is
received, not immediately on WebSocket open. Previously the hub sent
session.update in upstream.onopen, which violated this ordering and
risked the config being processed in an uninitialized session context.
Add pendingSetupMap to buffer the hub-owned session.update payload.
The onmessage handler now relays session.created to the browser first,
then immediately sends the pending session.update to DashScope — matching
the protocol ordering the old browser-side code used (which waited for
session.created before sending session.update).
Also remove maxHttpBufferSize from the socket.io Engine config. That
setting is unrelated to voice backends; upstream/main had no such limit
set and it is not introduced by this PR.
* fix(voice): use Realtime tool shape for Qwen session.update (not chat-completions)
Qwen Realtime session.update expects tools as flat objects:
{ type: 'function', name, description, parameters }
The previous code used the chat-completions shape:
{ type: 'function', function: { name, description, parameters } }
DashScope may reject session.update or silently ignore tools with the
nested shape, causing tool calls to fail at runtime. Fix applied in
buildQwenSessionUpdateMessage(); test updated to assert flat shape and
that no nested `function` key is present.
* fix(voice): update Qwen Realtime model, voice, and endpoint for intl service
Live-tested against DashScope international API:
- Model: qwen3-omni-flash-realtime → qwen3.5-omni-flash-realtime
(previous model ID did not exist on DashScope)
- Default voice: Cherry → Tina
(confirmed from session.created response on qwen3.5-omni-flash-realtime)
- Default WS base: dashscope.aliyuncs.com → dashscope-intl.aliyuncs.com
(international accounts use the -intl endpoint; China endpoint rejects
international API keys; QWEN_REALTIME_WS_URL env var still overrides)
* fix(voice): correct Qwen text injection and generalise language handling
Two dogfooding fixes verified against live Qwen Realtime session:
sendTextMessage: switch from instruction-injection to conversation.item.create
Qwen Realtime requires a user conversation item before response.create.
The previous approach (updateInstructions + response.create) produced
"input messages do not contain elements with role user" errors. Now sends
{type:message, role:user, content:[{type:input_text}]} then response.create.
sendContextualUpdate is unchanged (instruction-only, no response trigger).
Language handling: replace zh-only branch with buildVoiceLanguageBlock()
Previously, only language='zh' added any instruction; all other languages
(including English) sent no language block, causing Qwen to drift to Chinese.
buildVoiceLanguageBlock() now covers three cases:
- 'zh'/'zh-*': existing Chinese block (unchanged)
- explicit code ('en','es','fr',...): "Always respond in [Language]"
- undefined/auto: "Detect the user's language and maintain it"
Applied to buildGeminiLiveConfig, buildQwenSessionUpdateMessage, and the
client-side currentInstructions mirror in QwenVoiceSession.
Also removes the Gemini hub proxy's zh-only filter, which was discarding
explicit language selections other than Chinese.
* fix(hub): gate Gemini client frames until upstream setupComplete
Hub sends its owned setup on upstream open, then waits for Google's
setupComplete acknowledgment before flushing queued client frames.
isGeminiSetupCompleteFrame() detects the {"setupComplete":{}} message;
message() queues instead of forwarding while pendingMap is live.
Addresses the repeated Major finding from bot review on PR #743.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(hub): cap Gemini setup-window pending queue at 1 MiB
An authenticated client could flood the queue between upstream.onopen
and Google's setupComplete acknowledgment. Add pendingBytesMap tracking
and close with 1009 if the budget is exceeded.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(gemini): pass all language codes to hub proxy, not just zh
Language selection for French, Spanish, Japanese etc. was silently
dropped — only 'zh' was forwarded as a query param.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): expand LANGUAGE_NAMES to cover full ElevenLabs language set
Codes like 'no', 'da', 'fi', 'pt-br', 'bg', 'ro', 'cs', 'el', 'ms',
'tl', 'uk', 'hu', 'hr', 'sk' were falling through to raw-code prompts
("Always respond in no"). Now resolve to proper display names.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
---------
Co-authored-by: HAPI <noreply@hapi.run>
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(web): per-session scratchlist (workbench) panel
Adds a per-session "scratchlist" panel above the composer for parking
notes / drafts / parking-lot ideas that are explicitly held — never
auto-sent. This is distinct from the existing queue (QueuedMessagesBar):
- Queue = conveyor belt: messages auto-fire once the agent is idle.
- Scratchlist = workbench: held until the operator promotes them.
The amber accent and "held — not sent" pill make the visual distinction
obvious so operators don't mistake one for the other.
Features:
- Collapsible per-session panel (collapsed by default, persisted in
localStorage).
- Add (Enter) / delete / reorder (up/down) entries.
- Promote-to-composer copies into the composer for editing (entry
stays — copy semantics).
- Promote-to-queue routes through the existing onSend path so the
entry shows up in QueuedMessagesBar; entry is removed only on
accepted send.
- Entries persist per session under hapi.scratchlist.v1.<sessionId>.
- Confirm-on-delete only for entries longer than 100 chars.
- Ctrl/Cmd+Shift+S focuses the add-input.
- en + zh-CN strings.
v1 scope: localStorage-only. Hub-sync deferred to v2 to keep the
diff small and reviewable.
Test coverage:
- web/src/lib/scratchlist.test.ts — 21 tests (storage round-trip,
add/delete/reorder/cap, malformed-JSON resilience, confirm threshold).
- web/src/components/AssistantChat/ScratchlistPanel.test.tsx — 13
tests (collapse persistence, hydration, add/delete/reorder UI,
promote-to-composer copy semantics, promote-to-queue accepted /
rejected paths, per-session isolation).
Closes#11
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(scratchlist): block focus into collapsed panel via inert
Upstream review (tiann/hapi#772, codex bot) flagged that the collapsed
scratchlist body was visually hidden via CSS only - the textarea and
action buttons stayed mounted, focusable, and clickable while their
ancestor was aria-hidden. Tab into invisible controls + a hidden
subtree with focusable descendants is an a11y violation.
Apply `inert` to the inner content, gated on the collapsed state.
This removes the subtree from the focus, pointer, and accessibility
trees while keeping the grid-template-rows expand animation intact
(no conditional remount, so the open/close transition still runs).
Add a regression test that asserts `inert` is present while collapsed
and removed (or empty) while expanded, so a future revert of the fix
trips immediately.
Co-authored-by: Cursor <cursoragent@cursor.com>
* test(scratchlist): add Playwright e2e + isolated fixture page
The unit suite under jsdom can't verify the parts of the scratchlist
that actually live in the browser:
- `inert` blocks focus (jsdom ignores `inert`)
- the grid-template-rows collapse animation
- localStorage surviving a full page reload
- per-session keying surviving cross-route navigation
- Ctrl/Cmd+Shift+S firing the global expand+focus shortcut
Add a Playwright config + spec that drives a real Chromium against a
new Vite-served fixture (`web/e2e-fixtures/scratchlist-fixture.html`).
The fixture mounts the production `ScratchlistPanel` in isolation
inside an `I18nProvider` and exposes the promote callbacks on
`window.__scratchlistE2E` so the spec can assert that promote-to-
composer and promote-to-queue receive the right text without having
to spin up the hub, auth, or socket layer.
Nine specs cover:
1. starts collapsed, toggles
2. collapsed inner is `inert` and refuses focus / pointer
3. add: entry appears, draft clears, count updates
4. persistence across full page reload
5. promote-to-composer fires callback (entry stays - copy semantics)
6. promote-to-queue success path (entry removed)
7. promote-to-queue failure path (entry retained for retry)
8. Ctrl+Shift+S expands + focuses input
9. per-session isolation across navigation
Wires `bun run test:e2e` and `test:e2e:ui` at the repo root and
documents the harness in `web/README.md`. Bumps `playwright` 1.49.1
-> 1.60.0 alongside the new `@playwright/test` dep so the bundled
chromium-headless-shell-1223 (Chrome 148) is used; the older 131
binary SIGTRAPs on this kernel during launch. Adds
`test-results/` and `playwright-report/` to `.gitignore`.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(scratchlist): key host by session.id to prevent cross-session leak
Upstream review (tiann/hapi#772, codex bot follow-up) flagged a state
leak across same-route session switches. ScratchlistPanel reads
`sessionId` once via `useState(() => readScratchlist(sessionId))` and
rehydrates in a `useEffect`. SessionChat stays mounted when the
operator switches sessions on the same `/sessions/$sessionId` route,
so the panel sees a new `sessionId` prop without unmounting. Effect
order during the prop change:
1. render with sessionId=B but stale entries=[A's items]
2. rehydrate effect: setEntries(read(B)) -> queues correction
3. persist effect (deps [sessionId, entries] both changed):
persistScratchlist(B, [A's items]) -> writes A into B
4. re-render with sessionId=B, entries=B's items
5. persist effect: persistScratchlist(B, B's items)
-> overwrites the bug write
The bug is transient (step 3's write is corrected by step 5) but
real: any read between steps 3 and 5 (another tab, a SW prefetch,
manual inspection) sees A's data under B's key.
Fix is one line: `key={props.session.id}` on `<ScratchlistHost>`.
React unmounts and remounts the host when the key changes, so the
new mount's useState initializer reads B's storage from scratch and
never touches B's key with A's data. This is the React-canonical
"reset state on prop change" pattern; cleaner than chasing the race
inside the panel.
Add an e2e regression test that:
- installs a `localStorage.setItem` spy in `addInitScript`
- mounts the fixture under session A and adds an entry
- clears the spy, then switches to session B in-place via
`window.__scratchlistE2E.setSessionId('leak-B')` (no page reload)
- asserts no recorded write to `hapi.scratchlist.v1.leak-B`
contained A's text (catches the transient corrupting write
deterministically, before the correction overwrites it)
- round-trips back to A to confirm A's storage is intact
The fixture grows a `?key=0` mode that drops the host's `key=` prop.
Verified red/green: with `key=0` the regression test fails on the
spy-detected corrupting write; with the fix in place (default), all
10 e2e specs pass.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(web): add close button to dialog so modals are dismissable on mobile
The shared DialogContent had no close affordance — desktop users could
press Escape or click the overlay, but on mobile (no Escape key, dialog
spans calc(100vw-24px) leaving almost no tappable overlay) there was no
way to dismiss it. Add a DialogPrimitive.Close X button in the top-right,
fixing every dialog that uses this component at once.
* fix(web): reserve header space for dialog close button
Address review feedback: the absolutely-positioned close button overlaps
the top-right of every dialog. Long/breaking titles (e.g. DiffView's
break-all filename) could wrap underneath the 32px tap target. Add pr-12
to DialogHeader rather than padding DialogContent globally, so the title
row clears the button while body content (code blocks, diffs) keeps full
width.
* fix(web): localize dialog close button aria-label
Use the existing button.close locale string instead of a hardcoded
"Close" so screen-reader users get the label in their language (zh-CN: 关闭).
`PermissionHandler` stored its own `permissionMode` field and only updated
it inside `handleModeChange`, which is called when a new batch is pulled
from the queue. The `SetSessionConfig` RPC (web dropdown changes) updates
`runClaude.ts`'s `currentPermissionMode` and the session keepalive
metadata, but never reaches the handler — so switching to Yolo mid-turn
left `canCallTool` checking the stale mode and still prompting for
approval. Closes#735.
Drop the stored field and read live from `session.getPermissionMode()`,
mirroring how the OpenCode permission handler already works. Override
`Session.getPermissionMode()` in `claude/session.ts` to return the
Claude-narrow `PermissionMode`, sound because the matching
`setPermissionMode` setter only accepts that subset.
via [HAPI](https://hapi.run)
Co-authored-by: HAPI <noreply@hapi.run>
* fix(cli): fixed an issue where the codex cli failed to run successfully when using hapi codex in powershell
* fix(cli): Fixes the issue of Windows Codex npm shim bypassing the launcher
---------
Co-authored-by: xhd902 <xuhang@infypower.cn>
* feat(cursor): wire /summarize and /clear slash builtins for remote sessions
Seed cursor builtins for web autocomplete, parse summarize/clear in
cursorRemoteLauncher (pass-through to agent -p; reject /clear with args).
Fixes tiann/hapi#738
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(cursor): isolate slash commands before message queue batching
Parse summarize/clear at enqueue time (runCursor) with pushIsolateAndClear
so waitForMessagesAndGetAsString never merges a slash with the next prompt.
Adds queue policy tests for invalid /clear + following message.
Addresses PR #747 review.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(cursor): preserve pending messages when isolating slash commands
pushIsolateAndClear() wipes the entire queue, so a normal prompt queued
before /summarize or /clear would be silently dropped. Add pushIsolated()
- isolation without clearing - and route Cursor slash commands through
it instead. Adds queue tests covering the preserve-then-isolate path.
Addresses PR #747 review.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
PR #756 added a mid-turn emit in captureUsageUpdate to surface live
context usage via the web status bar. Testing against OpenCode 1.15.11
on a real session shows OpenCode emits a single usage_update per turn,
within ~1ms of session/prompt resolving — never during streaming.
That makes the mid-turn path dead code for OpenCode (and for any other
ACP agent that follows the same pattern). It also persists a useless
inputTokens:0/outputTokens:0 token_count message that gets immediately
overwritten by the finalize emit, churning the session history.
Drop the mid-stream emit and the activeOnUpdate plumbing it required.
Keep the finalize fallback for agents that don't return a usage block
on session/prompt (slash-handled turns, errored turns). The persistent
"live" counter requires the agent to emit usage_update during streaming;
filed upstream against anomalyco/opencode.
Refs #750
via [HAPI](https://hapi.run)
Co-authored-by: HAPI <noreply@hapi.run>
* fix(web): apply messages-consumed on global SSE connection
The global all-sessions SSE subscription returned early on message-stream
events without updating the message-window store. When session-scoped SSE
was reconnecting or the user had another session selected, messages-consumed
never cleared the queued bar even though the hub had stamped invoked_at.
Also harden mergeMessages so a stale invokedAt:null snapshot cannot clobber
an existing ack timestamp.
Fixestiann/hapi#758
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(web,hub): resume never-started inactive sessions on first send
Hub fresh-spawns when inactive session has path but no agent thread id and
zero messages. Web guards resume, updates inactive banner copy, and surfaces
resume_unavailable before POST /resume when resume is impossible.
Fixestiann/hapi#759
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(web): scope sessionResume guard to current flavor only
Hub `resolveAgentResumeId` only honors the metadata.flavor's id; the web
guard was falling back across all flavors so a cursor session with a stale
codexSessionId still tried to resume and 409'd. Mirror the hub switch and
default to claude when flavor is unknown.
Addresses HAPI Bot review on tiann/hapi#761.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(web): allow claude session resume via hub message-id recovery
Hub `resolveAgentResumeId` falls back to `recoverClaudeSessionIdFromMessages`
on the claude branch when `metadata.claudeSessionId` is absent, so the web
guard must not block inactive claude sessions that have stored messages but
no metadata id. Other flavors have no such recovery path and stay rejected.
Addresses second HAPI Bot review thread on tiann/hapi#761
(`web/src/lib/sessionResume.ts:41`).
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(acp): flush straggler chunks promptly after session/prompt returns
After session/prompt returns, HAPI drains buffered agentMessageChunk text
and marks the turn complete, but leaves the message handler alive. Models
with long streaming tails (DeepSeek, GPT-5.5) continue to push chunks
after that drain, causing text to accumulate in the buffer and only appear
when the next user prompt triggers the pre-prompt drain — showing up in
the wrong turn with broken markdown.
Start a 50ms interval timer after the post-prompt drain that keeps calling
drainBuffers() on the live handler for up to 6 seconds, so straggler
chunks are emitted within one poll tick instead of waiting for the next
prompt. The timer is cancelled when the next prompt starts (pre-prompt
drain replaces the handler) or on disconnect.
Fixes#609. Also applies to Gemini and Kimi which share the same
AcpSdkBackend code path.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(acp): gate next turn's handler swap on previous turn's late drain
Addresses the github-actions review on #730: the fire-and-forget late-flush
timer let `prompt()` resolve while stragglers were still possibly arriving,
so a rapid follow-up prompt could either drop those chunks (during the
old null-handler gap) or leak them into the new turn's onUpdate.
Pre-prompt phase now keeps the previous turn's handler alive across the
quiet wait (bounded by LATE_FLUSH_WINDOW_MS) and swaps in a single phase
immediately before sending the new session/prompt. The post-prompt late
flush timer is unchanged — it still emits idle-window stragglers promptly
without delaying the ready signal or `setModel` / `setConfigOption`.
Adds three regression tests: late-chunk flushing within the window,
pre-prompt straggler attribution to the previous turn's onUpdate, and
disconnect cancelling the timer. Removes now-unused
PRE_PROMPT_UPDATE_DRAIN_TIMEOUT_MS.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(acp): await late drain so ready never fires before stragglers emit
Follow-up to bot's repeated MAJOR on #730: even with the pre-prompt gate,
the fire-and-forget late-flush timer let `prompt()` resolve before slow
tails finished, so the launcher's `ready` signal (and any user follow-up
queued against it) raced with text still being emitted to the current
turn's onUpdate.
Replace the setInterval timer with a synchronous `drainLateBuffers()`
awaited in `prompt()`'s finally before turn_complete is sent. It polls
drainBuffers every LATE_FLUSH_INTERVAL_MS so the UI keeps streaming
smoothly during the wait, and exits early once the model has been quiet
for LATE_FLUSH_QUIET_PERIOD_MS (250 ms — adds negligible latency to fast
models like Claude whose tail is typically <100 ms) or the
LATE_FLUSH_WINDOW_MS upper bound (6 s) elapses.
Side effects:
- Restore PRE_PROMPT_UPDATE_DRAIN_TIMEOUT_MS (1200 ms): the pre-prompt
drain is now just a safety net since the post-prompt wait guarantees
the previous turn is quiet by the time the next prompt starts.
- Drop the `lateFlushTimer` field, `startLateFlushTimer`,
`stopLateFlushTimer`, and their disconnect/pre-prompt cleanup calls.
- Update the "emits straggler chunks" test to assert ordering before
turn_complete, and add a fast-path test confirming the drain exits
promptly when the model is quiet.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(acp): anchor late-flush quiet window to entry, not stale lastSessionUpdateAt
Bot's third MAJOR on #730: drainLateBuffers() compared elapsed time
against lastSessionUpdateAt, which can already be older than
LATE_FLUSH_QUIET_PERIOD_MS by the time the method starts — e.g. when the
model emits chunks early in the turn, pauses, then sends stopReason. In
that case the first loop iteration sees a stale "quiet" reading and
returns immediately, missing any straggler that arrives just after
session/prompt resolves; the chunk then sits in the buffer until the
next prompt's pre-prompt drain.
Anchor the quiet check to max(lastSessionUpdateAt, entry time) so we
always observe at least one quiet period from method entry regardless of
when the last chunk was. Adds a regression test that fires a chunk
early, awaits a 200ms pause, schedules a post-resolution straggler, and
asserts it lands before turn_complete.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* docs(acp): correct LATE_FLUSH_QUIET_PERIOD_MS comment after entry-anchor fix
The previous note claimed the 250ms quiet check "exits early for fast
models, adding negligible latency". That was true before commit 512d6a4
when the check compared against lastSessionUpdateAt; with the entry-time
anchor, drainLateBuffers always observes at least one full quiet period.
Document that this minimum wait is the price of catching post-resolution
stragglers from paused-mid-turn models.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
---------
Co-authored-by: HAPI <noreply@hapi.run>
resumeSession already passes permissionMode to spawnSession. The follow-up
applySessionConfig raced session-alive (handler not registered yet) and
returned resume_failed after hub restart.
Co-authored-by: Cursor <cursoragent@cursor.com>
Remote cursor launcher now mirrors local launcher by writing cursorSessionId
to hub metadata as soon as --resume is known, before the agent init event.
POST /sessions/:id/resume maps resume_unavailable to 409 with clearer guidance.
Fixestiann/hapi#744
Co-authored-by: Cursor <cursoragent@cursor.com>
The Claude effort selector (New Session config + in-session composer) only
offered auto/medium/high/max, missing `low` and `xhigh` — yet `claude --effort`
actually accepts low/medium/high/xhigh/max. Add the two missing levels in both
places so the selector faithfully mirrors the CLI.
Extract the level list + labels into one shared constant
(@hapi/protocol: shared/src/effort.ts, mirroring CLAUDE_MODEL_PRESETS) so the
two UIs derive from a single source and can't drift again. No backend change:
the effort string is free-form end-to-end through to the --effort flag.
ultracode is intentionally excluded — it is a TUI-only /effort session setting,
not an --effort value (the CLI rejects `--effort ultracode`).
* chore(web): upgrade @tanstack/react-router to ^1.170.8
Fixes QuotaExceededError in scroll restoration: upstream @tanstack/react-router
>=1.145.6 wraps sessionStorage.setItem with try-catch, preventing the crash when
scroll restoration cache exceeds quota.
Refs: #683, #716, #721
* fix(web): adapt scrollStorageGuard to @tanstack/router-core >=1.145.6 API
`scrollRestorationCache` was removed from the public exports; replace with
`storageKey` import and simplify `hardResetScrollRestorationPersistedState`
to a plain `removeItem`. Remove the now-stale in-memory cache sync path and
its associated tests. Upstream try-catch (>=1.145.6) covers crash prevention;
this guard continues to proactively prune sessionStorage.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
---------
Co-authored-by: HAPI <noreply@hapi.run>
Closes#724
The previous policy pinned acceptance to a single past model version that
is no longer current SOTA, was not reliably detectable by reviewers, and
penalized contributors who honestly disclosed their tooling. Swap it for
a disclosure-only line — code is judged on merit, transparency is kept.
The "Default" model in NewSession sends no --model flag, so Claude CLI
picks its own default (e.g. Opus 4.7 [1m] on Pro accounts). The web
status bar then falls back to 200K - 10K headroom = 190K because the
Claude SDK path never plumbs the real per-model contextWindow through
to the wire-level `modelContextWindow`, unlike the ACP/Codex backends.
Fix the gap in three places:
- cli/src/claude/sdk/types.ts: declare optional `modelUsage` on
SDKResultMessage to surface what Claude CLI already emits
(`modelUsage[<model>].contextWindow`).
- cli/src/claude/utils/sdkToLogConverter.ts: on system.init, capture
the resolved model name (full form with `[1m]` suffix) and derive
an initial contextWindow from the suffix. On every assistant
message, inject the cached contextWindow into `usage.context_window`
when absent. On result, refine the cache with the authoritative
value from `modelUsage`.
- web/src/chat/normalizeAgent.ts: forward `context_window` through
the assistant usage normalization, so the existing reducer path
(reducer.ts:175 → StatusBar.tsx:175) can render the real window.
Closes#719.
* feat(opencode): support plan mode
* feat(opencode): support reasoning effort
* feat(opencode): surface context usage in web
Bridge OpenCode ACP usage updates into the existing token-count pipeline so the web status bar can show live context and cache information without a separate UI path.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(opencode): restrict plan mode to remote, rollback reasoning effort on failure
- Block local OpenCode plan startup (tools not enforced in local path)
- Allow remote OpenCode plan only (ACP permission handler denies tools)
- Guard web /permission-mode endpoint for local OpenCode plan sessions
- Rollback session reasoning effort when OpenCode rejects set_config_option
- Wire rollback callback through opencodeLoop to runOpencode closure
- Add tests: local plan rejected, remote plan allowed, web guard, effort rollback
* fix(web): auto-retry OpenCode models query to populate model selector without refresh
- Retry early failures (RPC may still be registering on new sessions)
- Poll briefly until availableModels is non-empty
- Stop polling once model options are discovered
- Add tests for retry/poll/stop policy
* fix(opencode): cap model discovery polling
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(web): reset scroll restoration cache when sessionStorage is full
TanStack keeps scroll state in RAM; pruning only the JSON blob did not stop
repeat quota throws. On persist failure for the scroll key, clear the library
cache when guarding real sessionStorage. Treat any write error on that key
(not only QuotaExceededError-shaped) as recoverable.
Co-authored-by: Cursor <cursoragent@cursor.com>
* test(web): cover scroll cache hard reset and non-quota recovery
Exercise TanStack scrollRestorationCache reset on real sessionStorage,
mock-storage isolation, and generic storage write failures for #708.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(web): sync pruned scroll cache and avoid hard-reset recursion
After a successful storage prune, align TanStack's in-memory
scrollRestorationCache with the trimmed payload. Temporarily unwrap
sessionStorage.setItem when writing through the library cache so hard
reset cannot recurse through the guard (Codex PR review #707).
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Store the last known permission mode in session metadata so YOLO and other
modes survive hub restarts, resume after archive, and reconnect without
waiting for CLI keepalive.
Co-authored-by: Cursor <cursoragent@cursor.com>
When a session is open, the web app now keeps an always-on all:true SSE
connection for sidebar session-updated events while using a second
session-scoped stream for message delivery. Also bump session activity on
hub sendMessage so web-originated sends refresh list timestamps.
Fixestiann/hapi#693
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(web): embed agent text in voice ready event for readback
Voice onReady now extracts the last speakable assistant message and
embeds it in the ready inject so ConvAI can summarize without the user
re-prompting. Also formats Codex/Cursor stream-json messages for live
context updates and session history.
Co-authored-by: Cursor <cursoragent@cursor.com>
* test(web): use domain-neutral voice formatter fixtures
Replace jellybot/subtitle dogfood strings in tests with generic examples.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(web): guard extractSpeakableFromContent for non-arrays in formatMessage
extractSpeakableFromContent also handles content arrays (joins text items),
so calling it unconditionally before the existing array loop caused mixed
text+tool_use payloads to return early without formatting the tool_use item.
Guard with !isContentArray so the loop handles arrays as before.
Adds regression test: mixed text+tool_use array must produce both the text
and the tool-call line (was red before this fix).
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(web): narrow extractSpeakableFromContent to codex type only
The helper matched any object with a string type and a data property,
so sendSessionEvent({ type: 'message', message }) events (which arrive as
{ type: 'event', data: { type: 'message', message } }) were falsely formatted
as speakable assistant text and could be selected as the ready readback.
Narrow the Codex path to content.type === 'codex' as the comment already states.
Adds regression test: session status event must return null from formatMessage.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: HAPI <noreply@hapi.run>
