* feat(web): session header files and outline view toggles
Files and outline icons in SessionHeader act as depressed toggles; files
view shares the session header and places refresh beside the search box.
Co-authored-by: Cursor <cursoragent@cursor.com>
* chore(web): add Playwright handoff script for session view toggles
Supports new-feature-intake visual gate: files toggle pressed + refresh beside search.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(dev): handoff script avoid networkidle on live hub SSE
HAPI keeps connections open on :3006; domcontentloaded is the correct wait.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(web): place filesystem refresh outside search field
Refresh is a sibling of the search pill, not inside it, so the control
is visually and structurally separate from file search.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* test: reproduce issue #901 (active-only filter + paginated show more)
* fix: active-only session filter + paginated 'Show N more' (closes#901)
Add a persisted 'Active sessions only' toggle in Settings -> Display that
hides inactive sessions in the sidebar while keeping the selected session
visible. Change 'Show N more' to reveal one batch (preview-limit size) per
click instead of expanding every hidden session at once, with 'Show less'
to collapse back to the initial preview.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
---------
Co-authored-by: HAPI <noreply@hapi.run>
* fix(web): persist file explorer expanded tree and scroll position across navigation
Expanded folder state and scroll position in the Directories tab were
stored only in local React state, so navigating to a file and back
would reset the tree to the root and scroll to top.
Now both are saved to sessionStorage (keyed by sessionId) on every
change and restored on remount, so the explorer resumes exactly where
the user left off.
Closes#910
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(web): key DirectoryTree by sessionId to prevent stale expanded state across sessions
When navigating between sessions, React can reuse the same DirectoryTree
instance. The useState lazy initializer only runs on first mount, so the
tree would hydrate with the wrong session's expanded set and then
overwrite the new session's storage key.
Adding key={sessionId} forces a fresh mount per session.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
---------
Co-authored-by: HAPI <noreply@hapi.run>
Adds a download icon button to the file viewer toolbar (next to copy-path).
Clicking it decodes the existing base64 file content into a Blob and triggers
a browser download — no new backend endpoint required. Works for text, binary,
and image files. Button is hidden until the file has loaded successfully.
Closes#924
via [HAPI](https://hapi.run)
Co-authored-by: HAPI <noreply@hapi.run>
* feat(web): Web Share Target -> composer attachment preload
PWA manifest now declares a `share_target` so Android Chrome surfaces
HAPI in the system share sheet for any app (Photos, Files, browser).
Pipeline on share:
1. Service worker intercepts POST /share, parses the multipart payload
(title/text/url + N files), persists it in IndexedDB under a
transfer id, and 303-redirects to /share?id=<id>. The 303 forces
Chrome to convert the POST into a GET so the SPA route mounts.
2. New /share route loads the transfer, previews the content, and
lets the user pick a recent active session (top 5 by activeAt) or
a "+ New session". Tapping a session stashes the transfer id in
sessionStorage and navigates to /sessions/:id.
3. SessionChat mounts a ShareSeedConsumer once the AssistantRuntime
is up; it consumes the pending transfer once per mount, seeds
composer text + per-file attachments via the existing
attachmentAdapter, then deletes the IDB row so a refresh of the
session page does not replay the upload.
The whole feature reuses the existing /sessions/:id/upload endpoint;
no hub or shared changes.
Limitations (also disclosed in the PR body):
- PWA must be installed; Android Chrome only registers share_target
on install. iOS Safari ignores the manifest field entirely.
- File MIME accept list is broad (`*/*` fallback); some Chrome
versions still filter despite this.
Tests:
- shareTransfer.test.ts (8) covers payload parse, multi-file order,
type fallback, ingest redirect shape and error propagation.
- sharePendingState.test.ts (3) covers atomic consume + overwrite.
Closes: pending upstream issue (filed before PR per intake doc).
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(web/share): freeze picker session list at mount, sort by updatedAt, drop top-5 cap
The /share picker was visually re-shuffling under the operator's finger
as SSE events rolled in: every session metadata patch refreshed the
React Query cache, the useMemo recomputed, and items reordered (often
within a second of opening the share sheet). Sort key was activeAt,
which heartbeats every few seconds while a session is connected,
making the noise floor even higher.
Three changes:
- Snapshot the active-session list once when sessions finish loading
via useState + a deferred useEffect. The picker is a one-shot
interaction; closing the share sheet and re-sharing produces a
fresh snapshot, so freezing for the duration of the picker view is
the right trade.
- Sort by updatedAt desc to match SessionList's canonical "most
recent interaction first" order. updatedAt only moves on
user-meaningful events, not heartbeats.
- Drop the TOP_SESSIONS=5 cap. The picker is already inside an
app-scroll-y container, so showing all active sessions and letting
the operator scroll matches the operator's mental model better
than an arbitrary truncation.
Per operator dogfood report: "list of recent sessions is constantly
updating; should be just a scrollable list, from most recent
interaction to not."
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(web/share): base-aware share_target paths for subpath PWA deploys
Manifest share_target.action, SW POST matching, and ingest 303 redirects
were hard-coded to /share. Standalone builds with --base /<repo>/ put
scope/start_url under the subpath but left the share action at origin
root, so Chrome posted outside the SW scope and the handler never ran.
Extract shareTargetPathnameFromBase() (used at build time in
vite.config.ts and at runtime via import.meta.env.BASE_URL in sw.ts and
shareTransfer.ts). Normalizes base to a trailing slash before URL
resolution so /repo and /repo/ both resolve to /repo/share.
Addresses upstream PR #933 review (Major).
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(web/share): defer sessionStorage arm until new session spawn succeeds
The "+ New session" picker path called setSharePendingTransfer before a
session existed. Cancel, spawn failure, or backing out left a stale id in
sessionStorage that the next unrelated SessionChat mount would consume.
Pass shareTransferId via /sessions/new search params instead; arm the
consumer only in handleSuccess after spawn, and delete the IDB row on
cancel.
Addresses upstream PR #933 review (Major).
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(web/share): append share text to existing composer draft
ShareSeedConsumer called setText(seedText) unconditionally, clobbering
per-session drafts restored by useComposerDraft from sessionStorage.
Merge share title/text/url after any in-composer text or saved draft,
joined with a blank line. Pass sessionId into ShareSeedConsumer so
getDraft() can be consulted when the composer is still empty.
Addresses upstream PR #933 review (Major).
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(web/share): preserve shareTransferId through /browse detour
New-session spawn from the share picker could lose shareTransferId when
the operator opened /browse to pick a folder: handleChooseFolder and
BrowsePage handleStartSession dropped the search param, so handleSuccess
never armed the composer consumer.
Thread shareTransferId through browseRoute search validation and both
navigation hops.
Addresses upstream PR #933 review (Major).
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(web/share): consume pending transfer in effect for StrictMode
ShareSeedConsumer called consumeSharePendingTransfer during render.
React.StrictMode double-invokes render in dev; the discarded pass
deleted the sessionStorage key before the committed render seeded.
Move consume into a mount-only useEffect and gate the seed effect on
transferReady.
Addresses upstream PR #933 review (Minor).
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* test: reproduce issue #866
* feat(web): OLED Black theme + per-appearance custom colors (closes#866)
Add an explicit OLED Black appearance (true #000 canvas, border-based
elevation) alongside system/dark/light, and a curated "key color"
customizer. Each key color (background, surface, text, hint, accent,
border, user bubble) cascades to its --app-* tokens and is stored per
appearance so a color tuned for light never leaks onto pure black.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
---------
Co-authored-by: HAPI <noreply@hapi.run>
When the terminal page loaded, focus stayed on the page body — the user
had to click/tap into the xterm area before keystrokes were captured.
Add terminal.focus() at the end of handleTerminalMount so focus lands
inside the terminal as soon as the xterm instance is attached to the DOM.
The quick-input buttons already call terminalRef.current?.focus() after
each press; this extends the same pattern to the initial mount.
Closes#875
Co-authored-by: Cursor <cursoragent@cursor.com>
When the shell exited inside the remote terminal view, the page kept
showing a banner ("Terminal exited with code 0.") and left the user
stranded with no obvious next step. On mobile this is awkward, and it
does not match the muscle memory from native terminal emulators where
typing `exit` closes the tab/window.
Schedule a goBack() shortly after `terminal:exit` fires so the user
briefly sees the exit info, then returns to the session chat (same
destination as the existing back arrow via useAppGoBack).
The auto-close timer is cleared on unmount, on sessionId change, and
when the socket reconnects after a transient drop so a stale exit
event cannot navigate away from a freshly reconnected terminal.
Closes#856
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(voice): voice personality, picker catalog, and prompt layer foundation
- voicePickerCatalog.ts: per-backend voice lists for Gemini and Qwen with
resolve helpers (resolveGeminiLiveVoice, resolveQwenRealtimeVoice)
- voicePersonality.ts: VoicePersonalityPreferences schema, presets, composed
system prompt with identity/character/response-length layers
- voicePromptLayers.ts: buildResolvedVoiceSystemPrompt, preset delivery snippets
- voiceSystemPromptParam.ts: hub-side base64url decode for ?systemPrompt=
- voicePickerPreferences.ts, voicePersonalitySession.ts: browser-side encode,
decode, and storage helpers
- useVoicePersonality: React hook for preferences persistence
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): preset delivery included when non-balanced preset selected; restore test typecheck
- isDefaultVoicePersonality: add preset check so warm/calm/direct presets
trigger the delivery snippet instead of being treated as default
- web/tsconfig.json: remove test file exclusion from typecheck (restoring
strict coverage of test code); fix resulting type error in mock declaration
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): include use_speaker_boost in ElevenLabs TTS override payload
The checkbox persisted the pref but ttsDiffersFromDefault and
buildElevenLabsTtsOverride both omitted it, so the setting was never
sent to the agent.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* test(voice): update speaker_boost test to assert it IS included in override
The previous test asserted use_speaker_boost was omitted; now it's
correctly included in the TTS payload.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): authorize use_speaker_boost in ElevenLabs override schema
Add use_speaker_boost to both the VoiceAgentConfig tts override type
and the buildVoiceAgentConfig() platform_settings so the field is
accepted by the ElevenLabs agent runtime.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): propagate full language code through composed prompt, not just zh
getDefaultVoiceSystemPrompt and resolveComposedVoiceSystemPrompt were
filtering language to zh-only before passing to composeVoiceAgentPrompt.
Now append buildVoiceLanguageBlock(language) after composition so French,
Spanish, Japanese etc. reach Gemini/Qwen sessions correctly.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): only append language block when language explicitly set
Building language block unconditionally when no language is given
caused getDefaultVoiceSystemPrompt() to diverge from VOICE_SYSTEM_PROMPT.
Only append the block when a code is explicitly provided.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): always include language block for Gemini/Qwen in composed prompt
When auto-detect is on (language=undefined), the composed prompt sent
via hub proxy was losing the language auto-detect instruction because
the block was only added when language was explicitly set.
Now: ElevenLabs skips the block (has its own language field); Gemini/Qwen
always include it — undefined produces the auto-detect block, an explicit
code produces the appropriate language instruction.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
---------
Co-authored-by: HAPI <noreply@hapi.run>
* feat(voice): pluggable voice backend with Gemini Live & Qwen Realtime
Rebased from Overbaker/hapi#401 onto current main. Adds a pluggable voice
backend architecture that extends the existing ElevenLabs integration:
- Gemini 2.5 Live (gemini-live): Google real-time audio via WebSocket
with full function calling (messageCodingAgent, processPermissionRequest)
- Qwen Realtime (qwen-realtime): Alibaba DashScope via hub WebSocket
proxy (browser cannot set Authorization header directly)
- VoiceBackendSession: dynamic backend selector with React.lazy loading,
gates voice button until backend module is registered
- Hub WS proxies: JWT-authenticated /api/voice/gemini-ws and
/api/voice/qwen-ws endpoints in Bun.serve, with message queueing during
upstream connect to prevent dropped setup frames
- AudioWorklet pipeline: inline Blob URL recorder, 24 kHz PCM player,
serial tool call execution, AudioContext created in user gesture for mobile
- Backend discovery: GET /voice/backend + POST /voice/gemini-token /
POST /voice/qwen-token hub routes; frontend auto-detects active backend
Merge notes:
- Rebased 135 upstream commits cleanly; HappyComposer keeps upstream's
configurable enter-behavior setting (supersedes hard-coded Ctrl+Enter)
- Converted gemini test files from bun:test to vitest (web package uses vitest)
- All 221 hub tests and 636 web tests pass; TypeScript clean
* fix(voice): restore user mic mute state after Gemini turn completes
turnComplete handler was unconditionally calling setMuted(false), which
re-enabled the mic track even when the user had manually muted. Now
restores to state.micMuted instead.
* fix(voice): remove hard-coded Chinese language from Gemini backend
buildGeminiLiveConfig was appending VOICE_CHINESE_LANGUAGE_BLOCK which
forced Gemini to always respond in Mandarin regardless of user locale.
Gemini now uses the neutral base prompt and responds in the language the
user speaks to it, consistent with the ElevenLabs behaviour.
* fix(voice): reset modelSpeaking in cleanup to unblock mic on restart
If the session closes while Gemini is mid-speech, cleanup() left
state.modelSpeaking=true. The next startSession() would then drop all
mic audio in sendAudioChunk() until a model turn eventually flipped
the flag — effectively deaf until page reload.
* fix(voice): guard stale close handlers in Gemini and Qwen sessions
ws.onclose operated on module-level state.ws, not the socket that fired
the event. A rapid stop/restart could cause the old socket's onclose to
call cleanup() after the new socket was assigned, tearing down the live
session. Guard with `if (state.ws !== ws) return` before cleanup.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): remove hard-coded Chinese language from Qwen backend
Matches the Gemini fix — both backends now use VOICE_SYSTEM_PROMPT
without the Chinese language block, giving consistent English-default
behaviour across all non-ElevenLabs backends.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* feat(voice): proactive/reactive toggle in voice settings
Adds a "Proactive voice" toggle (default: off = reactive) to the Voice
Assistant settings section.
Reactive (default): initial context and agent-ready events are fed
silently; the assistant waits for the user to speak first.
Proactive: original behaviour — Gemini/Qwen narrate context on connect
and speak unprompted when the agent finishes a task. ElevenLabs is also
affected via onReady sending a user message rather than a silent update.
Covers all three backends uniformly. localStorage key: hapi-voice-proactive.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): normalize WS close codes, drop barrel re-exports, fix SSE visibility
- hub/server.ts: add toClientCloseCode() to normalize reserved upstream
close codes (1005/1006/1015) to 1011 before forwarding to browser;
abnormal upstream drops (1006) would otherwise throw on clientWs.close()
and leave the browser socket open
- realtime/index.ts: remove static GeminiLiveVoiceSession and QwenVoiceSession
barrel exports; VoiceBackendSession lazy-imports both, so barrel re-exports
created static dependencies that defeated the intended code-split
- App.tsx: gate global useVisibilityReporter on !sessionEventSubscription so
the always-on SSE connection does not suppress native Web Push notifications
for sessions the user is not currently viewing
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): respect language setting in Gemini/Qwen; fix voice-start toggle label
- buildGeminiLiveConfig() now accepts optional language param; appends
VOICE_CHINESE_LANGUAGE_BLOCK only when language === 'zh'
- GeminiLiveVoiceSession passes config.language through
- QwenVoiceSession conditionally builds basePrompt from language setting
- Fixes silent no-op when user selects Chinese in voice settings on
Gemini/Qwen backends (was ElevenLabs-only)
- Rename voice-start toggle label to 'Start voice session with summary'
- Fix description: clarifies the choice is about session-open behaviour
(summary vs greeting), not ongoing narration
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): send greeting trigger in reactive mode for Gemini
Gemini Live has no built-in first-message like ElevenLabs agents do;
without an explicit turnComplete:true it sits silently. In reactive mode
(default, toggle off) now sends a greeting instruction after any silent
context feed so Gemini introduces itself and invites the user to speak.
Proactive mode is unchanged: the context summary is the opening speech.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): suppress Gemini self-identification and context leak in greeting
- VOICE_SYSTEM_PROMPT: explicit instruction never to call itself Gemini,
Google, or any underlying model/provider name — always HAPI
- Greeting trigger text: instruct to greet as HAPI only, suppress model
name and any reference to context/recent activity in the opening line
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): address code review findings — error handling, proxy, audio
Gemini + Qwen client:
- onerror now sets setupDone/sessionReady and nulls state.ws before
calling reject(), so the stale-close guard trips in onclose and
prevents a duplicate statusCallback('error') on WS failure
Gemini client:
- Proactive mode with no initialContext now falls through to the
greeting trigger instead of sitting silently
- Remove unused handleBargeIn callback (dead code)
Qwen client:
- Add input_audio_sample_rate: 16000 to session.update so PCM rate
is declared explicitly rather than relying on DashScope's default
Hub proxy:
- Remove no-op ternary in Gemini flush loop and message handler
(typeof x === 'string' ? x : x); use upstream.send(msg) directly
- Qwen onerror now calls upstreamMap.delete() before closing client,
eliminating the stale map entry window
- Align Qwen hub fallback model string with QWEN_REALTIME_MODEL
constant ('qwen3-omni-flash-realtime')
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): trailing-slash WS URL, Qwen session.update schema
hub/voice.ts:
- Replace string-concat WS URL construction with buildVoiceWsUrl() which
uses URL API to set protocol/pathname cleanly — fixes double-slash when
HAPI_PUBLIC_URL has a trailing slash (would silently skip the proxy route)
QwenVoiceSession.tsx:
- Wrap tool definitions in {type:'function', function:{...}} as required
by Qwen-Omni realtime schema — previous flat shape caused session.update
rejection before audio capture could start
- Use pcm16/pcm24 audio formats matching DashScope spec; remove
input_audio_sample_rate (encoded in format name)
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): await audio capture before setMuted; sanitize upstream close codes
GeminiLiveVoiceSession + QwenVoiceSession:
- startAudioCapture() is now async and awaits recorder.start() before
calling setMuted() — previously setMuted ran before getUserMedia resolved
so a session restarted while muted would open the mic anyway
- statusCallback('connected') now fires after audio is ready
- setMuted() called unconditionally (not just when true) to correctly
apply saved state in either direction
hub/src/web/server.ts:
- Both Gemini and Qwen close() handlers now pass the client code through
toClientCloseCode() before forwarding to upstream — prevents reserved
codes (e.g. 1006) from causing WebSocket.close() to throw and leave
the upstream session open until provider timeout
- Reason string capped at 123 bytes (WebSocket protocol limit)
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): wrap startAudioCapture in try/catch to propagate mic errors
An unhandled rejection inside the async onmessage callback does not
propagate to the outer startSession Promise — the UI hangs on
'connecting' and the provider socket stays partially open. Wrapping
the await in try/catch calls cleanup()/statusCallback('error')/reject()
so failures surface correctly.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): propagate backend discovery failure instead of silently falling back to ElevenLabs
fetchVoiceBackend no longer catches errors and defaults to 'elevenlabs' — any
network or server failure now throws so VoiceBackendSession can surface it via
onStatusChange('error', ...) rather than silently mounting the wrong backend.
VoiceBackendSession also resets backend state to null when api changes, so
a stale ElevenLabs registration from a prior discovery cannot persist into
a new session.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): throw on unrecognised backend value instead of silently falling back to ElevenLabs
Unknown backend strings (future values, typos) now throw rather than defaulting
to elevenlabs, closing the narrow remaining form of the original misrouting bug.
Also removes the unnecessary `as VoiceBackendResponse` cast.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): add Qwen greeting/proactive trigger; fix socket buffer for base64 uploads
Qwen session.updated handler now sends the same proactive summary or greeting
trigger that Gemini does — previously it started silently in both proactive and
reactive modes.
maxHttpBufferSize raised to 68 MiB to account for base64 expansion: 50 MiB
decoded files become ~66.7 MiB as base64 JSON, so the previous 55 MiB ceiling
would disconnect uploads above ~41 MiB before they reached the CLI.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): replace unsupported conversation.item.create with session.update for Qwen text
Qwen's realtime API only supports conversation.item.create for function_call_output.
Sending it with type:'message' for greetings/context was invalid and could fail
before the user spoke.
sendTextMessage and sendContextualUpdate now update session instructions via
session.update (accumulating context into the system prompt) and trigger
response.create only when a spoken reply is needed — matching Qwen's supported
client event surface.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): guard session.updated re-entry and reset config on session start
session.updated now returns early after the first ack — subsequent session.update
calls (instruction appends) also echo session.updated but must not re-trigger
audio capture or the greeting path.
currentSessionConfig is now reset to null at the top of startSession so a stale
config from a failed previous session cannot leak into the new one.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): assert wsUrl presence for Gemini proxy connections
Without this guard, a missing wsUrl in the hub token response would
silently attempt to connect directly to Google with "proxied" as the
API key — producing a confusing auth failure instead of a clear error.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): correct Qwen audio formats and default voice
DashScope realtime API accepts only 'pcm' for both input and output
audio formats. The pcm16/pcm24 values caused session.update rejection
before audio capture could start, leaving the Qwen backend unusable.
Also updates the default voice from Mia (not in the qwen3-omni-flash-
realtime voice list) to Cherry, which is documented as supported.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): close AudioContext on failed voice session start
Failed token fetch, microphone denial, or WebSocket error during
setup left state.playbackContext open. Each failure path now calls
cleanup() before throwing/rejecting, preventing AudioContext leaks
on mobile browsers with hard limits on concurrent contexts.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* chore: restore non-voice files to upstream/main state
Reverts changes to files that shouldn't differ from upstream:
- .gitignore: remove fork-only AGENTS.local.md entry
- web/src/App.tsx: restore dual-subscription SSE pattern (scope-aware)
- web/src/hooks/useSSE.ts: restore SSEScope/scope parameter
- web/src/hooks/useSSE.test.ts: restore (was accidentally deleted)
- web/src/lib/appSseSubscriptions.ts: restore (was accidentally deleted)
- web/src/lib/appSseSubscriptions.test.ts: restore (was accidentally deleted)
- hub/src/sync/syncEngine.ts: restore (off-topic change)
* fix(voice): harden Gemini and Qwen WS proxies against client abuse
Hub sends HAPI-owned Gemini setup on proxy connect and rejects client
setup frames. Qwen proxy always uses QWEN_REALTIME_MODEL instead of a
client query parameter. Shared buildGeminiLiveSetupMessage() keeps wire
format in one place.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(voice): harden Qwen proxy — hub-owned setup, client frame allowlist
Mirror the Gemini proxy security model for Qwen:
- Hub sends initial session.update (voice/tools/instructions) on upstream
connect so the browser cannot override config fields.
- Proxy message() now calls isQwenSafeClientFrame() and closes the
connection (1008) if a client session.update touches any field other
than 'instructions' (blocks tool/voice/modality overrides).
- QwenVoiceSession no longer sends session.update on session.created;
it waits for the hub-relayed session.updated and then sends only
instruction-only updates for context/proactive content.
- Language passed as query param (?language=zh) so hub builds the
correct Chinese system prompt without a client-supplied session.update.
- buildQwenSessionUpdateMessage() and isQwenSafeClientFrame() added to
@hapi/protocol/voice; 9 new unit tests cover filter edge cases.
* fix(voice): respect Qwen session.created→session.update protocol ordering
DashScope requires session.update to be sent AFTER session.created is
received, not immediately on WebSocket open. Previously the hub sent
session.update in upstream.onopen, which violated this ordering and
risked the config being processed in an uninitialized session context.
Add pendingSetupMap to buffer the hub-owned session.update payload.
The onmessage handler now relays session.created to the browser first,
then immediately sends the pending session.update to DashScope — matching
the protocol ordering the old browser-side code used (which waited for
session.created before sending session.update).
Also remove maxHttpBufferSize from the socket.io Engine config. That
setting is unrelated to voice backends; upstream/main had no such limit
set and it is not introduced by this PR.
* fix(voice): use Realtime tool shape for Qwen session.update (not chat-completions)
Qwen Realtime session.update expects tools as flat objects:
{ type: 'function', name, description, parameters }
The previous code used the chat-completions shape:
{ type: 'function', function: { name, description, parameters } }
DashScope may reject session.update or silently ignore tools with the
nested shape, causing tool calls to fail at runtime. Fix applied in
buildQwenSessionUpdateMessage(); test updated to assert flat shape and
that no nested `function` key is present.
* fix(voice): update Qwen Realtime model, voice, and endpoint for intl service
Live-tested against DashScope international API:
- Model: qwen3-omni-flash-realtime → qwen3.5-omni-flash-realtime
(previous model ID did not exist on DashScope)
- Default voice: Cherry → Tina
(confirmed from session.created response on qwen3.5-omni-flash-realtime)
- Default WS base: dashscope.aliyuncs.com → dashscope-intl.aliyuncs.com
(international accounts use the -intl endpoint; China endpoint rejects
international API keys; QWEN_REALTIME_WS_URL env var still overrides)
* fix(voice): correct Qwen text injection and generalise language handling
Two dogfooding fixes verified against live Qwen Realtime session:
sendTextMessage: switch from instruction-injection to conversation.item.create
Qwen Realtime requires a user conversation item before response.create.
The previous approach (updateInstructions + response.create) produced
"input messages do not contain elements with role user" errors. Now sends
{type:message, role:user, content:[{type:input_text}]} then response.create.
sendContextualUpdate is unchanged (instruction-only, no response trigger).
Language handling: replace zh-only branch with buildVoiceLanguageBlock()
Previously, only language='zh' added any instruction; all other languages
(including English) sent no language block, causing Qwen to drift to Chinese.
buildVoiceLanguageBlock() now covers three cases:
- 'zh'/'zh-*': existing Chinese block (unchanged)
- explicit code ('en','es','fr',...): "Always respond in [Language]"
- undefined/auto: "Detect the user's language and maintain it"
Applied to buildGeminiLiveConfig, buildQwenSessionUpdateMessage, and the
client-side currentInstructions mirror in QwenVoiceSession.
Also removes the Gemini hub proxy's zh-only filter, which was discarding
explicit language selections other than Chinese.
* fix(hub): gate Gemini client frames until upstream setupComplete
Hub sends its owned setup on upstream open, then waits for Google's
setupComplete acknowledgment before flushing queued client frames.
isGeminiSetupCompleteFrame() detects the {"setupComplete":{}} message;
message() queues instead of forwarding while pendingMap is live.
Addresses the repeated Major finding from bot review on PR #743.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(hub): cap Gemini setup-window pending queue at 1 MiB
An authenticated client could flood the queue between upstream.onopen
and Google's setupComplete acknowledgment. Add pendingBytesMap tracking
and close with 1009 if the budget is exceeded.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(gemini): pass all language codes to hub proxy, not just zh
Language selection for French, Spanish, Japanese etc. was silently
dropped — only 'zh' was forwarded as a query param.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
* fix(voice): expand LANGUAGE_NAMES to cover full ElevenLabs language set
Codes like 'no', 'da', 'fi', 'pt-br', 'bg', 'ro', 'cs', 'el', 'ms',
'tl', 'uk', 'hu', 'hr', 'sk' were falling through to raw-code prompts
("Always respond in no"). Now resolve to proper display names.
via [HAPI](https://hapi.run)
Co-Authored-By: HAPI <noreply@hapi.run>
---------
Co-authored-by: HAPI <noreply@hapi.run>
Co-authored-by: Cursor <cursoragent@cursor.com>
crypto.randomUUID is only exposed in secure contexts (HTTPS or
localhost). When the web app is served over HTTP on a LAN IP the
attachment adapter, toast provider, message localId helper, file
attachment metadata and terminal id creation all call
crypto.randomUUID() synchronously and throw TypeError, so the UI
silently does nothing (e.g. the file picker opens and closes with no
chip).
Add a small web/src/lib/randomId helper that tries crypto.randomUUID
first, then falls back to crypto.getRandomValues-derived UUID v4,
and finally to a Date.now/Math.random string for very old
environments. Route all five call sites through it. Output format is
identical for secure contexts and UUID v4 for the getRandomValues
path, so existing DB/SSE/RPC consumers see the same shape.
* feat(web): add appearance setting (follow system / dark / light)
Add user-facing appearance preference to the settings page, allowing
users to choose between Follow System, Dark, and Light themes. The
preference is persisted to localStorage and takes priority over
automatic detection in the existing theme pipeline.
* fix(web): update theme on cross-tab appearance change
The storage event handler only updated React state without calling
updateScheme(), leaving data-theme and useTheme subscribers stale.
* fix(web): move cross-tab appearance sync to global initializeTheme
The storage event listener was inside useAppearance(), which is only
mounted on the settings page. Other pages never received cross-tab
theme updates. Move the listener into initializeTheme() so all pages
respond to appearance changes from other tabs.
* feat(web): add About section to settings page
- Add website link to hapi.run
- Display app version from CLI package
- Display protocol version from shared module
- Add Vitest testing setup with settings page tests
🤖 Generated with Claude Code
* test(web): add tests for website link and i18n key usage
Address residual risks mentioned in PR review:
- Test website link URL and security attributes (target, rel)
- Verify correct i18n keys are used for About section via spy
Simplify test setup by using real I18nProvider and en locale.
🤖 Generated with Claude Code
- Add Settings > Display > Font Size selector (80/90/100/110/120%)
- Persist preference (hapi-font-scale) and apply globally via --app-font-scale
- Normalize main UI typography to match Settings
Add support for users to select voice language preference, which is passed through to ElevenLabs agents via platform settings overrides. Includes new language mapping utilities, UI controls in settings page, and updated voice session initialization.
* feat: expand terminal quick keys
Add navigation keys plus ctrl/alt modifiers for the web terminal.
* feat: make terminal modifiers sticky
Keep Ctrl/Alt active until toggled off, Termux-style.
* style: improve terminal quick key buttons
Boost active contrast and adjust sizing for mobile.
* feat: add terminal key popups
Use long-press popups for / and - keys.
* fix: make terminal modifiers mutually exclusive
* fix(web): keep soft keyboard open on terminal buttons
* fix(web): allow mouse clicks on terminal quick keys
* refactor(web): simplify terminal quick input modifier handling
Remove the applyModifiers callback and inline modifier state application.
Add automatic modifier reset after quick input when appropriate.
* refactor(web): extract dispatchSequence helper for terminal input handling
Create a reusable dispatchSequence callback to handle modifier application
and reset logic for both terminal data input and quick input handlers.
---------
Co-authored-by: weishu <twsxtd@gmail.com>
Previously, useTerminalSocket used a relative path `/terminal` for Socket.IO,
which would connect to the frontend host instead of the actual HAPI server.
Now it uses the baseUrl from context to connect to the correct server, enabling
scenarios like hosting the frontend on GitHub Pages while the server runs elsewhere.
The selected button was using bg-[var(--app-link)] with white text, which
resulted in invisible text in dark mode (white background + white text).
Changed to use bg-[var(--app-button)] and text-[var(--app-button-text)]
with opacity-80 for proper contrast in both light and dark themes.
The old decodeBase64 implementation tried atob() first which returns
garbled text for UTF-8 content (e.g., Chinese characters), and only
fell back to correct UTF-8 decoding when atob() threw an exception
(which it doesn't for valid base64).
Changes:
- Add encodeBase64/decodeBase64 utilities to web/src/lib/utils.ts
using TextEncoder/TextDecoder for proper UTF-8 support
- Update file.tsx to use shared decodeBase64 instead of buggy local impl
- Update files.tsx to use shared encodeBase64 instead of deprecated
escape/unescape approach
Add p-3 class to terminal page container to create spacing between
the terminal and page edges, matching the padding used elsewhere on
the page (header, error messages, etc.).
- Add CLI-side terminal management via Bun.Terminal with TerminalManager
- Implement server-side Socket.IO proxy for terminal I/O between web and CLI
- Create web terminal UI component with xterm.js and support for resize/reconnect
- Add terminal route and navigation button in session chat
- Include comprehensive terminal implementation plan and architecture docs
Introduces two reusable loading components to standardize loading states:
- Spinner: Base spinner with accessibility support (role, aria-label)
- LoadingState: Semantic loading indicator combining spinner and label
Updates loading patterns across the app:
- Full-screen/block loading now uses LoadingState (App, router, files, file)
- Inline button loading states show spinner + text with aria-busy
- Message list loading replaced with MessageSkeleton component
- Removes duplicate SpinnerIcon definition from PermissionFooter
Improves UX and accessibility:
- Loading screens centered and full-height
- Consistent ellipsis character (…)
- Semantic accessibility (role="status", aria-live, aria-busy)
- Adds CSS variables for banner styling
Adds maxWidth.content: '720px' to tailwind.config.ts for centralized width management across the web app. Replaces all hardcoded max-w-[720px] references with the new max-w-content class. Adds missing max-width constraints to file.tsx, files.tsx, and SessionList.tsx. Moves border-b dividers from outer full-width divs to inner max-w-content divs for consistent visual hierarchy.
This fixes an issue where viewport-fit=cover and black-translucent statusbar
configuration allowed content to extend under the iOS statusbar. Added env(safe-area-inset-top)
CSS handling to all main layout components:
- SessionHeader with pt-[env(safe-area-inset-top)]
- SessionsPage, MachinesPage, SpawnPage with safe-area padding
- FilesPage and FilePage headers with safe-area padding
