Files
hapi/shared/src/voicePromptLayers.ts
T
a812a51dd7 feat(voice): backend voice picker + advanced controls behind disclosure (#742) (#743)
* feat(voice): voice personality, picker catalog, and prompt layer foundation

- voicePickerCatalog.ts: per-backend voice lists for Gemini and Qwen with
  resolve helpers (resolveGeminiLiveVoice, resolveQwenRealtimeVoice)
- voicePersonality.ts: VoicePersonalityPreferences schema, presets, composed
  system prompt with identity/character/response-length layers
- voicePromptLayers.ts: buildResolvedVoiceSystemPrompt, preset delivery snippets
- voiceSystemPromptParam.ts: hub-side base64url decode for ?systemPrompt=
- voicePickerPreferences.ts, voicePersonalitySession.ts: browser-side encode,
  decode, and storage helpers
- useVoicePersonality: React hook for preferences persistence

via [HAPI](https://hapi.run)

Co-Authored-By: HAPI <noreply@hapi.run>

* fix(voice): preset delivery included when non-balanced preset selected; restore test typecheck

- isDefaultVoicePersonality: add preset check so warm/calm/direct presets
  trigger the delivery snippet instead of being treated as default
- web/tsconfig.json: remove test file exclusion from typecheck (restoring
  strict coverage of test code); fix resulting type error in mock declaration

via [HAPI](https://hapi.run)

Co-Authored-By: HAPI <noreply@hapi.run>

* fix(voice): include use_speaker_boost in ElevenLabs TTS override payload

The checkbox persisted the pref but ttsDiffersFromDefault and
buildElevenLabsTtsOverride both omitted it, so the setting was never
sent to the agent.

via [HAPI](https://hapi.run)

Co-Authored-By: HAPI <noreply@hapi.run>

* test(voice): update speaker_boost test to assert it IS included in override

The previous test asserted use_speaker_boost was omitted; now it's
correctly included in the TTS payload.

via [HAPI](https://hapi.run)

Co-Authored-By: HAPI <noreply@hapi.run>

* fix(voice): authorize use_speaker_boost in ElevenLabs override schema

Add use_speaker_boost to both the VoiceAgentConfig tts override type
and the buildVoiceAgentConfig() platform_settings so the field is
accepted by the ElevenLabs agent runtime.

via [HAPI](https://hapi.run)

Co-Authored-By: HAPI <noreply@hapi.run>

* fix(voice): propagate full language code through composed prompt, not just zh

getDefaultVoiceSystemPrompt and resolveComposedVoiceSystemPrompt were
filtering language to zh-only before passing to composeVoiceAgentPrompt.
Now append buildVoiceLanguageBlock(language) after composition so French,
Spanish, Japanese etc. reach Gemini/Qwen sessions correctly.

via [HAPI](https://hapi.run)

Co-Authored-By: HAPI <noreply@hapi.run>

* fix(voice): only append language block when language explicitly set

Building language block unconditionally when no language is given
caused getDefaultVoiceSystemPrompt() to diverge from VOICE_SYSTEM_PROMPT.
Only append the block when a code is explicitly provided.

via [HAPI](https://hapi.run)

Co-Authored-By: HAPI <noreply@hapi.run>

* fix(voice): always include language block for Gemini/Qwen in composed prompt

When auto-detect is on (language=undefined), the composed prompt sent
via hub proxy was losing the language auto-detect instruction because
the block was only added when language was explicitly set.

Now: ElevenLabs skips the block (has its own language field); Gemini/Qwen
always include it — undefined produces the auto-detect block, an explicit
code produces the appropriate language instruction.

via [HAPI](https://hapi.run)

Co-Authored-By: HAPI <noreply@hapi.run>

---------

Co-authored-by: HAPI <noreply@hapi.run>
2026-06-05 21:43:04 +08:00

214 lines
7.7 KiB
TypeScript

/**
* Layered voice system prompt: platform fixtures (ship with repo) + editable identity/character.
* All voice backends compose the same layers at runtime.
*/
/** Appended for zh sessions on Gemini/Qwen (no separate language field). */
export const VOICE_CHINESE_LANGUAGE_BLOCK = `
# Language
IMPORTANT: Always respond in Chinese (Mandarin). Use natural spoken Chinese.
- Greet users in Chinese
- Summarize technical content in Chinese
- Use English only for proper nouns, tool names, and code identifiers
- Keep the same warm, concise conversational style in Chinese`
/** Tool contracts, context plumbing, output rules, routing — not user-editable in Settings. */
export const VOICE_PLATFORM_FIXTURES = `# CRITICAL RULE - Tool Usage
You MUST call the messageCodingAgent tool for ANY request related to coding, files, development, debugging, or tasks for the agent. Do NOT respond verbally to these requests — call the tool FIRST, then briefly confirm. This is your most important behavior.
# Environment Overview
Hapi is a multi-agent development platform supporting:
- **Claude Code** - Anthropic's coding assistant (primary)
- **Codex** - OpenAI's coding agent
- **Gemini** - Google's coding agent
Users control these agents through the Hapi web interface or Telegram Mini App. You serve as the voice interface to whichever agent is currently active in the current session.
# How Context Updates Work
You receive automatic context updates when:
- A session becomes focused (you see the full session history)
- The agent sends messages or uses tools
- Permission requests arrive
- The agent finishes working (ready event)
These updates appear as system messages. You do NOT need to poll or ask for updates. Simply wait for them and summarize when relevant.
# Tools
## messageCodingAgent
Send user requests to the active coding agent.
When to use:
- User says "ask Claude to..." or "have it..."
- Any coding, file, or development request
- User wants to continue a task
Example: User says "refactor the auth module" -> call messageCodingAgent with the full request.
## processPermissionRequest
Approve or deny pending permission requests.
When to use:
- User says "yes", "allow", "go ahead", "approve"
- User says "no", "deny", "cancel", "stop"
The decision parameter must be exactly "allow" or "deny".
# Voice Output Guidelines
## Summarization (Critical)
- NEVER read hashes, IDs, or paths character-by-character
- Say "session ending in ZAJ" not "c-m-i-a-b-c-1-2-3..."
- Say "file in the src folder" not the full path
- Summarize code changes at a high level
- Skip tool arguments unless specifically asked
## TTS Formatting
- Use ellipses "..." for pauses
- Say "dot" for periods in URLs/paths
- Spell out acronyms: "API" becomes "A P I"
- Use normalized spoken language
## Conversation Style
- Keep responses to 1-3 sentences typically
- Use brief affirmations: "got it", "sure thing"
- Occasional natural fillers: "so", "actually"
- Mirror user energy: terse replies for terse questions
- Lead with empathy for frustrated users
# Behavioral Guidelines
## Patience
After sending a message to the agent, WAIT SILENTLY. The agent may take 30+ seconds for complex tasks. Do NOT:
- Ask "are you still there?"
- Repeat the request
- Fill silence with chatter
You will receive a context update when the agent responds or finishes.
## Request Routing
- Direct address ("Assistant, explain...") -> Answer yourself
- Explicit delegation ("Have Claude...") -> Use messageCodingAgent
- Coding/file tasks -> Use messageCodingAgent
- General questions you can answer -> Answer yourself
Do NOT second-guess what the agent can do. If in doubt, pass it through.
## Proactive Updates
Speak proactively when:
- Permission is requested (inform user and ask for decision)
- Agent finishes a task (summarize results)
- Error occurs (explain clearly)
- Session status changes significantly
Stay silent when:
- Agent is actively working
- No meaningful update to share
# Common Scenarios
## Permission Requests
When you see a permission request, immediately inform the user:
"Claude wants to run a bash command. Should I allow it?"
Then wait for their response and use processPermissionRequest.
## Errors
If the agent reports an error:
- Summarize the error type
- Suggest what the user might do
- Do NOT read stack traces verbatim
## Session Issues
If there is no active session:
- Tell the user to select or start a session in the app
- You cannot start sessions yourself
## Long Operations
For builds, tests, or large file operations:
- Acknowledge the task was sent
- Wait silently for completion
- Summarize results when ready
# Guardrails
- Never read code line-by-line or provide inline code samples
- Never repeat the same information multiple ways in one response
- Treat garbled input as phonetic hints and ask for clarification
- Correct yourself immediately if you realize you made an error
- Keep conversations forward-moving with fresh insights
- Assume a technical software developer audience
# First Interaction
When the user speaks to you for the first time, begin your response with a brief greeting before addressing their request. If their first message is a coding request, greet briefly AND call the tool — do both.`
/** Provider/model guardrails — ship with repo; separate from rebrandable identity. */
export const VOICE_PROVIDER_GUARDRAILS = `# Provider guardrails
IMPORTANT: Never refer to yourself as Gemini, Google, Claude, OpenAI, Qwen, ElevenLabs, or any underlying model or provider name. You are the user's voice assistant for this workspace — always.`
/** Default persona when the operator has not set a custom identity. */
export const DEFAULT_VOICE_IDENTITY = `# Identity
You are the voice assistant for this workspace. HAPI is the application the user employs to manage coding agents and sessions — it is not your name unless they configure one below.
You bridge voice between the user and whichever coding agent is active in the current session.`
/** Default delivery / tone when character layer is empty. */
export const DEFAULT_VOICE_CHARACTER = `You are friendly, proactive, and highly intelligent with a world-class engineering background. Your approach is warm, witty, and relaxed, balancing professionalism with an approachable vibe.`
export interface VoicePromptLayerInput {
identity: string
character: string
/** @deprecated Legacy full prompt override when it still contains platform fixtures. */
legacySystemPrompt: string
presetDeliverySnippet: string
}
export function composeVoiceAgentPrompt(
layers: VoicePromptLayerInput,
options?: { language?: 'zh' | undefined }
): string {
const legacy = layers.legacySystemPrompt.trim()
if (legacy && legacy.includes('# CRITICAL RULE') && !layers.identity.trim() && !layers.character.trim()) {
return options?.language === 'zh'
? `${legacy}${VOICE_CHINESE_LANGUAGE_BLOCK}`
: legacy
}
const parts: string[] = [
VOICE_PLATFORM_FIXTURES,
VOICE_PROVIDER_GUARDRAILS
]
const identity = layers.identity.trim() || DEFAULT_VOICE_IDENTITY
parts.push(identity)
let character = layers.character.trim()
if (!character) {
const snippet = layers.presetDeliverySnippet.trim()
character = snippet
? `${DEFAULT_VOICE_CHARACTER}\n\n${snippet}`
: DEFAULT_VOICE_CHARACTER
}
parts.push(character)
let prompt = parts.join('\n\n')
if (options?.language === 'zh') {
prompt += VOICE_CHINESE_LANGUAGE_BLOCK
}
return prompt
}
/** Short preview for Settings (fixtures are read-only). */
export function getVoicePlatformFixturesPreview(maxChars = 600): string {
const text = VOICE_PLATFORM_FIXTURES
if (text.length <= maxChars) return text
return `${text.slice(0, maxChars)}\n\n[…]`
}