Commit Graph
1437 Commits
Author SHA1 Message Date
Randark 901a0439f1 feat: 国产供应商一等支持(Kimi/Zhipu/DeepSeek 多协议 + 配额/余额监控)
后端:
- 协议凭证维度 credentials[api_protocol] ∈ chat_completions(默认)/anthropic/responses(deepseek)
- /v1/messages 零转换直通原生 Anthropic 端点(kimi/zhipu/deepseek),CC/Responses
  入站交叉组合走 apicompat 双向转换链(responses/chat_completions anthropic-native 转发器)
- count_tokens:anthropic 协议透传原生端点;其余 CN 协议本地 tiktoken 估算
- Coding Plan 额度探测(5h/weekly 滚动窗口)+ payg 余额探测(kimi/deepseek),
  deepseek 双币种 CNY+USD 明细,任一币种达标不停调
- 周期任务 [CNBalance] 并发探测 + 预算随工作量放大;响应式 429 冷却到最早窗口
  重置点;余额不足可恢复临时停调;智谱 CREDIT_LIMIT 不污染窗口解析
- CC→anthropic 流式客户端断开后继续排水上游保住 usage 计量

前端:
- 创建/编辑弹窗 account_mode + api_protocol + base_url 联动预设(含 watcher 竞态防护)
- 用量单元格:kimi/zhipu coding 显示 5h/weekly 窗口,kimi/deepseek payg 显示余额,
  多币种并列展示;探测失败保留快照;挂载自动探测 5min 去抖
- 调度阈值设置面板补 kimi/zhipu 平台(对齐后端 AllowedSchedulingThresholdPlatforms)
2026-08-15 10:37:51 +00:00
shaw 8ae6d8f67e fix(openai): send session-level beta features and probe native compaction v2
OpenAI sunset the legacy unary /responses/compact endpoint (404, #5598,
#5624), so the account "compact probe" in the admin UI kept failing even for
healthy accounts, and the beta-feature negotiation header was only attached
to compaction turns.

Beta features (codex-rs session/mod.rs build_model_client_beta_features_header
+ client.rs build_responses_headers): the header is a session-level constant
attached to every /responses request, the WS handshake and /responses/compact.
Enumerating FEATURES shows no Experimental feature is enabled by default, so a
default install sends exactly "remote_compaction_v2". Mirror that:

- OAuth requests without a client-declared header get the default shape, so we
  no longer produce a "header only on compaction turns" pattern real Codex
  never emits (#5586 chains that strip the header)
- a client-declared header is preserved as-is: non-empty without v2 means the
  user disabled the feature and the gateway must not rewrite that
- native v2 turns (compaction_trigger in body) always ensure v2 is present
- non-OAuth upstreams keep the compaction-turn-only behaviour
- the WS injection sits outside the client-header copy block so prewarm and
  turn handshakes cannot land in different pool compatibility buckets

Compact probe now exercises native v2 (streaming /responses +
compaction_trigger) instead of the dead endpoint. Success requires an actual
compaction output item — scanning output_item.done/added, the terminal
response.output[] and the whole-JSON fallback — so a 2xx that silently drops
the trigger is reported as unsupported (the "got 0 items" class, #5478,
#5648). Probe identity is now UUID-shaped and applies the account's
convergence, matching real traffic on the same endpoint.
2026-08-15 16:35:08 +08:00
Wesley LiddickandGitHub 1d3b9665c8 Merge pull request #5641 from InCerryGit/fix/issue-5624-remote-compaction-v2
fix(openai): preserve remote compaction v2 responses endpoint
2026-08-15 13:46:31 +08:00
lyen1688 cb7b03795d feat: 优化分组用量统计 2026-08-14 22:34:43 +08:00
InCerryGit a8b9ea22b7 fix(openai): separate native and legacy compaction routing 2026-08-14 18:18:53 +08:00
InCerryGit 9662cff2e7 fix(openai): preserve remote compaction v2 responses endpoint 2026-08-14 16:23:52 +08:00
IanShaw027 678eb22a40 fix: Realtime 仅在观察到音频后计费,并修正标志位求值顺序
原先 elapsed>0 就出账,握手失败也会扣费;随后用 audioObserved,
但又在同一个 return 里先 Load 再收 errCh,标志位恒为 false,会话全部漏计。

- 先等中继结束再读 audioObserved
- 无音频或零时长不出账;每次连接独立 request id
2026-08-13 08:38:10 +08:00
IanShaw027 c4d883b8da feat: Chat 与 Responses 往返保留 x_search,并补 sources 抽取
Chat Completions 的 {"type":"x_search"} 会被 apicompat 丢掉,独立
/x_search 又没带 include 与结构化提示,主路径容易返回空结果仍计费。

- Chat↔Responses 保留 x_search 过滤字段与 tool_choice
- declared 只注册实际存活的 x_search,web_search 选择项仍丢弃
- 上游补 include x_search_call.action.sources 与结构化输出提示
2026-08-13 08:37:57 +08:00
IanShaw027 0de6d7e9ba feat: 新增独立 /x_search,走原生 x_search 并沿用搜索计费
Grok 分组原先只有 /web_search,无法带 handle/日期过滤,也无法走
xAI 的 x_search tool。

- POST /x_search(仅 Grok 分组),复用 web_search 的审计、failover 与按次计费
- 上游 Responses 强制 x_search;计费模型记为 grok-x-search
2026-08-13 07:49:19 +08:00
IanShaw027 f3d9491071 feat: 分组支持逐模型定价,并可关闭长上下文阶梯
运营需要按分组覆盖渠道/内置价,且部分套餐不应自动吃 200k 倍率。
原先只能改渠道价卡,分组侧只剩 Voice 三列。

- groups 新增 model_pricing / long_context_pricing_enabled,解析链改为 Group → Channel → 内置
- 关闭长上下文时 token 模型只取最低档;video 按秒计费可写进同一价卡
- 回退价对齐官方卡:4.5 缓存 $0.30、4.3/imagine/audio/search 默认值一并校正
2026-08-13 07:49:08 +08:00
IanShaw027 a04ce49016 feat: 新增 grok-4.6 目录、官方定价与请求路径支持
官方目录价:<200k 为 $2 / 缓存读取 $0.50 / $6,≥200k 全部 2 倍。
原先没有独立价卡,/models 宣称可配 reasoning 但请求路径会剥掉 effort。

- 目录与别名接入 grok-4.6 / grok-4.6-latest,默认文本模型仍为 grok-4.5
- 独立回退价卡含缓存读取价与 200k 长上下文倍率
- 保留 reasoning_effort;Chat→Responses 的 cache/vision 桥接同样识别 4.6
2026-08-13 01:23:45 +08:00
Wesley LiddickandGitHub a29fce4a61 Merge pull request #5511 from wucm667/fix/pr-5234-ws-audit-logging
fix(security-audit): restore websocket audit logs
2026-08-12 09:58:24 +08:00
shaw a3bbf35cbd Merge branch 'main' into fix/issue-5029-openai-passthrough-pool-auth-retry
Resolve conflict in backend/internal/handler/openai_gateway_handler_test.go.

main and this branch each appended a passthrough upstream stub plus a test at
the same two insertion points:

  main   openAIHTTPPassthroughSSERateLimitUpstream
         TestOpenAIResponses_APIKeyPassthroughSSERateLimitUsesConfiguredPoolRetry
  branch openAIHTTPPassthroughAuthFailoverUpstream
         TestOpenAIResponses_APIKeyPassthroughPoolAuthFailureRetriesThenSwitchesToHealthyAccount

Both sides are kept verbatim; the only edit is giving each stub its own
calls() body instead of sharing the trailing one. No assertion was changed.

openai_gateway_passthrough.go and openai_oauth_passthrough_test.go merged
automatically.
2026-08-11 14:09:07 +08:00
Wesley LiddickandGitHub b918874f81 Merge pull request #5403 from cyhhao/fix/codex-capacity-exponential-backoff
fix(openai): back off capacity retries exponentially
2026-08-11 13:58:06 +08:00
wucm667 2d9920ba7d fix(security-audit): restore websocket audit logs 2026-08-11 11:56:46 +08:00
pigzwyandshaw 9096492b55 feat(billing): support safe upstream response model billing 2026-08-10 18:45:14 +08:00
Wesley LiddickandGitHub 10a4c6e3ad Merge pull request #5234 from wucm667/fix/issue-5230-deduplicate-latest-turn-audit
fix(security-audit): deduplicate websocket turn audits
2026-08-10 10:53:17 +08:00
Wesley LiddickandGitHub f3c7a1a8c4 Merge pull request #5295 from wucm667/fix/issue-5289-streaming-upstream-error
fix: emit response.failed when compact keepalive commits headers but no SSE payload
2026-08-10 10:52:49 +08:00
Wesley LiddickandGitHub 30d0405388 Merge pull request #5464 from wucm667/fix/issue-5455-api-key-input-validation
fix(api-key): validate quota and expiry inputs
2026-08-10 10:52:04 +08:00
wucm667 f5c108c836 fix(api-key): validate quota and expiry inputs 2026-08-09 22:42:51 +08:00
lyen1688andlyen1688 bbc8b6e906 完善大文件备份分卷上传与恢复 2026-08-09 20:58:07 +08:00
shaw 563a72ca73 feat: add default-off switch for email domain registration quota
PR #5423 relaxed the email suffix whitelist: once a whitelist is
configured, non-whitelisted registrable domains are each allowed to
register one account. That behavior activated unconditionally.

Add registration_email_domain_quota_enabled (default false) to gate it:

- Off (default): restore pre-#5423 strict whitelist semantics — with a
  non-empty whitelist, non-whitelisted domains are rejected with
  EMAIL_SUFFIX_NOT_ALLOWED; the register/verify views restore the
  client-side whitelist pre-check and allowed-domain hint.
- On: keep #5423 behavior — one account per non-whitelisted registrable
  domain (EMAIL_DOMAIN_REGISTRATION_LIMIT).
- Empty whitelist keeps allowing all domains in both states.

Gating lives in validateRegistrationEmailQuota and (as a race-safety
backstop) createUserWithRegistrationEmailGuard; the repository-level
domain lock + in-tx recheck is unchanged. The admin update field is
*bool (omitted = keep current) so stale full-payload saves cannot
silently flip the switch. Email binding and OAuth auto-signup keep
their strict policy, and pending-OAuth bind-login for existing
accounts is unaffected because the handler resolves existing emails
before the quota check.

Frontend adds the toggle to admin settings (zh/en copy; whitelist hint
restored to strict wording, quota wording moved to the new toggle) and
exposes the flag via public settings + SSR injection payload.

Tests: #5423 quota tests now enable the switch explicitly; new
default-off regression tests cover register/send-code/async/pending
OAuth/OIDC create-account plus both register views; API contract JSON
and the injection drift guard are updated.
2026-08-09 15:53:40 +08:00
Wesley LiddickandGitHub f2da30bcd9 Merge pull request #5423 from lyen1688/feat/email-domain-registration-quota
完善邮箱域名注册额度策略
2026-08-09 15:16:26 +08:00
shaw d92edc01be Merge origin/main into feat/channel-monitor-v2-ops-ui
Resolves three conflicts, all of the "both branches appended to the same
block" shape. Every one is resolved as a union of both sides; nothing from
either parent is dropped.

- handler/admin/setting_handler_update.go: keep ChannelMonitorHideThroughput
  (V2) alongside GrokDefaultTextModel / GrokCrossClientModelMapEnabled /
  GrokDefaultBaseURLMode (#5408). UpdateSettings writes every key on each
  save, so dropping either side would reset those settings to zero values.
- service/domain_constants.go: keep SettingKeyChannelMonitorHideThroughput
  and the three SettingKeyGrok* constants.
- repository/migrations_runner.go: keep the 195 checksum rule (V2) and the
  218/219/220 rules (#5408).
2026-08-09 12:11:35 +08:00
lyen1688andlyen1688 4999231d61 修复邮箱域名注册额度策略 2026-08-08 21:05:20 +08:00
IanShaw027 7eb1310701 fix(grok): close free-by-default billing and related review blockers
H1/H2: bill search and voice with code defaults when group prices are
nil (explicit 0 remains free); bump API key auth snapshot to v19 and
refresh incomplete media/search/audio projections.

M1–M6: free-quota soft gate fails open on cache miss with background
refresh and 60s default TTL; correct password_auth config docs; default
cross-client model map to true (→ grok-4.5); audit /tts and /web_search;
exclude composite from migration 220 video-price clears; never let a
search surcharge mask token pricing failures.
2026-08-08 14:39:22 +08:00
IanShaw027 cec922d335 fix(grok): clear golangci-lint findings on complete-integration branch
Check Close/CloseNow errors, drop unused helpers and dead constants,
lowercase ST1005 error strings, and stop discarding unwrap status as an
unused assignment so CI golangci-lint passes.
2026-08-08 12:56:40 +08:00
IanShaw027 1f58e25ab3 Merge upstream/main into feat/grok-complete-integration
冲突集中在 chat completions / messages 两条 Responses 转发路径:
upstream 给 OpenAIForwardResult 增加了 UpstreamResponseModel 与
UpstreamResponseModelConflict(配套 beginUpstreamResponseModelObservation
观测器),本分支在同样位置把返回值改成了具名变量以便挂 Grok 原生搜索计数。
两侧不互斥,合并结果同时保留上游的响应模型观测字段与 Grok SearchCount 逻辑。

frontend/pnpm-lock.yaml 取 upstream 版本:package.json 与 upstream 完全一致,
本地差异只是 pnpm install 的重解析噪音。
2026-08-08 11:12:56 +08:00
IanShaw027 f3bac4619e feat(keys): expand Grok client samples and tune free soft-gate default
Ship Use Key templates that match Grok Build / Codex best practice: env
vars + multi-model config.toml with api_backend=responses, env_key over
hardcoded secrets, and clearer shell/path guidance for Claude/Codex/OpenCode.

Also set free_quota_token_limit default to 500k (24h soft-gate), clean up
personal-dev-only comments, and keep billing test fixtures aligned.
2026-08-08 10:26:16 +08:00
IanShaw027 e01ce90d47 fix(grok): harden voice request ids, video pending, and search pricing alerts
Mint durable grok_audio/grok_realtime usage ids, avoid CLI headers on api.x.ai
voice, retry video pending store and fail-closed when snapshot is missing without
status duration, align pure-video ImageCount tests, and escalate unset search
price_per_1k to error-level logs.
2026-08-08 09:45:12 +08:00
IanShaw027 12db0f906a fix(grok): drop account-test ZDR path and align media CLI headers
Remove optional upload_url / fake connectivity-only success from admin video
tests. Stamp Grok CLI headers only on the CLI proxy so OAuth media against
api.x.ai can complete and preview video like the gateway path.
2026-08-08 09:45:12 +08:00
IanShaw027 35faaa6d21 feat(grok): register custom-voices CRUD and audio download gateway routes
Forward list/get/patch/delete and reference-audio paths with safe path segment
encoding, method passthrough, and empty-body GET/DELETE handling.
2026-08-08 08:48:38 +08:00
IanShaw027 85b65284ec fix(grok): set async video duration_ms from create accept to done discovery
Store CreatedAt on pending billing at video create and use wall-clock E2E
latency when status/content first observes official done+video.url, so usage
logs no longer record only the single poll hop.
2026-08-08 08:48:38 +08:00
IanShaw027 0d98176c59 fix(channel-monitor-v2): correct aggregation privacy and backfill 2026-08-08 01:59:44 +08:00
IanShaw027 68faeac837 fix(grok): restore base URL resolution and operator settings wiring
Honor account GetGrokBaseURLOr policy for official vs custom endpoints,
and wire settings resolution used by responses/chat URL builders.
2026-08-08 01:07:19 +08:00
IanShaw027 6d632eec45 fix(grok): tighten OAuth SSO flow and hide password login
Require oauth state/redirect consistency, fail closed on missing proxy,
and remove password login from create/reauth UI (admin-only password path stays off by default).
2026-08-08 01:07:19 +08:00
IanShaw027 d0767eab9d feat(grok): admin account test modes with real media preview
Add mode-first connectivity probes for text/image/video/search/tts/stt/realtime,
standalone voice and web-search paths, media upload options, and in-browser
image/audio/video preview (including ZDR-safe b64 images and edit validation).
2026-08-08 01:07:08 +08:00
cyh 74fcdf3d42 fix(openai): preserve exponential capacity backoff 2026-08-08 01:02:02 +08:00
Wesley LiddickandGitHub 8991574873 Merge pull request #5345 from puppywang/fix/oauth-pending-account-takeover
fix(security): block OAuth account takeover via pending exchange
2026-08-07 23:20:22 +08:00
Brisbanehuang db0bff82c7 feat(usage): audit upstream response models
(cherry picked from commit 839036224f795c8ee5dc6718a2a14372a45eea44)
2026-08-07 09:40:11 -04:00
li 02fbcbe3ad fix(ratelimit): 守卫按端点来源门控,并与冷却键对齐模型口径
上一版守卫只看模型类型,不区分请求从哪个端点进来。OAuth 账号的 /v1/images/*
上游同样是 Codex Responses(openai_images_responses.go → handleOpenAIImagesErrorResponse
→ handleOpenAIAccountUpstreamError → HandleUpstreamModelNotFound),所以专用生图
端点也会命中 plan-gated 分支。账号确实不具备生图能力时跳过冷却,会让调度层失去
唯一的刹车:每个请求都完整走一遍号池,对上游形成无上界的 400 放大。

改动:
- 新增 ctxkey.OpenAIImagesEndpoint 与 WithOpenAIImagesEndpoint /
  OpenAIImagesEndpointFromContext,在 handler/openai_images.go 入口置位;
  与 OpenAIImageGenerationIntent 区分——后者在 /v1/responses 带图片模型时也会置位。
- 守卫下移到 modelKey 计算之后,抽成 shouldSkipCodexPlanGatedImageModelCooldown,
  仅在 plan-gated 分支、且非 /v1/images/* 入站时生效。
- 同时判断 requestedModel 与最终 modelKey:冷却键走 account.GetMappedModel,
  账号可以把文本别名映射到 gpt-image-*,只判请求模型会漏掉这种形态。
2026-08-07 20:49:39 +08:00
IanShaw027 d7c9e7167b fix(grok): 三轮评审 — 流式 Search 去重与调度/计费加固
- P0: 直播 SSE SearchCount 跨事件 call_id 去重,避免 ~2× 附加费
- SearchCount/Audio/WebSearchCalls 走 mandatory usage task
- web_search:uuid 等 forced request_id 优先于 client/local
- Sanitize 始终剥离 cookie;ApplyOAuth 清 grok_needs_reauth_at
- free 判定:paid 证据压过陈旧 free 凭据
- Gateway 列表应用 free soft-gate;token/body-read 可 failover
- web_search 重试支持 WaitPlan 获取;周 PeriodEnd 不再回填月 end
2026-08-07 17:58:23 +08:00
IanShaw027 245d069602 fix(grok): 二轮评审残留 — Search 叠加计费与 fail-closed 安全
- SearchCost 叠加 token(openai/gateway),未定价 warn
- Token URL 校验失败回落 DefaultTokenURL,禁止 Effective 旁路
- free 判定收窄 paidSignal(仅 plan/月额度),usage% 不否决 free
- web_search: mandatory 计费、uuid request_id、上游 failover 重选账号
- 调度阈值:7d/30d 不跨期 until + 48h stale 可选跳过
- SanitizeStoredCredentials 接入 create/update/bulk/SSO/ApplyOAuth
- ApplyOAuth 成功清除 grok_needs_reauth;SSO 允许 header_override_enabled
- VideoModelPrices 视为媒体定价完整;realtime 正常关闭仍计费
2026-08-07 17:40:21 +08:00
IanShaw027 ccf7ba3ba7 fix(grok): P2 分层清理、主对话 SearchCount 与调度 7d/30d
- DoGrokNativeResponsesJSON 去 gin.Context,UA 固定 CLI 身份
- GrokOAuthService 去掉 redis 直依赖,session store 由 wire 注入
- 主对话/流式响应统计 web_search_call/x_search/tool_search 写入 SearchCount
- Grok 调度阈值候选并入官方 weekly/monthly billing %
2026-08-07 17:21:01 +08:00
IanShaw027 7a81468282 fix(grok): 按 review 优先级修复计费漏扣、auth 投影与 free 门禁
P0: content 与 status 共用 claim 计费;稳定 grok-video request_id;
auth 热路径投影 video_model_prices。
P1: claim 失败释放可重试;视频绑定 TTL≥24h;SSO 凭据白名单与脱敏;
Token URL 校验;free soft-gate 默认 1M 并统一 free 判定;
Voice 预检余额;STT 抗低报;Realtime 失败不计费;search 未定价告警。
2026-08-07 17:15:43 +08:00
IanShaw027 93a04567d4 style(grok): 清理搜索处理器末尾空行 2026-08-07 17:03:48 +08:00
IanShaw027 856a96a217 test(grok): 对齐配额模型同步与 CLI 身份断言 2026-08-07 16:55:36 +08:00
IanShaw027 61b975b5dd test(grok): 对齐跨客户端映射与路由审计契约 2026-08-07 16:45:25 +08:00
IanShaw027 21d0905c78 fix(grok): 按官方 status=done + video.url 计费并对齐字段
官方完成响应为 status=done、video.url、video.duration、顶层 model;
分辨率仅在创建请求中声明,status 无此字段故回退 create 快照/默认 480p。
拒绝 completed 等非官方状态与仅 URL 的宽松判定。
2026-08-07 16:36:15 +08:00
IanShaw027 962da308cc fix(grok): 限制导入探测队列并去重任务 2026-08-07 16:36:12 +08:00