Commit Graph
828 Commits
Author SHA1 Message Date
Wesley LiddickandGitHub bfabfe60c8 Merge pull request #4593 from StarryKira/fix/image-storage-env-unreachable
fix: 异步生图开关配了却不生效(环境变量被静默丢弃 + 迁移到后台开关)
2026-07-20 09:20:15 +08:00
harukaandClaude Opus 4.8 b08cab91a9 feat(image-storage): 异步生图对象存储改为后台配置,保存即生效
此前开启异步生图必须改服务器上的 config.yaml 并重启容器(#4542),且若想
复用已配置的备份 S3,还得把同一套凭证再填一遍(#4458)。

- 新增 ImageStorageSettingService:配置存 settings 表,SecretAccessKey 经
  SecretEncryptor 加密落库、读回脱敏、留空表示沿用旧值,与备份 S3 配置同一套做法。
- reuse_backup_s3(默认开)直接借用 backup_s3_config 的端点与密钥,只用自己的
  bucket/prefix 区分对象,因此备份走 backups/、图片走 images/,且密钥不会在库里存两份。
- ImageTaskService 的启用状态改由 ImageStorageResolver 在运行时解析并缓存,
  保存设置后 Invalidate 使下次请求重建客户端——不再需要重启。
- repository 侧由提供实例改为提供工厂,客户端才可能在运行期重建。
- 轮询接口的门控从 enabled() 放宽为 Pollable():关掉开关只拒绝新提交,
  已受理的任务仍可取回结果,不再被中途吞掉。
- config.yaml 的 image_storage 保留为回落,后台从未保存过时沿用,
  升级前已用配置文件开启的部署不受影响。
- 管理端 GET/PUT/POST /admin/backups/image-storage,PUT 与备份 S3 配置一样要求
  step-up 2FA:改写存储目标同样能把生成内容导向外部账号。

注:go generate ./cmd/server 在当前 upstream 基线上即失败(securityaudit.
PromptAdminService 缺 provider),故 wire_gen.go 为手工同步。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VHreE5pzCkSYz7J45fmd2Y
2026-07-19 00:56:21 -07:00
Wesley LiddickandGitHub a3f2b8fd87 Merge pull request #4540 from heathermhuang/codex/investigate-grok-oauth-test-4525
fix(grok): retry CLI chat permission denial
2026-07-18 20:50:07 +08:00
Wesley LiddickandGitHub a62b821b5f Merge pull request #4478 from yardbirds0/codex/fix-upstream-billing-probe-refresh
fix: 完善上游 Sub2API 倍率探测刷新、展示与账号配置
2026-07-18 20:49:14 +08:00
Wesley LiddickandGitHub 774ff5d8c8 Merge pull request #4515 from BenjaminAaron196/feat/filter-noise-rejected-requests
(fix) 过滤入口拒绝日志并强化鉴权安全边界
2026-07-18 20:46:50 +08:00
Heatherm Huang 2946281a43 fix(grok): retry CLI chat permission denial 2026-07-18 14:32:56 +08:00
benjamin 71afe24710 fix: 修复CI检查问题 2026-07-18 00:42:16 +08:00
benjamin b92bbf0299 fix: 过滤入口拒绝日志并强化鉴权边界 2026-07-18 00:11:18 +08:00
wucm667 2594950993 fix: classify transient account exhaustion as 503 instead of 404 2026-07-17 22:10:18 +08:00
Tian Lee d2585097f3 fix: 完善上游 Sub2API 计费倍率探测与账号展示 2026-07-17 20:59:29 +08:00
Heatherm Huang eaf0691752 fix(grok): preserve media eligibility in scheduler cache 2026-07-17 15:45:19 +08:00
shaw c1fd1cb44e Merge origin/main into codex/fix-grok-regressions-4412-4421
Reconcile the OAuth media route with the manual endpoint-switch redesign
(7f5d067af): media leaves for api.x.ai only when text traffic resolves to
the CLI gateway host; manually selected official/regional/custom endpoints
keep serving media as-is.
2026-07-16 19:57:48 +08:00
Wesley LiddickandGitHub 7e13b6d039 Merge pull request #4425 from AdrianZhaoDev/agent/admin-users-batch-limits
feat(admin): batch update user concurrency and RPM
2026-07-16 19:33:32 +08:00
yan9651688 9fc006546c Make repeated group setup safer
Admins often recreate groups with the same pricing, routing, and account membership. A server-side duplicate creates an inactive copy for review, preserves eligible account priorities, and recovers ambiguous retries without creating extra groups.

Constraint: Group has no neutral JSON metadata field for durable operation recovery
Constraint: Model routing references account IDs, so copied configuration requires matching bindings
Rejected: Rebuild from the list response | it omits configuration and account priority details
Rejected: Store operation identity in business configuration | it would pollute real group settings
Confidence: high
Scope-risk: moderate
Reversibility: clean
Directive: Keep duplicated groups inactive until an administrator reviews the copied configuration
Tested: Go unit and full tests, go vet, integration-tag compile, frontend Vitest, lint, typecheck, production build, and Playwright duplicate flow
Not-tested: PostgreSQL container integration locally because Docker is unavailable; CI will execute the database-backed suite
2026-07-16 18:18:28 +08:00
Heatherm Huang 115116e8bf fix(grok): repair OAuth routing regressions 2026-07-16 16:52:25 +08:00
yan9651688 e2e375d694 Make repeated channel-monitor setup safer
Admins often recreate monitors with the same endpoint, model, and request settings. A server-side duplicate keeps the stored API key out of the browser, creates a disabled copy for review, and uses stable operation identity to recover ambiguous retries without creating extra rows.

Constraint: Stored monitor API keys must never be returned to the browser
Constraint: Applying a request template must preserve internal duplicate recovery metadata
Rejected: Rebuild the monitor from list data | list responses only contain a masked API key
Rejected: Copy runtime state and history | a duplicate should start as an unverified configuration
Confidence: high
Scope-risk: moderate
Reversibility: clean
Directive: Keep duplicated monitors disabled until an administrator reviews and enables them
Tested: Go unit tests for repository, service, and admin handler; integration-tag compile; go vet; golangci-lint v2.9; frontend Vitest, ESLint, typecheck, production build; Playwright duplicate flow
Not-tested: PostgreSQL container integration locally because Docker is unavailable; CI will execute the database-backed suite
2026-07-16 16:30:19 +08:00
zhaozewu 7947619cc3 feat(admin): batch update user limits 2026-07-16 15:37:35 +08:00
shaw 590efe29a5 Merge remote-tracking branch 'origin/main' into agent/fix-4326-async-image-object-storage
# Conflicts:
#	backend/cmd/server/wire_gen.go
2026-07-16 15:32:38 +08:00
shaw 2de6ccb071 fix(security): 补齐审计日志脱敏缺口 + step-up 下载/取消体验修复
审计发现修复:

高危——审计日志请求体脱敏不全(audit_logs 沦为明文凭证聚合点):
- 键名归一化比对(小写+去分隔符),覆盖 privateKey/apiv3key 等无分隔符与 camelCase 写法
- 程序化并入 SensitiveCredentialKeys 与 providerSensitiveConfigFields 两份权威敏感表,防清单漂移
- 补齐 proxy_key(内嵌代理密码)、custom_key(自设 API Key 明文)精确键
- Codex session 导入路由 body 整体由粘贴的 auth JSON 构成,键级脱敏无法覆盖,整体不入库
- 新增守卫测试:两份权威表的每个键必须被审计脱敏命中;provider_key 等渠道标识保留以便追责

中危——step-up 前端体验:
- 备份下载改同页 anchor 导航(预签名 URL 后端强制 attachment disposition),
  避免 step-up 弹窗 await 耗尽瞬态激活后 window.open 被浏览器拦截
- useStepUp.run 用户取消时抛 StepUpCancelledError sentinel,
  三个调用点静默处理,不再把取消误报为红色错误 toast

低危:
- 修正审计中间件挂载位置的陈旧注释(实际挂在认证之后)
- 审计 body 捕获改 LimitReader 按 256KB 上限截断读取,超出部分拼接回填,
  避免大体积导入请求被完整复制进内存两次
- TOTP step-up 弹窗验证成功后即时清空验证码输入
2026-07-16 14:38:16 +08:00
shaw 0ddd58aaf9 feat(security): 操作审计日志 + 会话IP/UA绑定 + 敏感操作 step-up 2FA
应对管理员访问凭证失守导致的数据外泄风险,新增三层防护:

审计日志(admin-only 可见,用户不可见)
- 新增 append-only audit_logs 表(migration 180)+ 异步批量写入 + 保留期清理
- 审计中间件挂在 admin/user/auth/admin-payment 组认证之后:记录所有变更类
  请求 + 白名单敏感读取(账号/代理导出、备份下载、admin/user API key 读取)
- 请求头凭证首尾掩码;请求体 JSON 递归脱敏(api_key/password 等擦除,base_url
  保留以便追责);非 JSON body 不入库
- 无单条删除;全量清空需现场 TOTP 校验、拒绝 admin API key、未启用 2FA 不允许,
  清空后同步写入留痕记录

会话 IP/UA 绑定(默认开启,可在系统设置关闭)
- JWT 携带 session id + IP/UA 指纹哈希;IP 或 UA 任一变化即撤销会话家族并要求
  重新登录;旧 token 无指纹时放行以平滑升级

敏感操作 step-up 2FA(sudo 窗口 15 分钟)
- 账号/代理导出、DB 备份创建/下载、S3 目标修改要求近期 TOTP 二次验证;admin API
  key 一律拒绝;前端 useStepUp 组合式 + TotpStepUpDialog 弹码后自动重试
- API key 查看按需求暂不加强管控

前端:新增 /admin/audit-logs 操作日志页面(筛选/详情/2FA 清空)、侧边栏入口、
step-up 弹窗接入导出与备份流程、安全设置项(绑定开关 + 日志保留天数)、zh/en i18n
2026-07-16 13:47:50 +08:00
harukaandClaude Opus 4.8 0eb6e21aaa feat: 异步图片任务结果落对象存储
为异步生图任务增加 S3 兼容对象存储支持,任务结果不再把大图内联存进 Redis:

- 新增可插拔接口 service.ImageStorage(Save -> url),适配别的厂商只需实现它
- S3 实现 S3ImageStorage(AWS S3 / R2 / 阿里云 OSS / MinIO),与备份共用 S3 客户端构造
- 新增 image_storage 配置(config.yaml + IMAGE_STORAGE_* 环境变量),默认关闭
- enabled 同时作为总开关:关闭或未配置对象存储时,异步生图接口返回 404 且不写
  Redis,从根上避免几 MB 的 b64_json 结果撑爆 Redis
- 完成时把图片上传对象存储并把结果改写为短链接(公开直链或 presigned),
  上传失败则任务标记为失败

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SM1tf3CFVRzC7guuhBXvMd
2026-07-15 19:57:37 -07:00
haruka 1fb942dd77 feat: add async image generation tasks 2026-07-15 19:57:37 -07:00
shaw 0408bdb34f Merge remote-tracking branch 'origin/main' into feat/upstream-rate-scheduling
# Conflicts:
#	backend/internal/handler/openai_gateway_handler.go
2026-07-16 10:31:21 +08:00
Wesley LiddickandGitHub ab978e615e Merge pull request #4385 from yardbirds0/feat/upstream-billing-probe
feat: 增加上游 Sub2API 计费倍率探测与账号展示
2026-07-16 10:27:45 +08:00
Wesley LiddickandGitHub 502097026f Revert "feat: 支持异步生图任务与结果轮询" 2026-07-16 09:47:27 +08:00
Wesley LiddickandGitHub a3a227c858 Merge pull request #4381 from StarryKira/agent/fix-4326-async-image-tasks
feat: 支持异步生图任务与结果轮询
2026-07-16 09:44:26 +08:00
Wesley LiddickandGitHub 807850769f Merge pull request #4396 from StarryKira/fix/4386-image-input-pricing
fix(billing): 图像输入 token 按独立单价计费(gpt-image-2 图片编辑)
2026-07-16 09:32:29 +08:00
jjaw f552448fb2 复用调度快照账号载荷写入 2026-07-16 04:07:15 +08:00
harukaandClaude Opus 4.8 62d57c02d8 feat(billing): usage_logs 单独记录图片输入 token 与费用
图片编辑/图生图请求的图片输入 token 此前并入 input_tokens/input_cost,
无法对账。拆分上报口径,total_cost 保持不变。

后端:
- CostBreakdown 新增 ImageInputCost;computeTokenBreakdown 将图片输入费用
  从 InputCost 拆出(InputCost 从此仅含文本输入),并纳入 tier 倍率与总额;
  长上下文合并路径同步携带 ImageInputCost
- 迁移 179:usage_logs 新增 image_input_tokens / image_input_cost 列
- UsageLog、insert/query 仓储(含定位参数数组、CTE 列表、扫描顺序)、
  DTO 与 mapper 补齐两列
- openai_gateway_usage 从 usage 与 cost 落库图片输入 token/费用

前端:
- UsageLog 类型、imageUsage 工具(hasImageInputTokens/Cost、textInputTokens)
- 用量表 token 徽标、Token/费用 tooltip 与单价行按图/文输入拆分展示
- zh/en usage.* i18n

测试:
- 新增 gpt-image-2 图片编辑复现用例(复现 #4386 的 $0.016081 期望值)
- 新增 usage 提取器图片输入 token 解析用例(input_tokens_details.image_tokens)
- 更新 doubao 图文分价用例与仓储/契约测试以匹配新的 input/image 拆分口径

相关 #4386。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015wcJTKDddxXSQrepSs3wrU
2026-07-15 10:03:38 -07:00
Tian Lee 90ee85f3ef feat: 按上游计费倍率调度 OpenAI 账号 2026-07-16 00:40:46 +08:00
harukaandClaude Opus 4.8 06e03f467a feat(billing): 渠道自定义定价支持图片输入 token 单价 image_input_price
渠道 token 计费模式此前无法为图片输入 token 单独定价,gpt-image-2
图片编辑等请求的图像输入被按文本 input_price 计费。新增
channel_model_pricing.image_input_price 列及全链路支持。

后端:
- 迁移 178:channel_model_pricing 新增 image_input_price 列
- ChannelModelPricing 新增 ImageInputPrice 字段,repo 读写、校验补齐
- model_pricing_resolver / GetModelPricingWithChannel 映射到
  ImageInputPricePerToken;未配置时归零,由 computeTokenBreakdown
  回退文本输入价(向后兼容,与 image_output_price 的渠道权威规则一致)
- admin / 用户侧定价 DTO 与 model-pricing 自动填充接口补充该字段

前端:
- 渠道定价表单新增「图片输入」价格输入(token 模式)
- API 类型、表单模型、form↔API 换算、自动填充、用户侧模型定价卡展示
- zh/en i18n 标签

相关 #4386。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015wcJTKDddxXSQrepSs3wrU
2026-07-15 09:35:34 -07:00
Tian Lee 0765d10c1d feat: 增加上游 Sub2API 计费倍率探测与账号展示 2026-07-15 23:58:46 +08:00
haruka 134179085c feat: add async image generation tasks 2026-07-15 22:13:37 +08:00
Wesley LiddickandGitHub 0de768e8be Merge pull request #4221 from heathermhuang/codex/fix-grok-oauth-pool-health
fix(grok): refresh OAuth pools proactively
2026-07-15 16:07:09 +08:00
Wesley LiddickandGitHub 2d218fbe61 Merge pull request #4317 from yan9651688/feat/account-one-click-copy
feat(accounts): add safe one-click account duplication
2026-07-15 14:23:35 +08:00
Wesley LiddickandGitHub 5b5e41450c Merge pull request #4310 from jianjianai/codex/perf-scheduler-final
perf(scheduler): 完善缓存桶生命周期并复用重建批次查询
2026-07-15 11:09:47 +08:00
yan9651688 60ff61132d feat(accounts): make repeated static account setup safer
Admins often need another account with the same provider and routing configuration. Duplicate on the server so credentials never return to the browser, preserve exact group priorities atomically, start the copy paused, and recover the same copy after ambiguous idempotency-store failures.

Constraint: Admin account responses redact credentials, so duplication must remain server-side

Constraint: OAuth and setup-token credentials rotate and must not be shared across account rows

Rejected: Copy raw account JSON to the clipboard | exposes credentials outside the server

Rejected: Duplicate rotating credentials | account-scoped refresh locks can race token rotation

Confidence: high

Scope-risk: moderate

Reversibility: clean

Directive: Keep copies paused, avoid automatic upstream probes, and exclude rotating credential types unless token ownership is redesigned

Tested: Targeted Go tests, Go vet, server build; frontend lint, typecheck, Vitest suite, production build; integration test compiled

Not-tested: Docker-backed PostgreSQL execution because Docker is unavailable

Related: Wei-Shaw/sub2api#1379

Related: Wei-Shaw/sub2api#2928
2026-07-15 10:51:13 +08:00
Heatherm Huang b245ab5848 fix(grok): satisfy pool health lint 2026-07-15 09:40:08 +08:00
Heatherm Huang 6b25900403 fix(grok): refresh OAuth pools proactively 2026-07-15 09:40:08 +08:00
Wesley LiddickandGitHub 1c61225b28 Merge pull request #4271 from jianjianai/codex/perf-u10-ops-projection
优化 Ops 队列深度账号查询
2026-07-15 09:39:09 +08:00
jjaw 62be9db9df 补充调度缓存并发语义注释 2026-07-15 07:10:22 +08:00
jjaw 9da4cfbdcc 增加调度分组生命周期租约 2026-07-15 00:12:40 +08:00
jjaw 8523a52c07 优化调度缓存桶退休原语 2026-07-14 23:08:17 +08:00
jjaw a8afc20969 优化 Ops 队列深度账号查询 2026-07-14 16:18:26 +08:00
Heatherm Huang b32b815e46 fix(grok): harden OAuth pool recovery 2026-07-14 14:55:30 +08:00
Heatherm Huang 343390057d fix(grok): fail over OAuth credential errors safely 2026-07-14 14:55:30 +08:00
Wesley LiddickandGitHub 24d908b257 Merge pull request #4203 from jianjianai/codex/fix-scheduler-pending-lag
修复调度器使用已消费事件计算 outbox 延迟
2026-07-14 10:59:57 +08:00
Wesley LiddickandGitHub 9c3c560d49 Merge pull request #4206 from jianjianai/codex/fix-scheduler-proxy-expiry-event
修复代理到期改投触发全量调度重建
2026-07-14 10:59:43 +08:00
Wesley LiddickandGitHub 8711882918 Merge pull request #4205 from jianjianai/codex/fix-scheduler-auto-pause-event
修复账号到期暂停触发全量调度重建
2026-07-14 10:59:34 +08:00
Wesley LiddickandGitHub 27fcbace89 Merge pull request #4207 from hongheshan-svg/upstream-pr/http2-keepalive
fix(openai): 为 Codex HTTP/2 上游启用 PING 健康探测,剔除死连接
2026-07-14 10:59:22 +08:00