diff --git a/Docs/api-client.md b/Docs/api-client.md index 1d46e4d..3bab24c 100644 --- a/Docs/api-client.md +++ b/Docs/api-client.md @@ -30,8 +30,8 @@ | 7 | `/api/v1/sessions/{id}` | DELETE | JWT(owner/admin) | ✅ | **删除次级会话及其消息** → `{data:{deleted:}}`；主会话不可删回 `403 forbidden`；未注入 HistoryStore 回 `501` | | 8 | `/api/v1/sessions/{id}` | PATCH | JWT(owner/admin) | ✅ | **会话改名**（A4）：body `{title}`（trim、≤80 rune，空/超长回 `400`）→ `{data:{id,title}}`；不存在回 `404`；主会话(`id="web"`)可改名但不可删；未注入 HistoryStore 回 `501`。另：会话**首条用户消息到达时自动命名** title（取前 20 rune 折叠空白；纯空白回落 `主会话`/`次级会话`），列表不再空名 | | 9 | `/api/v1/personas` | GET | JWT | ✅ | `{data:[persona...]}` | -| 10 | `/api/v1/audio/transcribe` | POST | JWT(owner) | ✅ | **语音→文本**：body `{audio:"",format?,deviceId?}` → `{data:{text}}`（走后端 STT，百炼 `qwen-audio-3.0-asr-flash`） | -| 11 | `/api/v1/audio/speech` | POST | JWT(owner) | ✅ | **文本→音频**：body `{text,voice?,format?,instruction?}` → `200` 音频字节（`Content-Type: audio/`，非 JSON envelope，走后端 TTS，百炼 `cosyvoice-v3-flash`，复刻音色可带 `instruction` 情绪指令） | +| 10 | `/api/v1/audio/transcribe` | POST | JWT(owner) | ✅ | **语音→文本**：body `{audio:"",format?,deviceId?}` → `{data:{text, provider?}}`（走后端 STT，百炼 `qwen-audio-3.0-asr-flash`；**当结果为空文本/silent 降级时响应体带 `provider`**（如 `"silent"`，非空文本省略），客户端据此提示「没识别到声音」） | +| 11 | `/api/v1/audio/speech` | POST | JWT(owner) | ✅ | **文本→音频**：body `{text,voice?,format?,instruction?}` → `200` 音频字节（`Content-Type: audio/`，非 JSON envelope，走后端 TTS，百炼 `cosyvoice-v3-flash`，复刻音色可带 `instruction` 情绪指令；**静默降级空音频时仍 `500 tts_empty`，并附 `X-Voice-Provider: silent` 响应头**，客户端据此提示「没生成声音」） | | 12 | `/api/v1/audio/voice-clone` | POST | JWT(owner) | ✅ | **声音复刻建音色**：body `{url:"<公网参考音频URL>",prefix?}` → `{data:{voiceId}}`（走百炼 `voice-enrollment` create_voice） | > **语音 REST 源（`/api/v1/audio/*`，均 owner-only + 审计）**：原生/移动端用它上传/获取音频，**绕开 WS 4KB 帧限**。 @@ -136,7 +136,7 @@ - **上行（客户端→服务端）**: `mode_switch` / `chat`（跑 loop 并广播流式） / `voice_input` / `subagent_ask` / `task_output_query` / `interrupt`。 - **`interrupt`**（上行，客户端→服务端）：`{sessionId}` → 会话编排器中断（design/15 §3）——取消该会话当前在跑的任务（context cancel）、清空其待处理队列、回 `session_interrupted`；用户说「停下/算了/换个话题」时发。命中空闲会话为 no-op。 - **`voice_input`**（上行，客户端→服务端）：`_{audio?, format?, text?, sessionId?, deviceId?}`。`audio` 为 base64（或 data: URI）音频，`format` 为其编码（`wav`/`mp3`/`ogg`）；`text` 为已转写文本时跳过 STT；`deviceId` 为**客户端具体端标识**（design/16 §4.5，如 `voiceassist:myapp:v1`，可选），随该语音轮次落库、聊天气泡可标注来源。服务端走 STT→agent 聊天→TTS（design/06 §1），回复经 `voice_output` 定向回该连接。**真 STT/TTS 已接通阿里百炼**（`DASHSCOPE_API_KEY` 必填：STT=`qwen-audio-3.0-asr-flash`、TTS=`cosyvoice-v3-flash`+`longxiaochun_v3`）；无凭据时后端走降级链（cloud → edge-tts/espeak（未实现）→ `silent` 空音频）。无注入语音服务时回 `error` 帧（`voice unavailable`）。 -- **`voice_output`**（下行，服务端→客户端）：`{audio, format, text, reply}`。`audio` 为 base64 合成音频，`format` 为其编码，`text` 为识别文本，`reply` 为助手回复文本。 +- **`voice_output`**（下行，服务端→客户端）：`{audio, format, text, reply, provider?}`。`audio` 为 base64 合成音频，`format` 为其编码，`text` 为识别文本，`reply` 为助手回复文本；**当合成音频为空（silent 降级）时带 `provider`**（如 `"silent"`，非空音频省略）——客户端据此提示「没生成声音」，但该帧**仍照常发出、不算业务错误**。 - **向后兼容**：未映射/无法确定的事件仍走 `stream_chunk` + `stream_end`（非流式回退路径）。 --- @@ -211,6 +211,7 @@ - 语音：上行 `voice_input`（base64 音频或已转写 text），下行 `voice_output`（合成音频 base64 + text）。 ## 5. 变更记录 +- `v0.23`（**C2 语音 silent 降级信号**）：三处统一用 **`provider`** 只在「结果为空」时出现（非空/成功时才省略）——`POST /api/v1/audio/transcribe` 空文本响应体带 `{text:"", provider:"silent"}`；WS `voice_output` 空音频 payload 带 `provider`；`POST /api/v1/audio/speech` 空音频仍 `500 tts_empty` 并加 `X-Voice-Provider: silent` 响应头（CORS `Access-Control-Expose-Headers` 已放开）。`voice.Chain/Service` 新增 `STTWithProvider/TTSWithProvider`（返回真实档位 cloud/local/edge-tts/espeak/silent），网关用可选接口 `voiceProviderReporter` 透传，自定义实现回退 `silent`。**含 `X-Voice-Provider`（`apperr` 包装、`providerSignal`）；`DASHSCOPE_API_KEY` 无/降级时信号为 `silent`。**（`go test -race` 在本机受内核 TSAN VMA 限制失败，非本代码问题。） - `v0.22`（**B3 会话列表 lastMessage 预览**）：`GET /api/v1/sessions` 与 `/api/v1/admin/sessions` 每个会话新增可选 **`lastMessage`** `{content,role,createdAt}`（content 折叠空白截前 60 rune + `…`；空会话省略），**单条 SQL**（LEFT JOIN 取每会话 `ORDER BY created_at DESC,rowid DESC LIMIT 1`，无 N+1，走 `idx_chat_msgs_session_created` 覆盖索引）。`unreadCount` 需已读游标设计，后置。 - `v0.21`（**A4 会话 title**）：`PATCH /api/v1/sessions/{id}`（**新增**，body `{title}` ≤80 rune → `{data:{id,title}}`；空/超长 400、不存在 404、主会话可改名）；会话**首条用户消息到达时自动命名**（`history.append` 前 `ensureTitle`：折叠空白取前 20 rune，纯空白回落 `主会话`/`次级会话`），仅首次命名、不覆盖。`HistoryStore` 新增 `RenameSession`；隔离会话（`sess-xxx`）不在 `chat_sessions` 有行、不自动命名（其本不在 `/sessions` 列表）。**边界**：前端 `createSubSession(title?)` 的 POST body 含 `title` 但后端暂忽略（会被首条消息自动命名覆盖），未处理。 - `v0.20`（**A3 消息 mode 落库**）：`POST /api/v1/chat` 与 `ws.chat` 上行新增**可选 `mode`**（行为模式，design/01 §3 / design/16 §5.1）。后端把每条持久化消息的 `Mode` **真实写入**（IM 路径恒 `chat`；Web 路径经 ctx 透传本轮 `mode`，缺省回落 `chat`；恢复/续接路径沿用发起轮次的 `mode`），消息 DTO、`message_persisted`、`channel_persisted` 的 `mode` 不再恒空，供前端按模式渲染气泡（**仅 `chat` 解析 `

` 拆气泡**，其它模式原样整条）。 diff --git a/Docs/api-reference.md b/Docs/api-reference.md index 78eb618..b94a29e 100644 --- a/Docs/api-reference.md +++ b/Docs/api-reference.md @@ -15,7 +15,7 @@ - 成功：`{"data": }`（HTTP 200 / 201 / 202 / 204）。 - 失败：`{"error": {"code": , "message": }}`（HTTP 400/401/403/404/409/500/501 等）。 - **错误码**：业务错误带稳定字符串 `code`（如 `unauthorized` / `not_found` / `bad_request` / `chat_busy` / `not_implemented`），便于客户端分支；`message` 面向人读。 -- **CORS**：`Access-Control-Allow-Origin` 为配置的 `ALLOWED_ORIGINS` 白名单（`*` 时放开）；`Allow-Methods: GET, POST, PUT, DELETE, OPTIONS`；`Allow-Headers: Authorization, Content-Type`；`Max-Age: 600`。`OPTIONS` 预检直接回 `204`。同源/无 Origin 请求放行。**注意：CORS 只允许 `Authorization, Content-Type` 头——若客户端要额外发送 `X-API-Version`，需自行在网关侧放开或仅作为文档惯例（当前服务端**不校验**该头）。** +- **CORS**：`Access-Control-Allow-Origin` 为配置的 `ALLOWED_ORIGINS` 白名单（`*` 时放开）；`Allow-Methods: GET, POST, PUT, DELETE, OPTIONS`；`Allow-Headers: Authorization, Content-Type`；`Access-Control-Expose-Headers: X-Voice-Provider`（让跨域浏览器能读到该语音降级信号头，非跨域场景不受影响）；`Max-Age: 600`。`OPTIONS` 预检直接回 `204`。同源/无 Origin 请求放行。**注意：CORS 只允许 `Authorization, Content-Type` 头——若客户端要额外发送 `X-API-Version`，需自行在网关侧放开或仅作为文档惯例（当前服务端**不校验**该头）。** - **请求日志**：每个请求由 `logging` 中间件记录一行（method/path/status/duration_ms），并计入可观测性指标。panic 由 `recover` 中间件兜底为 `500 {error:{code:internal,...}}`，不崩溃进程。 - **超时**：WS 连接 ping 周期 54s、pong 等待 60s、写超时 10s；读上限 4096 字节/帧。 diff --git a/internal/runtime/multimodal/voice/chain.go b/internal/runtime/multimodal/voice/chain.go index cc3d2e8..71f23ca 100644 --- a/internal/runtime/multimodal/voice/chain.go +++ b/internal/runtime/multimodal/voice/chain.go @@ -25,8 +25,16 @@ func NewChain(steps ...Provider) *Chain { // STT walks the chain for a transcription. func (c *Chain) STT(ctx context.Context, audio Audio) (string, error) { + text, _, err := c.STTWithProvider(ctx, audio) + return text, err +} + +// STTWithProvider is STT but also reports the name of the provider rung that +// produced the transcription (cloud / local / edge-tts / silent), so a caller can +// surface which tier actually served the request. +func (c *Chain) STTWithProvider(ctx context.Context, audio Audio) (string, string, error) { if len(c.steps) == 0 { - return "", ErrProviderFailed + return "", "", ErrProviderFailed } var lastErr error for _, p := range c.steps { @@ -36,11 +44,11 @@ func (c *Chain) STT(ctx context.Context, audio Audio) (string, error) { } text, err := stt.STT(ctx, audio) if err == nil { - return text, nil + return text, p.Name(), nil } lastErr = err } - return "", fmt.Errorf("voice: %w: %v", ErrProviderFailed, lastErr) + return "", "", fmt.Errorf("voice: %w: %v", ErrProviderFailed, lastErr) } // CloneVoice creates a custom voice via the cloud provider's voice-enrollment @@ -55,8 +63,15 @@ func (c *Chain) CloneVoice(ctx context.Context, referenceURL, prefix string) (st // TTS walks the chain for synthesized audio. func (c *Chain) TTS(ctx context.Context, text string) (Audio, error) { + audio, _, err := c.TTSWithProvider(ctx, text) + return audio, err +} + +// TTSWithProvider is TTS but also reports the name of the provider rung that +// produced the audio. +func (c *Chain) TTSWithProvider(ctx context.Context, text string) (Audio, string, error) { if len(c.steps) == 0 { - return Audio{}, ErrProviderFailed + return Audio{}, "", ErrProviderFailed } var lastErr error for _, p := range c.steps { @@ -66,11 +81,11 @@ func (c *Chain) TTS(ctx context.Context, text string) (Audio, error) { } audio, err := tts.TTS(ctx, text) if err == nil { - return audio, nil + return audio, p.Name(), nil } lastErr = err } - return Audio{}, fmt.Errorf("voice: %w: %v", ErrProviderFailed, lastErr) + return Audio{}, "", fmt.Errorf("voice: %w: %v", ErrProviderFailed, lastErr) } // Option customizes a chain built by New. diff --git a/internal/runtime/multimodal/voice/provider_test.go b/internal/runtime/multimodal/voice/provider_test.go new file mode 100644 index 0000000..56f9aa4 --- /dev/null +++ b/internal/runtime/multimodal/voice/provider_test.go @@ -0,0 +1,50 @@ +package voice + +import ( + "context" + "testing" +) + +func TestChain_STTWithProvider(t *testing.T) { + t.Parallel() + steps := []Provider{ + &stubProvider{name: "cloud", stt: &stubSTT{err: ErrNotConfigured}}, + &stubProvider{name: "local", stt: &stubSTT{text: "hi"}}, + } + text, rung, err := NewChain(steps...).STTWithProvider(context.Background(), Audio{Format: "wav", Data: []byte("x")}) + if err != nil { + t.Fatalf("STTWithProvider error: %v", err) + } + if text != "hi" || rung != "local" { + t.Errorf("text/rung = %q/%q, want hi/local", text, rung) + } +} + +func TestChain_TTSWithProvider_Silent(t *testing.T) { + t.Parallel() + steps := []Provider{ + &stubProvider{name: "cloud", tts: &stubTTS{err: ErrNotConfigured}}, + SilentProvider(), + } + audio, rung, err := NewChain(steps...).TTSWithProvider(context.Background(), "hi") + if err != nil { + t.Fatalf("TTSWithProvider error: %v", err) + } + if len(audio.Data) != 0 || rung != "silent" { + t.Errorf("audio/rung = %+v/%q, want empty/silent", audio, rung) + } +} + +func TestService_TTSWithProvider(t *testing.T) { + t.Parallel() + svc := NewService(NewChain(NewProvider("edge-tts", nil, &stubTTS{audio: Audio{Format: "mp3", Data: []byte("a")}}))) + defer svc.Close() + + audio, rung, err := svc.TTSWithProvider(context.Background(), "hi") + if err != nil { + t.Fatalf("TTSWithProvider error: %v", err) + } + if string(audio.Data) != "a" || rung != "edge-tts" { + t.Errorf("audio/rung = %+v/%q, want a/edge-tts", audio, rung) + } +} diff --git a/internal/runtime/multimodal/voice/service.go b/internal/runtime/multimodal/voice/service.go index 3bfa295..694d583 100644 --- a/internal/runtime/multimodal/voice/service.go +++ b/internal/runtime/multimodal/voice/service.go @@ -53,9 +53,10 @@ type job struct { // jobResult carries the outcome written back to the caller. type jobResult struct { - text string - audio Audio - err error + text string + audio Audio + provider string + err error } // NewService builds a Service backed by chain and starts the worker pool. @@ -99,9 +100,9 @@ func (s *Service) run(j job) { res := jobResult{} switch j.kind { case jobSTT: - res.text, res.err = s.chain.STT(s.bgCtx, j.audio) + res.text, res.provider, res.err = s.chain.STTWithProvider(s.bgCtx, j.audio) case jobTTS: - res.audio, res.err = s.chain.TTS(s.bgCtx, j.text) + res.audio, res.provider, res.err = s.chain.TTSWithProvider(s.bgCtx, j.text) } j.res <- res } @@ -109,21 +110,35 @@ func (s *Service) run(j job) { // STT enqueues a transcription request and waits for the result. ctx cancels // the wait and the underlying provider call. func (s *Service) STT(ctx context.Context, audio Audio) (string, error) { + text, _, err := s.STTWithProvider(ctx, audio) + return text, err +} + +// STTWithProvider is STT but also reports the provider rung that produced the +// transcription. +func (s *Service) STTWithProvider(ctx context.Context, audio Audio) (string, string, error) { res, err := s.submit(ctx, job{kind: jobSTT, audio: audio, res: make(chan jobResult, 1)}) if err != nil { - return "", err + return "", "", err } - return res.text, res.err + return res.text, res.provider, res.err } // TTS enqueues a synthesis request and waits for the result. ctx cancels the // wait and the underlying provider call. func (s *Service) TTS(ctx context.Context, text string) (Audio, error) { + audio, _, err := s.TTSWithProvider(ctx, text) + return audio, err +} + +// TTSWithProvider is TTS but also reports the provider rung that produced the +// audio. +func (s *Service) TTSWithProvider(ctx context.Context, text string) (Audio, string, error) { res, err := s.submit(ctx, job{kind: jobTTS, text: text, res: make(chan jobResult, 1)}) if err != nil { - return Audio{}, err + return Audio{}, "", err } - return res.audio, res.err + return res.audio, res.provider, res.err } // CloneVoice creates a custom voice, delegating to the underlying chain. It is a diff --git a/internal/services/gateway/audio.go b/internal/services/gateway/audio.go index 3dd492d..43c6a66 100644 --- a/internal/services/gateway/audio.go +++ b/internal/services/gateway/audio.go @@ -61,7 +61,7 @@ func (s *Server) handleTranscribe(w http.ResponseWriter, r *http.Request) { writeError(w, apperr.Wrap(apperr.ErrBadRequest, "bad_request", "audio is required")) return } - text, err := s.voice.STT(r.Context(), audio) + text, rung, err := sttWithProvider(r.Context(), s.voice, audio) if err != nil { s.auditClaims(r, "audio.transcribe", "error") s.log.Warn("voice transcribe", "err", err) @@ -69,7 +69,11 @@ func (s *Server) handleTranscribe(w http.ResponseWriter, r *http.Request) { return } s.auditClaims(r, "audio.transcribe", "ok") - writeJSON(w, http.StatusOK, map[string]any{"text": text}) + out := map[string]any{"text": text} + if sig := providerSignal(rung, strings.TrimSpace(text) == ""); sig != "" { + out["provider"] = sig + } + writeJSON(w, http.StatusOK, out) } // handleSpeech synthesizes speech for the given text and writes the raw audio @@ -96,7 +100,7 @@ func (s *Server) handleSpeech(w http.ResponseWriter, r *http.Request) { if req.Instruction != "" { ctx = voice.WithTTSInstruction(ctx, req.Instruction) } - audi, err := s.voice.TTS(ctx, req.Text) + audi, rung, err := ttsWithProvider(ctx, s.voice, req.Text) if err != nil { s.auditClaims(r, "audio.speech", "error") s.log.Warn("voice speech", "err", err) @@ -105,6 +109,7 @@ func (s *Server) handleSpeech(w http.ResponseWriter, r *http.Request) { } if len(audi.Data) == 0 { s.auditClaims(r, "audio.speech", "error") + w.Header().Set(voiceProviderHeader, providerSignal(rung, true)) writeError(w, apperr.Wrap(apperr.ErrInternal, "tts_empty", "no audio synthesized")) return } diff --git a/internal/services/gateway/middleware.go b/internal/services/gateway/middleware.go index 91b9eda..fcc0fd7 100644 --- a/internal/services/gateway/middleware.go +++ b/internal/services/gateway/middleware.go @@ -25,6 +25,7 @@ func (s *Server) cors(next http.Handler) http.Handler { w.Header().Set("Vary", "Origin") w.Header().Set("Access-Control-Allow-Methods", "GET, POST, PUT, DELETE, OPTIONS") w.Header().Set("Access-Control-Allow-Headers", "Authorization, Content-Type") + w.Header().Set("Access-Control-Expose-Headers", voiceProviderHeader) w.Header().Set("Access-Control-Max-Age", "600") } if r.Method == http.MethodOptions { diff --git a/internal/services/gateway/voice.go b/internal/services/gateway/voice.go index ffc3528..a43abd3 100644 --- a/internal/services/gateway/voice.go +++ b/internal/services/gateway/voice.go @@ -28,6 +28,54 @@ type VoiceService interface { CloneVoice(ctx context.Context, referenceURL, prefix string) (string, error) } +// voiceProviderHeader is the response header carrying the provider rung on the +// binary /api/v1/audio/speech response, where a JSON signal field cannot be added. +const voiceProviderHeader = "X-Voice-Provider" + +// voiceProviderReporter is the optional extension of VoiceService: a service that +// can report which provider rung (cloud / local / edge-tts / espeak / silent) +// produced a result. *voice.Chain and *voice.Service implement it; a custom +// VoiceService that does not is still supported and falls back to the plain +// STT/TTS methods with no reported rung. +type voiceProviderReporter interface { + STTWithProvider(ctx context.Context, audio voice.Audio) (string, string, error) + TTSWithProvider(ctx context.Context, text string) (voice.Audio, string, error) +} + +// sttWithProvider runs STT and returns the transcript plus the provider rung that +// served it ("" when the injected service cannot report one). +func sttWithProvider(ctx context.Context, v VoiceService, audio voice.Audio) (string, string, error) { + if r, ok := v.(voiceProviderReporter); ok { + return r.STTWithProvider(ctx, audio) + } + text, err := v.STT(ctx, audio) + return text, "", err +} + +// ttsWithProvider runs TTS and returns the audio plus the provider rung that +// served it. +func ttsWithProvider(ctx context.Context, v VoiceService, text string) (voice.Audio, string, error) { + if r, ok := v.(voiceProviderReporter); ok { + return r.TTSWithProvider(ctx, text) + } + audio, err := v.TTS(ctx, text) + return audio, "", err +} + +// providerSignal returns the "provider" degraded signal for a voice result: the +// actual provider rung when the result is empty, "silent" when the rung is +// unknown, and "" (omit the field) when the result carries real text/audio. It +// only flags "no real speech/audio"; it is never a business error. +func providerSignal(rung string, empty bool) string { + if !empty { + return "" + } + if rung == "" { + return "silent" + } + return rung +} + // voiceInputRequest is the decoded payload of a voice_input frame. Audio is a // base64 (or data: URI) audio blob; Format is its encoding ("wav"/"mp3"/"ogg"); // Text overrides STT when present (a client that already transcribed can skip @@ -119,18 +167,22 @@ func (h *hub) runVoiceTurn(c *wsConn, req voiceInputRequest) { if strings.TrimSpace(reply) == "" { reply = res.StopReason } - audi, err := h.voice.TTS(ctx, reply) + audi, rung, err := ttsWithProvider(ctx, h.voice, reply) if err != nil { h.log.Warn("voice tts", "err", err) h.voiceError(c, "voice synthesis failed") return } - c.enqueueJSON(wsEvent{Type: wsVoiceOutput, Data: map[string]any{ + data := map[string]any{ "audio": base64.StdEncoding.EncodeToString(audi.Data), "format": audi.Format, "text": transcript, "reply": reply, - }}) + } + if sig := providerSignal(rung, len(audi.Data) == 0); sig != "" { + data["provider"] = sig + } + c.enqueueJSON(wsEvent{Type: wsVoiceOutput, Data: data}) } // voiceError replies with a typed error frame for a failed voice turn. diff --git a/internal/services/gateway/voice_signal_test.go b/internal/services/gateway/voice_signal_test.go new file mode 100644 index 0000000..edb21bc --- /dev/null +++ b/internal/services/gateway/voice_signal_test.go @@ -0,0 +1,181 @@ +package gateway + +import ( + "context" + "encoding/base64" + "encoding/json" + "net/http" + "testing" + + "git.yeij.top/AskaEth/Cyrene_v2/internal/auth" + "git.yeij.top/AskaEth/Cyrene_v2/internal/runtime/loop" + "git.yeij.top/AskaEth/Cyrene_v2/internal/runtime/multimodal/voice" +) + +// signalVoice is a VoiceService that also implements voiceProviderReporter, so a +// test can control both the STT/TTS result and the reported provider rung. +type signalVoice struct { + text string + audio voice.Audio + rung string + sttErr error + ttsErr error +} + +func (s *signalVoice) STT(_ context.Context, _ voice.Audio) (string, error) { + return s.text, s.sttErr +} + +func (s *signalVoice) TTS(_ context.Context, _ string) (voice.Audio, error) { + return s.audio, s.ttsErr +} + +func (s *signalVoice) CloneVoice(_ context.Context, _, _ string) (string, error) { return "", nil } + +func (s *signalVoice) STTWithProvider(_ context.Context, _ voice.Audio) (string, string, error) { + return s.text, s.rung, s.sttErr +} + +func (s *signalVoice) TTSWithProvider(_ context.Context, _ string) (voice.Audio, string, error) { + return s.audio, s.rung, s.ttsErr +} + +func TestHandleTranscribe_SilentSignal(t *testing.T) { + env := newEnv(t, nil, false) + env.srv.SetVoice(&signalVoice{rung: "silent"}) + + token := env.token(t, auth.RoleOwner) + resp, body := doReq(t, http.MethodPost, env.ts.URL+"/api/v1/audio/transcribe", map[string]any{ + "audio": base64.StdEncoding.EncodeToString([]byte("raw")), + }, token) + if resp.StatusCode != http.StatusOK { + t.Fatalf("status = %d, body = %s", resp.StatusCode, body) + } + var data struct { + Text string `json:"text"` + Provider string `json:"provider"` + } + if err := json.Unmarshal(decodeBody(t, body)["data"], &data); err != nil { + t.Fatalf("decode data: %v (body=%s)", err, body) + } + if data.Text != "" || data.Provider != "silent" { + t.Errorf("data = %+v, want empty text + provider silent", data) + } +} + +func TestHandleTranscribe_SilentFallbackWithoutReporter(t *testing.T) { + env := newEnv(t, nil, false) + // audioStubVoice does not implement voiceProviderReporter, so the handler + // must still flag an empty transcription as silent. + env.srv.SetVoice(&audioStubVoice{}) + + token := env.token(t, auth.RoleOwner) + resp, body := doReq(t, http.MethodPost, env.ts.URL+"/api/v1/audio/transcribe", map[string]any{ + "audio": base64.StdEncoding.EncodeToString([]byte("raw")), + }, token) + if resp.StatusCode != http.StatusOK { + t.Fatalf("status = %d, body = %s", resp.StatusCode, body) + } + var data struct { + Provider string `json:"provider"` + } + if err := json.Unmarshal(decodeBody(t, body)["data"], &data); err != nil { + t.Fatalf("decode data: %v (body=%s)", err, body) + } + if data.Provider != "silent" { + t.Errorf("provider = %q, want silent (body=%s)", data.Provider, body) + } +} + +func TestHandleTranscribe_SuccessOmitsProvider(t *testing.T) { + env := newEnv(t, nil, false) + env.srv.SetVoice(&signalVoice{text: "你好", rung: "cloud"}) + + token := env.token(t, auth.RoleOwner) + resp, body := doReq(t, http.MethodPost, env.ts.URL+"/api/v1/audio/transcribe", map[string]any{ + "audio": base64.StdEncoding.EncodeToString([]byte("raw")), + }, token) + if resp.StatusCode != http.StatusOK { + t.Fatalf("status = %d, body = %s", resp.StatusCode, body) + } + var raw map[string]json.RawMessage + if err := json.Unmarshal(decodeBody(t, body)["data"], &raw); err != nil { + t.Fatalf("decode data: %v (body=%s)", err, body) + } + if _, present := raw["provider"]; present { + t.Errorf("provider must be omitted on a non-empty transcription: %s", body) + } +} + +func TestHandleSpeech_EmptyAudioSignal(t *testing.T) { + env := newEnv(t, nil, false) + env.srv.SetVoice(&signalVoice{rung: "silent"}) + + token := env.token(t, auth.RoleOwner) + resp, body := doReq(t, http.MethodPost, env.ts.URL+"/api/v1/audio/speech", map[string]any{ + "text": "你好", + }, token) + if resp.StatusCode != http.StatusInternalServerError { + t.Fatalf("status = %d, want 500 (body=%s)", resp.StatusCode, body) + } + if got := resp.Header.Get(voiceProviderHeader); got != "silent" { + t.Errorf("%s = %q, want silent", voiceProviderHeader, got) + } + var env2 struct { + Error struct { + Code string `json:"code"` + } `json:"error"` + } + if err := json.Unmarshal(body, &env2); err != nil { + t.Fatalf("decode error: %v (body=%s)", err, body) + } + if env2.Error.Code != "tts_empty" { + t.Errorf("error code = %q, want tts_empty (body=%s)", env2.Error.Code, body) + } +} + +func TestHubVoiceOutput_SilentSignal(t *testing.T) { + t.Parallel() + hb := newTestHub(&stubRunner{res: &loop.Result{Text: "你好", Steps: 1, StopReason: "stop"}}, nil) + hb.voice = &signalVoice{text: "你好", rung: "silent"} + c := newFakeConn(hb, 32) + hb.register(c) + + c.route(wsInbound{Type: "voice_input", Data: json.RawMessage(`{"audio":"aGVsbG8=","format":"wav"}`)}) + ev := readFakeFrame(t, c) + if ev.Type != wsVoiceOutput { + t.Fatalf("frame type = %q, want voice_output (frame=%+v)", ev.Type, ev) + } + r, ok := ev.Data.(map[string]any) + if !ok { + t.Fatalf("data = %T, want map[string]any", ev.Data) + } + if r["provider"] != "silent" { + t.Errorf("provider = %v, want silent (frame=%+v)", r["provider"], ev.Data) + } +} + +func TestHubVoiceOutput_SuccessOmitsProvider(t *testing.T) { + t.Parallel() + hb := newTestHub(&stubRunner{res: &loop.Result{Text: "你好", Steps: 1, StopReason: "stop"}}, nil) + hb.voice = &signalVoice{ + text: "你好", + audio: voice.Audio{Format: "wav", Data: []byte("audio-bytes")}, + rung: "edge-tts", + } + c := newFakeConn(hb, 32) + hb.register(c) + + c.route(wsInbound{Type: "voice_input", Data: json.RawMessage(`{"audio":"aGVsbG8=","format":"wav"}`)}) + ev := readFakeFrame(t, c) + if ev.Type != wsVoiceOutput { + t.Fatalf("frame type = %q, want voice_output (frame=%+v)", ev.Type, ev) + } + r, ok := ev.Data.(map[string]any) + if !ok { + t.Fatalf("data = %T, want map[string]any", ev.Data) + } + if _, present := r["provider"]; present { + t.Errorf("provider must be omitted for non-empty audio: %+v", ev.Data) + } +}