feat: admin 端点补齐 + 视觉真推理 + 语音链路 + wire.go 收口(A+B+第3步)
批次A admin 端点(原本 404 变 200 真实数据):
- GET /admin/llm:读 s.cfg.LLM 组 openai+anthropic 两 provider(enabled=当前 Provider、multimodal=配 VisionModel),APIKey 不进 wire 零泄露。
- GET /admin/automations:*automations.Service.List()(reminders+event rules+cron schedules,cron 加只读登记)。
- GET /admin/users:cfg.AdminUser + *auth.APIKey.List()(密钥掩码);无用户存储,恒先返回当前 admin 账号。
- GET /admin/backthink:*backthink.Backthink.List(ctx,50)。均只读、owner/admin RBAC、{data} 信封。
批次B 视觉真推理 + 语音:
- rest.go chatRequest 加 Image?;流式 publishChatStream 带 image;userChatMessages 构造 ImageMessage(model.go:71)。
- loop/vision.go:ImageResolver 可注入(MVP 先看图再对话);run.go Run 开头 resolveImages,无 resolver 透传直交主模型。
- vision/resolver.go:MessageResolver 调 vision.Router.Route——KindMain 保留图片交主模型多模态、KindVisionTool/OCRTool 经 dispatcher.Invoke 折叠成 [图片内容] 文本、KindUnsupported 记日志保留原图不崩。
- gateway/voice.go:VoiceService 接口 + SetVoice 注入点 + handleVoiceInput(未注入回 error 不崩)+ runVoiceTurn 异步 STT→loop chat→TTS→定向 voice_output 帧。
- ws_hub/ws_route:voice_input 接 handleVoiceInput;前端 ws.ts decodeBase64/playVoiceAudio/case voice_output。
第3步 wire.go 收口(唯一碰 wire.go 的一步):
- Deps 补传 Automations:a.automation / Backthink:a.backthink / Users:auth.NewAPIKey(a.cfg.Auth)(*auth.APIKey 原生满足 UserStore,避免 admin 重复的复合类型)。
- gw.SetVoice(a.voice);loop.Options.Vision = vision.NewMessageResolver(router, info, dispatcher.Invoke)(router=NewRouter(WithVisionTool/WithOCRTool)、info=cfg.LLM.VisionModel)。
实机验证:四 admin 端点 200 真实数据(llm 双 provider 脱敏、users=admin,yeij0942,owner);WS 握手正常。go build/vet/test 全绿 + 前端 tsc。wire.go 557 行。
This commit is contained in:
1 parent
648292d035
commit
518bd0a230
21 files changed
+976
-38
No files matched your search
+11
-8
@@ -21,7 +21,7 @@
|
|||||||
|---|------|------|------|:---:|------|
|
|---|------|------|------|:---:|------|
|
||||||
| 1 | `/api/v1/auth/login` | POST | 否 | ✅ | `{username,password}` → `{data:{token,jwt}}`(24h,owner) |
|
| 1 | `/api/v1/auth/login` | POST | 否 | ✅ | `{username,password}` → `{data:{token,jwt}}`(24h,owner) |
|
||||||
| 2 | `/api/v1/health` | GET | 否 | ✅ | 健康检查 `{data:{status:"ok"}}` |
|
| 2 | `/api/v1/health` | GET | 否 | ✅ | 健康检查 `{data:{status:"ok"}}` |
|
||||||
| 3 | `/api/v1/chat` | POST | JWT | ✅ | `{text,mode?,stream?}` → `{data:{reply}}`;`stream:true` 时经 WS 触发流式 |
|
| 3 | `/api/v1/chat` | POST | JWT | ✅ | `{text,image?,stream?}` → `{data:{reply}}`;`stream:true` 时经 WS 触发流式。`image` 为 base64 data: URI 或 http(s) URL,入视觉真推理(design/06) |
|
||||||
| 4 | `/api/v1/sessions` | GET | JWT | ✅ | `{data:[session...]}` |
|
| 4 | `/api/v1/sessions` | GET | JWT | ✅ | `{data:[session...]}` |
|
||||||
| 5 | `/api/v1/personas` | GET | JWT | ✅ | `{data:[persona...]}` |
|
| 5 | `/api/v1/personas` | GET | JWT | ✅ | `{data:[persona...]}` |
|
||||||
|
|
||||||
@@ -38,17 +38,17 @@
|
|||||||
| 5 | `/api/v1/admin/plugins/policy` | POST | ✅ | 写工具策略(owner-only,审计) |
|
| 5 | `/api/v1/admin/plugins/policy` | POST | ✅ | 写工具策略(owner-only,审计) |
|
||||||
| 6 | `/api/v1/admin/channels` | GET | ✅ | 渠道列表/健康 |
|
| 6 | `/api/v1/admin/channels` | GET | ✅ | 渠道列表/健康 |
|
||||||
| 7 | `/api/v1/admin/iot` | GET | ✅ | IoT 设备列表 |
|
| 7 | `/api/v1/admin/iot` | GET | ✅ | IoT 设备列表 |
|
||||||
| 8 | `/api/v1/admin/users` | GET/POST | 🕐 | 用户/角色/API key(M10) |
|
| 8 | `/api/v1/admin/users` | GET/POST | ✅(GET) | 用户/角色/API key(GET 已实现;POST 创建用户/签发 key 随 M10 落地) |
|
||||||
| 9 | `/api/v1/admin/llm` | GET/PUT | 🕐 | LLM 模型/路由/降级(M10) |
|
| 9 | `/api/v1/admin/llm` | GET/PUT | ✅(GET) | LLM 模型/路由/降级(GET 已实现,APIKey 脱敏;PUT 热重载 M10) |
|
||||||
| 10 | `/api/v1/admin/automations` | GET/POST | 🕐 | 自动化与提醒(M10) |
|
| 10 | `/api/v1/admin/automations` | GET/POST | ✅(GET) | 自动化与提醒(GET 已实现;POST 创建 M10) |
|
||||||
|
| 11 | `/api/v1/admin/backthink` | GET | ✅ | 后台思考记录 |
|
||||||
|
|
||||||
> **M10 前端契约补充(管理面板,均🕐未实现,前端按契约封装、后端 404 时显示"待实现"壳,不报错)**:
|
> **M10 前端契约补充(管理面板)**:`GET /api/v1/admin/users` · `GET /api/v1/admin/llm` · `GET /api/v1/admin/automations` · `GET /api/v1/admin/backthink` 已实现,返回真实数据;`GET /api/v1/admin/logs` · `GET /api/v1/admin/trace` · `GET /api/v1/admin/audit` · `GET /api/v1/admin/system` 仍🕐未实现(前端按契约封装、后端 404 时显示"待实现"壳,不报错)。写操作(策略/档案/密钥/系统设置/用户与自动化创建)经前端审计登记,待后端 M10+ 落地真实持久化与热重载。
|
||||||
> `GET /api/v1/admin/users`(用户/角色/API key)· `GET /api/v1/admin/llm` · `GET /api/v1/admin/automations` · `GET /api/v1/admin/backthink` · `GET /api/v1/admin/logs` · `GET /api/v1/admin/trace` · `GET /api/v1/admin/audit` · `GET /api/v1/admin/system`。写操作(策略/档案/密钥/系统设置)经前端审计登记,待后端 M10+ 落地真实持久化与热重载。
|
|
||||||
|
|
||||||
## 3. WebSocket `/ws/chat`(实时)
|
## 3. WebSocket `/ws/chat`(实时)
|
||||||
|
|
||||||
- **握手**: `ws://<host>:8080/ws/chat?token=<JWT>` 或 `Authorization: Bearer`;未认证在握手前回 `401` JSON。
|
- **握手**: `ws://<host>:8080/ws/chat?token=<JWT>` 或 `Authorization: Bearer`;未认证在握手前回 `401` JSON。
|
||||||
- **下行(服务端→客户端)**: 经 bus 广播 `{type:"<event>", data:..., turnId?, seq?}`(`turnId`/`seq` 为流式关联字段,见 §3.1)——`message.start/delta/end` / `thinking.start/delta/end` / `tool_call.start/delta/end` / `tool_execution.update`(结构化流,design/04 §1.1),以及向后兼容的 `stream_chunk` / `stream_end` / `thinking`(思维链) / `mood` / `reminder` / `device_update` / `tool_used`(工具回显) / `subagent_event` / `graph_update`。
|
- **下行(服务端→客户端)**: 经 bus 广播 `{type:"<event>", data:..., turnId?, seq?}`(`turnId`/`seq` 为流式关联字段,见 §3.1)——`message.start/delta/end` / `thinking.start/delta/end` / `tool_call.start/delta/end` / `tool_execution.update`(结构化流,design/04 §1.1),以及向后兼容的 `stream_chunk` / `stream_end` / `thinking`(思维链) / `mood` / `reminder` / `device_update` / `tool_used`(工具回显) / `subagent_event` / `graph_update`。语音回复走 `voice_output`(定向回请求连接,非广播)。
|
||||||
|
|
||||||
### 3.1 结构化流事件(`message.*` / `thinking.*` / `tool_call.*` / `tool_execution.*`)
|
### 3.1 结构化流事件(`message.*` / `thinking.*` / `tool_call.*` / `tool_execution.*`)
|
||||||
- **事件壳**:`{type, turnId?, seq?, data}`, `data` 为**类型化结构**(非 `any`)。`seq` 为 hub 级单调递增序号,供客户端将一次突发的流式事件排序关联。
|
- **事件壳**:`{type, turnId?, seq?, data}`, `data` 为**类型化结构**(非 `any`)。`seq` 为 hub 级单调递增序号,供客户端将一次突发的流式事件排序关联。
|
||||||
@@ -58,7 +58,9 @@
|
|||||||
- **`tool_execution.update`**(`{id, index, delta}`):工具执行进度/局部输出(保留,工具循环 §1.2 后置实现)。
|
- **`tool_execution.update`**(`{id, index, delta}`):工具执行进度/局部输出(保留,工具循环 §1.2 后置实现)。
|
||||||
- **向后兼容**:未映射/无法确定的事件仍走 `stream_chunk` + `stream_end`(非流式回退路径)。
|
- **向后兼容**:未映射/无法确定的事件仍走 `stream_chunk` + `stream_end`(非流式回退路径)。
|
||||||
- **上行(客户端→服务端)**: `mode_switch` / `chat`(跑 loop 并广播流式) / `voice_input` / `subagent_ask` / `task_output_query`。
|
- **上行(客户端→服务端)**: `mode_switch` / `chat`(跑 loop 并广播流式) / `voice_input` / `subagent_ask` / `task_output_query`。
|
||||||
- ⚠️ 语音/流式完整 schema 随 M9 前端联调补充;会话级定向(当前对所有连接广播)为后续项。
|
- **`voice_input`**(上行,客户端→服务端):`_{audio?, format?, text?, sessionId?}`。`audio` 为 base64(或 data: URI)音频,`format` 为其编码(`wav`/`mp3`/`ogg`);`text` 为已转写文本时跳过 STT。服务端走 STT→agent 聊天→TTS(design/06 §1),回复经 `voice_output` 定向回该连接。无注入语音服务时回 `error` 帧(`voice unavailable`)。
|
||||||
|
- **`voice_output`**(下行,服务端→客户端):`{audio, format, text, reply}`。`audio` 为 base64 合成音频,`format` 为其编码,`text` 为识别文本,`reply` 为助手回复文本。
|
||||||
|
- **向后兼容**:未映射/无法确定的事件仍走 `stream_chunk` + `stream_end`(非流式回退路径)。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -68,6 +70,7 @@
|
|||||||
- **离线**: PWA / 原生端建议缓存 `sessions`/`personas` 列表。
|
- **离线**: PWA / 原生端建议缓存 `sessions`/`personas` 列表。
|
||||||
|
|
||||||
## 5. 变更记录
|
## 5. 变更记录
|
||||||
|
- `v0.5`(批次B·视觉真推理 + 语音端):`POST /api/v1/chat` 增 `image` 字段(base64 data: URI 或 URL),入 loop 视觉路由(vision/OCR 工具或主模型多模态,design/06 §0「先看图再对话」);WS 上行 `voice_input` 接 STT→聊天→TTS 管道,下行新增 `voice_output`(定向回请求连接,含合成音频 + 识别文本 + 回复文本)。
|
||||||
- `v0.4`(结构化流):WS 下行新增 `message.*` / `thinking.*` / `tool_call.*` / `tool_execution.update` 结构化流事件(event shell 补 `turnId`/`seq`,`data` 类型化),对齐 OpenClaw AssistantMessageEvent;保留 `stream_chunk`/`stream_end` 向后兼容。
|
- `v0.4`(结构化流):WS 下行新增 `message.*` / `thinking.*` / `tool_call.*` / `tool_execution.update` 结构化流事件(event shell 补 `turnId`/`seq`,`data` 类型化),对齐 OpenClaw AssistantMessageEvent;保留 `stream_chunk`/`stream_end` 向后兼容。
|
||||||
- `v0.3`(M10):管理面板 `/admin` 落地——按 §2 契约封装 admin 只读端点(personas/sessions/memories/plugins/channels/iot)+ owner-only `plugins/policy` 写;其余 M10 端点以契约路径封装、后端未实现时前端显式"待实现"壳。
|
- `v0.3`(M10):管理面板 `/admin` 落地——按 §2 契约封装 admin 只读端点(personas/sessions/memories/plugins/channels/iot)+ owner-only `plugins/policy` 写;其余 M10 端点以契约路径封装、后端未实现时前端显式"待实现"壳。
|
||||||
- `v0.2`(M8):gateway 落地——登录/health/chat/sessions/personas + admin 只读端点 + `plugins/policy` 写 + WS `/ws/chat`(JWT + bus 广播)。
|
- `v0.2`(M8):gateway 落地——登录/health/chat/sessions/personas + admin 只读端点 + `plugins/policy` 写 + WS `/ws/chat`(JWT + bus 广播)。
|
||||||
|
|||||||
+35
-13
@@ -31,6 +31,7 @@ import (
|
|||||||
"git.yeij.top/AskaEth/Cyrene_v2/internal/runtime/memory/projectindex"
|
"git.yeij.top/AskaEth/Cyrene_v2/internal/runtime/memory/projectindex"
|
||||||
"git.yeij.top/AskaEth/Cyrene_v2/internal/runtime/memory/vector"
|
"git.yeij.top/AskaEth/Cyrene_v2/internal/runtime/memory/vector"
|
||||||
"git.yeij.top/AskaEth/Cyrene_v2/internal/runtime/model"
|
"git.yeij.top/AskaEth/Cyrene_v2/internal/runtime/model"
|
||||||
|
"git.yeij.top/AskaEth/Cyrene_v2/internal/runtime/multimodal/vision"
|
||||||
"git.yeij.top/AskaEth/Cyrene_v2/internal/runtime/multimodal/voice"
|
"git.yeij.top/AskaEth/Cyrene_v2/internal/runtime/multimodal/voice"
|
||||||
"git.yeij.top/AskaEth/Cyrene_v2/internal/runtime/orchestration"
|
"git.yeij.top/AskaEth/Cyrene_v2/internal/runtime/orchestration"
|
||||||
"git.yeij.top/AskaEth/Cyrene_v2/internal/runtime/orchestration/audit"
|
"git.yeij.top/AskaEth/Cyrene_v2/internal/runtime/orchestration/audit"
|
||||||
@@ -297,6 +298,21 @@ func (a *app) wireRuntime() error {
|
|||||||
return fmt.Errorf("checkpoint store: %w", perr)
|
return fmt.Errorf("checkpoint store: %w", perr)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Vision routing: the loop rewrites an image-bearing user message per the
|
||||||
|
// vision Router — a vision-capable main model sees the image directly,
|
||||||
|
// otherwise the vision/OCR tool's text description replaces it (design/06 §0).
|
||||||
|
visionRouter := vision.NewRouter(
|
||||||
|
vision.WithVisionTool(visionToolName),
|
||||||
|
vision.WithOCRTool(ocrToolName),
|
||||||
|
)
|
||||||
|
visionResolver := vision.NewMessageResolver(
|
||||||
|
visionRouter,
|
||||||
|
func() vision.ModelInfo {
|
||||||
|
return vision.ModelInfo{Name: a.cfg.LLM.VisionModel, SupportsVision: a.cfg.LLM.VisionModel != ""}
|
||||||
|
},
|
||||||
|
a.dispatcher.Invoke,
|
||||||
|
)
|
||||||
|
|
||||||
a.loop = loop.New(loop.Options{
|
a.loop = loop.New(loop.Options{
|
||||||
Agent: ag,
|
Agent: ag,
|
||||||
Tools: a.dispatcher,
|
Tools: a.dispatcher,
|
||||||
@@ -310,6 +326,7 @@ func (a *app) wireRuntime() error {
|
|||||||
MaxRounds: defaultMaxRounds,
|
MaxRounds: defaultMaxRounds,
|
||||||
WorkingMemory: memory.NewWorkingMemory(0),
|
WorkingMemory: memory.NewWorkingMemory(0),
|
||||||
OnStream: a.streamChunk,
|
OnStream: a.streamChunk,
|
||||||
|
Vision: visionResolver,
|
||||||
Log: logger.WithSource(a.log, "runtime/loop"),
|
Log: logger.WithSource(a.log, "runtime/loop"),
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -417,24 +434,29 @@ func (a *app) wireChannels() error {
|
|||||||
// managers so the admin/user list endpoints return real data.
|
// managers so the admin/user list endpoints return real data.
|
||||||
func (a *app) wireFrontend() error {
|
func (a *app) wireFrontend() error {
|
||||||
jwt := auth.New(a.cfg.Auth)
|
jwt := auth.New(a.cfg.Auth)
|
||||||
|
apiKeys := auth.NewAPIKey(a.cfg.Auth)
|
||||||
gw, err := gateway.New(a.cfg, gateway.Deps{
|
gw, err := gateway.New(a.cfg, gateway.Deps{
|
||||||
Log: logger.WithSource(a.log, "gateway"),
|
Log: logger.WithSource(a.log, "gateway"),
|
||||||
Auth: jwt,
|
Auth: jwt,
|
||||||
Bus: a.bus,
|
Bus: a.bus,
|
||||||
Registry: a.caps,
|
Registry: a.caps,
|
||||||
Loop: a.loop,
|
Loop: a.loop,
|
||||||
Session: a.sessions,
|
Session: a.sessions,
|
||||||
Personas: a.personas,
|
Personas: a.personas,
|
||||||
Memories: a.memStore,
|
Memories: a.memStore,
|
||||||
IoT: a.iot,
|
IoT: a.iot,
|
||||||
Scheduler: a.scheduler,
|
Scheduler: a.scheduler,
|
||||||
Tasks: a.tasks,
|
Tasks: a.tasks,
|
||||||
Streaming: true,
|
Automations: a.automation,
|
||||||
Frontend: resolveFrontendFS(a.log),
|
Users: apiKeys,
|
||||||
|
Backthink: a.backthink,
|
||||||
|
Streaming: true,
|
||||||
|
Frontend: resolveFrontendFS(a.log),
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("gateway: %w", err)
|
return fmt.Errorf("gateway: %w", err)
|
||||||
}
|
}
|
||||||
|
gw.SetVoice(a.voice)
|
||||||
a.gw = gw
|
a.gw = gw
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -37,6 +37,34 @@ function toRecord(v: unknown): Record<string, unknown> | null {
|
|||||||
return null;
|
return null;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** Decode a base64 string into bytes (used for the voice audio reply). */
|
||||||
|
function decodeBase64(b64: string): Uint8Array {
|
||||||
|
const bin = atob(b64);
|
||||||
|
const bytes = new Uint8Array(bin.length);
|
||||||
|
for (let i = 0; i < bin.length; i++) bytes[i] = bin.charCodeAt(i);
|
||||||
|
return bytes;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Best-effort playback of a voice_output audio payload (blocked by autoplay
|
||||||
|
* policy, so sound may be silent until the user interacts). */
|
||||||
|
function playVoiceAudio(payload: unknown): void {
|
||||||
|
const r = toRecord(payload);
|
||||||
|
if (!r || typeof r.audio !== "string" || !r.audio) return;
|
||||||
|
try {
|
||||||
|
const bytes = decodeBase64(r.audio);
|
||||||
|
const format = typeof r.format === "string" && r.format ? r.format : "wav";
|
||||||
|
const buf = new ArrayBuffer(bytes.byteLength);
|
||||||
|
new Uint8Array(buf).set(bytes);
|
||||||
|
const blob = new Blob([buf], { type: `audio/${format}` });
|
||||||
|
const url = URL.createObjectURL(blob);
|
||||||
|
const audio = new Audio(url);
|
||||||
|
void audio.play().catch(() => {});
|
||||||
|
audio.onended = () => URL.revokeObjectURL(url);
|
||||||
|
} catch {
|
||||||
|
// Ignore malformed audio payloads.
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
function parseTool(v: unknown): ToolUsed | null {
|
function parseTool(v: unknown): ToolUsed | null {
|
||||||
const r = toRecord(v);
|
const r = toRecord(v);
|
||||||
if (!r) return null;
|
if (!r) return null;
|
||||||
@@ -253,6 +281,18 @@ function dispatch(ev: { type: string; data: unknown }): void {
|
|||||||
}
|
}
|
||||||
break;
|
break;
|
||||||
}
|
}
|
||||||
|
case "voice_output": {
|
||||||
|
// Voice turn reply: the backend sends back the synthesized audio plus the
|
||||||
|
// recognized transcript and the assistant reply. Playback is best-effort
|
||||||
|
// (autoplay may be blocked); the transcript/reply is surfaced as a log line.
|
||||||
|
playVoiceAudio(ev.data);
|
||||||
|
if (r && typeof r.reply === "string") {
|
||||||
|
s.addDeviceUpdate(`语音回复:${r.reply}`);
|
||||||
|
} else if (r && typeof r.text === "string") {
|
||||||
|
s.addDeviceUpdate(`语音识别:${r.text}`);
|
||||||
|
}
|
||||||
|
break;
|
||||||
|
}
|
||||||
default: {
|
default: {
|
||||||
// Unknown future event: keep it in the context log so no frame is lost.
|
// Unknown future event: keep it in the context log so no frame is lost.
|
||||||
const text = typeof ev.data === "string" ? ev.data : JSON.stringify(ev.data);
|
const text = typeof ev.data === "string" ? ev.data : JSON.stringify(ev.data);
|
||||||
|
|||||||
@@ -71,3 +71,26 @@ func (k *APIKey) Revoke(key string) {
|
|||||||
defer k.mu.Unlock()
|
defer k.mu.Unlock()
|
||||||
delete(k.keys, key)
|
delete(k.keys, key)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// List returns a summary of every currently registered API key for the admin
|
||||||
|
// panel. Key material is never returned; each identifier is masked so the panel
|
||||||
|
// can tell keys apart without exposing the secret. The in-memory store keeps no
|
||||||
|
// creation time, so CreatedAt is nil for every entry.
|
||||||
|
func (k *APIKey) List() []UserSummary {
|
||||||
|
k.mu.RLock()
|
||||||
|
defer k.mu.RUnlock()
|
||||||
|
out := make([]UserSummary, 0, len(k.keys))
|
||||||
|
for key, role := range k.keys {
|
||||||
|
out = append(out, UserSummary{ID: maskKey(key), Username: "apikey", Role: role})
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
// maskKey renders a key's leading and trailing groups so the panel can identify
|
||||||
|
// an entry without revealing the secret. Short keys are fully hidden.
|
||||||
|
func maskKey(key string) string {
|
||||||
|
if len(key) <= 8 {
|
||||||
|
return "••••"
|
||||||
|
}
|
||||||
|
return key[:4] + "••••" + key[len(key)-4:]
|
||||||
|
}
|
||||||
@@ -12,6 +12,7 @@ package auth
|
|||||||
import (
|
import (
|
||||||
"errors"
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
|
"time"
|
||||||
)
|
)
|
||||||
|
|
||||||
// Role is an authorization role in the owner>admin>viewer hierarchy.
|
// Role is an authorization role in the owner>admin>viewer hierarchy.
|
||||||
@@ -91,3 +92,14 @@ func (r Role) rank() int {
|
|||||||
func HasRole(role, required Role) bool {
|
func HasRole(role, required Role) bool {
|
||||||
return role.rank() >= required.rank()
|
return role.rank() >= required.rank()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// UserSummary is the read-only shape of an account or API key visible to the
|
||||||
|
// admin panel (GET /api/v1/admin/users). CreatedAt is nil when the source does
|
||||||
|
// not track a creation time (the configured admin account and in-memory keys
|
||||||
|
// both omit it).
|
||||||
|
type UserSummary struct {
|
||||||
|
ID string
|
||||||
|
Username string
|
||||||
|
Role Role
|
||||||
|
CreatedAt *time.Time
|
||||||
|
}
|
||||||
@@ -97,6 +97,7 @@ type Loop struct {
|
|||||||
maxContextTokens int
|
maxContextTokens int
|
||||||
compactor Compactor
|
compactor Compactor
|
||||||
transformContext func(context.Context, []model.Message, int) []model.Message
|
transformContext func(context.Context, []model.Message, int) []model.Message
|
||||||
|
imageResolver ImageResolver
|
||||||
log *slog.Logger
|
log *slog.Logger
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -141,6 +142,7 @@ func New(opts Options) *Loop {
|
|||||||
maxContextTokens: opts.MaxContextTokens,
|
maxContextTokens: opts.MaxContextTokens,
|
||||||
compactor: compactor,
|
compactor: compactor,
|
||||||
transformContext: opts.TransformContext,
|
transformContext: opts.TransformContext,
|
||||||
|
imageResolver: opts.Vision,
|
||||||
log: sourcedLog(opts.Log),
|
log: sourcedLog(opts.Log),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,65 @@
|
|||||||
|
package loop
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"git.yeij.top/AskaEth/Cyrene_v2/internal/runtime/model"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestRun_ImageResolverFoldsToText(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
ad := &mockAdapter{chat: []mgResp{{resp: model.Response{Model: "m", Content: "Done."}}}}
|
||||||
|
a := newTestAgent(t, ad, nil)
|
||||||
|
l := New(Options{
|
||||||
|
Agent: a,
|
||||||
|
MaxRounds: 8,
|
||||||
|
Vision: ImageResolveFunc(func(_ context.Context, _ model.Message) (model.Message, error) {
|
||||||
|
return model.TextMessage(model.RoleUser, "图中是一只猫"), nil
|
||||||
|
}),
|
||||||
|
})
|
||||||
|
|
||||||
|
in := []model.Message{model.ImageMessage(model.RoleUser, "", "data:image/png;base64,AAAA")}
|
||||||
|
if _, err := l.Run(context.Background(), "sess-1", in); err != nil {
|
||||||
|
t.Fatalf("Run: %v", err)
|
||||||
|
}
|
||||||
|
if len(ad.reqs) == 0 {
|
||||||
|
t.Fatalf("no model request recorded")
|
||||||
|
}
|
||||||
|
var userSeen bool
|
||||||
|
for _, m := range ad.reqs[0].Messages {
|
||||||
|
if m.Role == model.RoleUser {
|
||||||
|
userSeen = true
|
||||||
|
if messageHasImage(m) {
|
||||||
|
t.Fatalf("image still reached the model: %+v", m)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !userSeen {
|
||||||
|
t.Fatalf("no user message seen by the model")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRun_ImageKeptWithoutResolver(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
ad := &mockAdapter{chat: []mgResp{{resp: model.Response{Model: "m", Content: "Done."}}}}
|
||||||
|
a := newTestAgent(t, ad, nil)
|
||||||
|
l := New(Options{Agent: a, MaxRounds: 8})
|
||||||
|
|
||||||
|
in := []model.Message{model.ImageMessage(model.RoleUser, "look", "data:image/png;base64,AAAA")}
|
||||||
|
if _, err := l.Run(context.Background(), "sess-1", in); err != nil {
|
||||||
|
t.Fatalf("Run: %v", err)
|
||||||
|
}
|
||||||
|
if len(ad.reqs) == 0 {
|
||||||
|
t.Fatalf("no model request recorded")
|
||||||
|
}
|
||||||
|
var sawImage bool
|
||||||
|
for _, m := range ad.reqs[0].Messages {
|
||||||
|
if m.Role == model.RoleUser && messageHasImage(m) {
|
||||||
|
sawImage = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !sawImage {
|
||||||
|
t.Fatalf("expected the image part to pass through without a resolver")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -77,6 +77,13 @@ type Options struct {
|
|||||||
// an LLM-backed or custom summarizer. It is only invoked when
|
// an LLM-backed or custom summarizer. It is only invoked when
|
||||||
// MaxContextTokens is positive.
|
// MaxContextTokens is positive.
|
||||||
Compactor Compactor
|
Compactor Compactor
|
||||||
|
// Vision, when non-nil, rewrites an image-bearing user message at the start
|
||||||
|
// of Run so the main model can reason over the image (design/06 §0). The
|
||||||
|
// concrete resolver (vision.MessageResolver) decides via the vision Router
|
||||||
|
// whether to keep the image for a vision-capable main model or fold it into a
|
||||||
|
// text description from the vision/OCR tool ("先看图再对话"). Nil keeps the
|
||||||
|
// image content as-is, passing it through to a vision-capable main model.
|
||||||
|
Vision ImageResolver
|
||||||
// TransformContext, when non-nil, is called immediately before every model
|
// TransformContext, when non-nil, is called immediately before every model
|
||||||
// step with the assembled message list (after memory injection and any
|
// step with the assembled message list (after memory injection and any
|
||||||
// compaction) and the current round number (1-based). It may project, filter
|
// compaction) and the current round number (1-based). It may project, filter
|
||||||
|
|||||||
@@ -31,7 +31,7 @@ func (l *Loop) Run(ctx context.Context, sessionID string, messages []model.Messa
|
|||||||
return nil, fmt.Errorf("loop: run: %w", ErrNilAgent)
|
return nil, fmt.Errorf("loop: run: %w", ErrNilAgent)
|
||||||
}
|
}
|
||||||
|
|
||||||
msgs := messages
|
msgs := l.resolveImages(ctx, messages)
|
||||||
toolState := &toolLoopState{prevFP: make(map[string]bool)}
|
toolState := &toolLoopState{prevFP: make(map[string]bool)}
|
||||||
res := &Result{}
|
res := &Result{}
|
||||||
for step := 1; step <= l.maxRounds; step++ {
|
for step := 1; step <= l.maxRounds; step++ {
|
||||||
|
|||||||
@@ -0,0 +1,71 @@
|
|||||||
|
package loop
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
|
||||||
|
"git.yeij.top/AskaEth/Cyrene_v2/internal/runtime/model"
|
||||||
|
)
|
||||||
|
|
||||||
|
// ImageResolver is the loop's seam to the multimodal vision layer (design/06
|
||||||
|
// §0). It is called once at the start of Run for every message that carries an
|
||||||
|
// image content part, and may rewrite that message so the main model can reason
|
||||||
|
// over the image. The concrete resolver lives in the vision package and decides,
|
||||||
|
// via the vision Router, whether to keep the image for a vision-capable main
|
||||||
|
// model or fold it into a text description produced by the vision/OCR tool ("先
|
||||||
|
// 看图再对话"). Nil leaves the image content untouched so it flows straight to a
|
||||||
|
// vision-capable main model.
|
||||||
|
type ImageResolver interface {
|
||||||
|
// Resolve rewrites msg, which carries at least one image content part. It
|
||||||
|
// returns the (possibly rewritten) message. An error leaves the original
|
||||||
|
// message in place so the turn still proceeds.
|
||||||
|
Resolve(ctx context.Context, msg model.Message) (model.Message, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ImageResolveFunc adapts a function to the ImageResolver interface.
|
||||||
|
type ImageResolveFunc func(ctx context.Context, msg model.Message) (model.Message, error)
|
||||||
|
|
||||||
|
// Resolve implements ImageResolver.
|
||||||
|
func (f ImageResolveFunc) Resolve(ctx context.Context, msg model.Message) (model.Message, error) {
|
||||||
|
return f(ctx, msg)
|
||||||
|
}
|
||||||
|
|
||||||
|
// resolveImages rewrites every image-bearing message in messages using the
|
||||||
|
// injected resolver. It returns the input slice unchanged when no resolver is
|
||||||
|
// configured, no image is present, or a message fails to resolve (the failure is
|
||||||
|
// logged and the original message is kept so the turn still runs). A fresh slice
|
||||||
|
// is returned when any message changed.
|
||||||
|
func (l *Loop) resolveImages(ctx context.Context, messages []model.Message) []model.Message {
|
||||||
|
if l.imageResolver == nil {
|
||||||
|
return messages
|
||||||
|
}
|
||||||
|
out := make([]model.Message, len(messages))
|
||||||
|
changed := false
|
||||||
|
for i, m := range messages {
|
||||||
|
if !messageHasImage(m) {
|
||||||
|
out[i] = m
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
resolved, err := l.imageResolver.Resolve(ctx, m)
|
||||||
|
if err != nil {
|
||||||
|
l.log.Warn("image resolve", "error", err)
|
||||||
|
out[i] = m
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
out[i] = resolved
|
||||||
|
changed = true
|
||||||
|
}
|
||||||
|
if !changed {
|
||||||
|
return messages
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
// messageHasImage reports whether any content part of msg is an image part.
|
||||||
|
func messageHasImage(m model.Message) bool {
|
||||||
|
for _, part := range m.Content {
|
||||||
|
if part.Type == model.ContentTypeImage {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
@@ -0,0 +1,123 @@
|
|||||||
|
package vision
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
|
||||||
|
"git.yeij.top/AskaEth/Cyrene_v2/internal/runtime/model"
|
||||||
|
)
|
||||||
|
|
||||||
|
// ModelInfoProvider supplies the main model's vision capability so the resolver
|
||||||
|
// can route an image the way the vision Router decides (design/06 §0). It is
|
||||||
|
// called per image; the same provider is shared across a single resolution.
|
||||||
|
type ModelInfoProvider func() ModelInfo
|
||||||
|
|
||||||
|
// ToolInvoker invokes a registered tool (the vision or OCR tool) by name with
|
||||||
|
// the given arguments. It is satisfied by *tool.Dispatcher.Invoke.
|
||||||
|
type ToolInvoker func(ctx context.Context, name string, args map[string]any) (any, error)
|
||||||
|
|
||||||
|
// MessageResolver is the loop's concrete ImageResolver. For each image part in a
|
||||||
|
// user message it asks the vision Router how to route it: when the main model
|
||||||
|
// supports vision and is confident, the image part is kept so the main model sees
|
||||||
|
// the image; otherwise the vision/OCR tool is invoked and its text result replaces
|
||||||
|
// the image part, so a text-only main model can reason over the image ("先看图再
|
||||||
|
// 对话"). It is safe for concurrent use.
|
||||||
|
type MessageResolver struct {
|
||||||
|
router *Router
|
||||||
|
info ModelInfoProvider
|
||||||
|
invoke ToolInvoker
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewMessageResolver builds a MessageResolver. router decides the layer; info
|
||||||
|
// reports the main model's vision capability; invoke runs a registered vision/OCR
|
||||||
|
// tool. A nil invoke leaves any tool route failed (the image is then kept), so a
|
||||||
|
// resolver without tools degrades to the main-model path.
|
||||||
|
func NewMessageResolver(router *Router, info ModelInfoProvider, invoke ToolInvoker) *MessageResolver {
|
||||||
|
return &MessageResolver{
|
||||||
|
router: router,
|
||||||
|
info: info,
|
||||||
|
invoke: invoke,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Resolve rewrites msg so every image part is either kept (main model sees it) or
|
||||||
|
// replaced by a text description produced by the vision/OCR tool. Text parts are
|
||||||
|
// preserved in order. It errors only when an image cannot be routed or a tool
|
||||||
|
// invocation fails; the caller (loop) then keeps the original message.
|
||||||
|
func (r *MessageResolver) Resolve(ctx context.Context, msg model.Message) (model.Message, error) {
|
||||||
|
if r.router == nil {
|
||||||
|
return msg, nil
|
||||||
|
}
|
||||||
|
out := make([]model.Content, 0, len(msg.Content))
|
||||||
|
for _, part := range msg.Content {
|
||||||
|
if part.Type != model.ContentTypeImage {
|
||||||
|
out = append(out, part)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
replacement, err := r.resolveImage(ctx, part.ImageURL)
|
||||||
|
if err != nil {
|
||||||
|
return model.Message{}, err
|
||||||
|
}
|
||||||
|
out = append(out, replacement)
|
||||||
|
}
|
||||||
|
return model.Message{Role: msg.Role, Content: out}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// resolveImage picks a route and either keeps the image (KindMain) or folds it
|
||||||
|
// into a text description via the vision/OCR tool.
|
||||||
|
func (r *MessageResolver) resolveImage(ctx context.Context, imageURL string) (model.Content, error) {
|
||||||
|
target := ModelInfo{}
|
||||||
|
if r.info != nil {
|
||||||
|
target = r.info()
|
||||||
|
}
|
||||||
|
decision := r.router.Route(target, RouteRequest{})
|
||||||
|
switch decision.Kind {
|
||||||
|
case KindMain:
|
||||||
|
// A vision-capable main model sees the image directly.
|
||||||
|
return model.Content{Type: model.ContentTypeImage, ImageURL: imageURL}, nil
|
||||||
|
case KindVisionTool:
|
||||||
|
return r.invokeTool(ctx, decision.Tool, map[string]any{
|
||||||
|
"image": imageURL,
|
||||||
|
"prompt": "Describe this image in detail.",
|
||||||
|
}, "description")
|
||||||
|
case KindOCRTool:
|
||||||
|
return r.invokeTool(ctx, decision.Tool, map[string]any{
|
||||||
|
"image": imageURL,
|
||||||
|
}, "text")
|
||||||
|
default:
|
||||||
|
return model.Content{}, fmt.Errorf("image route: %w: %s", ErrUnsupportedSource, decision.Reason)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// invokeTool runs a registered vision/OCR tool and turns its text result into a
|
||||||
|
// text content part. field is the result map key ("description" or "text").
|
||||||
|
func (r *MessageResolver) invokeTool(ctx context.Context, toolName string, args map[string]any, field string) (model.Content, error) {
|
||||||
|
if r.invoke == nil {
|
||||||
|
return model.Content{}, fmt.Errorf("%s: %w", toolName, ErrNotConfigured)
|
||||||
|
}
|
||||||
|
result, err := r.invoke(ctx, toolName, args)
|
||||||
|
if err != nil {
|
||||||
|
return model.Content{}, fmt.Errorf("image tool %s: %w", toolName, err)
|
||||||
|
}
|
||||||
|
return model.Content{Type: model.ContentTypeText, Text: descriptionFragment(fieldText(result, field))}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// fieldText extracts a string field from a tool result map, tolerating the
|
||||||
|
// map[string]any shape returned by the vision/OCR tools.
|
||||||
|
func fieldText(result any, field string) string {
|
||||||
|
m, ok := result.(map[string]any)
|
||||||
|
if !ok {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
s, _ := m[field].(string)
|
||||||
|
return s
|
||||||
|
}
|
||||||
|
|
||||||
|
// descriptionFragment labels a tool-produced description so the model can tell it
|
||||||
|
// refers to the picture the user attached.
|
||||||
|
func descriptionFragment(desc string) string {
|
||||||
|
if desc == "" {
|
||||||
|
return "[图片内容] 无法识别该图片"
|
||||||
|
}
|
||||||
|
return "[图片内容] " + desc
|
||||||
|
}
|
||||||
@@ -0,0 +1,94 @@
|
|||||||
|
package vision
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"git.yeij.top/AskaEth/Cyrene_v2/internal/runtime/model"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestMessageResolver_FoldsImageToDescription(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
rt := NewRouter(WithVisionTool("vision"), WithOCRTool("ocr"))
|
||||||
|
invoked := ""
|
||||||
|
res := NewMessageResolver(rt, func() ModelInfo {
|
||||||
|
return ModelInfo{Name: "text-model", SupportsVision: false}
|
||||||
|
}, func(_ context.Context, name string, args map[string]any) (any, error) {
|
||||||
|
invoked = name
|
||||||
|
return map[string]any{"description": "一只橘色的猫"}, nil
|
||||||
|
})
|
||||||
|
|
||||||
|
msg, err := res.Resolve(context.Background(), model.ImageMessage(model.RoleUser, "看到了吗", "data:image/png;base64,AAAA"))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Resolve: %v", err)
|
||||||
|
}
|
||||||
|
if invoked != "vision" {
|
||||||
|
t.Fatalf("invoked tool = %q, want vision", invoked)
|
||||||
|
}
|
||||||
|
var text string
|
||||||
|
for _, p := range msg.Content {
|
||||||
|
if p.Type == model.ContentTypeText {
|
||||||
|
text += p.Text
|
||||||
|
}
|
||||||
|
if p.Type == model.ContentTypeImage {
|
||||||
|
t.Fatalf("image part still present after tool route: %+v", p)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if text == "" || text == "看到了吗" {
|
||||||
|
t.Fatalf("expected a description fragment, got %q", text)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestMessageResolver_KeepsImageForVisionModel(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
rt := NewRouter(WithVisionTool("vision"), WithOCRTool("ocr"))
|
||||||
|
res := NewMessageResolver(rt, func() ModelInfo {
|
||||||
|
return ModelInfo{Name: "vision-model", SupportsVision: true}
|
||||||
|
}, func(_ context.Context, _ string, _ map[string]any) (any, error) {
|
||||||
|
t.Fatalf("tool should not be invoked when the main model is sighted")
|
||||||
|
return nil, nil
|
||||||
|
})
|
||||||
|
|
||||||
|
msg, err := res.Resolve(context.Background(), model.ImageMessage(model.RoleUser, "x", "data:image/png;base64,AAAA"))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Resolve: %v", err)
|
||||||
|
}
|
||||||
|
if !messageHasImagePart(msg) {
|
||||||
|
t.Fatalf("expected the image part to be kept for a vision-capable main model")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestMessageResolver_OCRToolForText(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
rt := NewRouter(WithVisionTool("vision"), WithOCRTool("ocr"))
|
||||||
|
invoked := ""
|
||||||
|
res := NewMessageResolver(rt, func() ModelInfo {
|
||||||
|
return ModelInfo{Name: "text-model", SupportsVision: false}
|
||||||
|
}, func(_ context.Context, name string, _ map[string]any) (any, error) {
|
||||||
|
invoked = name
|
||||||
|
return map[string]any{"text": "HELLO"}, nil
|
||||||
|
})
|
||||||
|
|
||||||
|
// Force the text-primary hint by using a router route request path: this is
|
||||||
|
// exercised directly through Resolve which passes no hint, so the vision tool
|
||||||
|
// is preferred; assert at least a text part replaces the image.
|
||||||
|
msg, err := res.Resolve(context.Background(), model.ImageMessage(model.RoleUser, "x", "data:image/png;base64,AAAA"))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Resolve: %v", err)
|
||||||
|
}
|
||||||
|
if invoked != "vision" {
|
||||||
|
t.Fatalf("invoked tool = %q, want vision (router prefers vision when no text hint)", invoked)
|
||||||
|
}
|
||||||
|
if messageHasImagePart(msg) {
|
||||||
|
t.Fatalf("expected the image to be replaced by text")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func messageHasImagePart(m model.Message) bool {
|
||||||
|
for _, p := range m.Content {
|
||||||
|
if p.Type == model.ContentTypeImage {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
@@ -140,6 +140,20 @@ func (d Deps) pipeline() Pipeline {
|
|||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// AutomationRecord is the read-only summary of one automation or reminder,
|
||||||
|
// returned by Service.List for the admin panel. Kind is "cron", "event" or
|
||||||
|
// "reminder"; Cron carries the cron expression for the cron kind; LastRun and
|
||||||
|
// NextRun are zero times when the value is unknown (typically before Start).
|
||||||
|
type AutomationRecord struct {
|
||||||
|
ID string
|
||||||
|
Name string
|
||||||
|
Kind string
|
||||||
|
Cron string
|
||||||
|
Enabled bool
|
||||||
|
LastRun time.Time
|
||||||
|
NextRun time.Time
|
||||||
|
}
|
||||||
|
|
||||||
// Result is the structured outcome of an executed automation, published to
|
// Result is the structured outcome of an executed automation, published to
|
||||||
// SubjectResult. Upper layers subscribe to it to react or push notifications.
|
// SubjectResult. Upper layers subscribe to it to react or push notifications.
|
||||||
type Result struct {
|
type Result struct {
|
||||||
|
|||||||
@@ -22,6 +22,7 @@ type Service struct {
|
|||||||
|
|
||||||
mu sync.RWMutex
|
mu sync.RWMutex
|
||||||
cronRules map[string]ScheduleID // rule id -> schedule id
|
cronRules map[string]ScheduleID // rule id -> schedule id
|
||||||
|
cronMeta map[string]Rule // rule id -> cron rule (name/expression for listing)
|
||||||
}
|
}
|
||||||
|
|
||||||
// NewService builds an automation engine from cfg and deps. The bus in deps is
|
// NewService builds an automation engine from cfg and deps. The bus in deps is
|
||||||
@@ -50,6 +51,7 @@ func NewService(cfg Config, deps Deps) (*Service, error) {
|
|||||||
engine: engine,
|
engine: engine,
|
||||||
remind: remind,
|
remind: remind,
|
||||||
cronRules: make(map[string]ScheduleID),
|
cronRules: make(map[string]ScheduleID),
|
||||||
|
cronMeta: make(map[string]Rule),
|
||||||
}, nil
|
}, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -87,6 +89,7 @@ func (s *Service) AddRule(ctx context.Context, r Rule) error {
|
|||||||
}
|
}
|
||||||
s.mu.Lock()
|
s.mu.Lock()
|
||||||
s.cronRules[r.ID] = id
|
s.cronRules[r.ID] = id
|
||||||
|
s.cronMeta[r.ID] = r
|
||||||
s.mu.Unlock()
|
s.mu.Unlock()
|
||||||
return nil
|
return nil
|
||||||
default:
|
default:
|
||||||
@@ -100,6 +103,7 @@ func (s *Service) RemoveRule(ctx context.Context, id string) error {
|
|||||||
s.mu.Lock()
|
s.mu.Lock()
|
||||||
if schedID, ok := s.cronRules[id]; ok {
|
if schedID, ok := s.cronRules[id]; ok {
|
||||||
delete(s.cronRules, id)
|
delete(s.cronRules, id)
|
||||||
|
delete(s.cronMeta, id)
|
||||||
s.mu.Unlock()
|
s.mu.Unlock()
|
||||||
s.sched.RemoveSchedule(schedID)
|
s.sched.RemoveSchedule(schedID)
|
||||||
return nil
|
return nil
|
||||||
@@ -116,6 +120,51 @@ func (s *Service) AddReminder(ctx context.Context, r Reminder) error {
|
|||||||
// Reminders returns a sorted snapshot of registered reminders.
|
// Reminders returns a sorted snapshot of registered reminders.
|
||||||
func (s *Service) Reminders() []Reminder { return s.remind.List() }
|
func (s *Service) Reminders() []Reminder { return s.remind.List() }
|
||||||
|
|
||||||
|
// List returns a read-only snapshot of every registered automation and reminder
|
||||||
|
// for the admin panel, composing the reminder store, the event rules and the
|
||||||
|
// cron schedules. It never mutates state and is safe for concurrent use. Cron
|
||||||
|
// entries' LastRun/NextRun are zero until the scheduler has run.
|
||||||
|
func (s *Service) List() []AutomationRecord {
|
||||||
|
out := make([]AutomationRecord, 0)
|
||||||
|
|
||||||
|
for _, r := range s.remind.List() {
|
||||||
|
at := r.At
|
||||||
|
out = append(out, AutomationRecord{
|
||||||
|
ID: r.ID,
|
||||||
|
Name: r.Message,
|
||||||
|
Kind: "reminder",
|
||||||
|
Enabled: true,
|
||||||
|
NextRun: at,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, r := range s.engine.Rules() {
|
||||||
|
out = append(out, AutomationRecord{
|
||||||
|
ID: r.ID,
|
||||||
|
Name: r.Name,
|
||||||
|
Kind: string(r.Trigger),
|
||||||
|
Enabled: true,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
s.mu.RLock()
|
||||||
|
for id, rule := range s.cronMeta {
|
||||||
|
entry := s.sched.Entry(s.cronRules[id])
|
||||||
|
out = append(out, AutomationRecord{
|
||||||
|
ID: id,
|
||||||
|
Name: rule.Name,
|
||||||
|
Kind: string(rule.Trigger),
|
||||||
|
Cron: rule.Expression,
|
||||||
|
Enabled: true,
|
||||||
|
LastRun: entry.Prev,
|
||||||
|
NextRun: entry.Next,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
s.mu.RUnlock()
|
||||||
|
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
// CancelReminder delegates to the ReminderStore.
|
// CancelReminder delegates to the ReminderStore.
|
||||||
func (s *Service) CancelReminder(ctx context.Context, id string) error {
|
func (s *Service) CancelReminder(ctx context.Context, id string) error {
|
||||||
return s.remind.Cancel(ctx, id)
|
return s.remind.Cancel(ctx, id)
|
||||||
|
|||||||
@@ -5,12 +5,19 @@ import (
|
|||||||
"encoding/json"
|
"encoding/json"
|
||||||
"errors"
|
"errors"
|
||||||
"net/http"
|
"net/http"
|
||||||
|
"strconv"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"git.yeij.top/AskaEth/Cyrene_v2/internal/auth"
|
||||||
capreg "git.yeij.top/AskaEth/Cyrene_v2/internal/capability/registry"
|
capreg "git.yeij.top/AskaEth/Cyrene_v2/internal/capability/registry"
|
||||||
apperr "git.yeij.top/AskaEth/Cyrene_v2/internal/errors"
|
apperr "git.yeij.top/AskaEth/Cyrene_v2/internal/errors"
|
||||||
"git.yeij.top/AskaEth/Cyrene_v2/pkg/plugins/sdk"
|
"git.yeij.top/AskaEth/Cyrene_v2/pkg/plugins/sdk"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
// backthinkListLimit is the number of most-recent think records the admin
|
||||||
|
// panel asks for when no limit is specified.
|
||||||
|
const backthinkListLimit = 50
|
||||||
|
|
||||||
// PluginSummary is the wire shape of a capability/plugin in the admin list.
|
// PluginSummary is the wire shape of a capability/plugin in the admin list.
|
||||||
type PluginSummary struct {
|
type PluginSummary struct {
|
||||||
Name string `json:"name"`
|
Name string `json:"name"`
|
||||||
@@ -33,6 +40,55 @@ type policyRequest struct {
|
|||||||
AllowInStranger bool `json:"allowInStranger"`
|
AllowInStranger bool `json:"allowInStranger"`
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// AdminLlmProvider is the wire shape of one LLM provider in the admin config.
|
||||||
|
// The API key is deliberately absent: it is never returned to the client.
|
||||||
|
type AdminLlmProvider struct {
|
||||||
|
ID string `json:"id"`
|
||||||
|
Name string `json:"name"`
|
||||||
|
BaseURL string `json:"baseUrl"`
|
||||||
|
Model string `json:"model"`
|
||||||
|
Multimodal bool `json:"multimodal"`
|
||||||
|
Enabled bool `json:"enabled"`
|
||||||
|
Fallback string `json:"fallback,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// AdminLlmConfig is the wire shape of GET /api/v1/admin/llm.
|
||||||
|
type AdminLlmConfig struct {
|
||||||
|
Providers []AdminLlmProvider `json:"providers"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// AdminAutomationRecord is the wire shape of one automation/reminder in the
|
||||||
|
// admin list. Cron carries the cron expression for the cron kind; lastRun and
|
||||||
|
// nextRun are omitted when the value is unknown.
|
||||||
|
type AdminAutomationRecord struct {
|
||||||
|
ID string `json:"id"`
|
||||||
|
Name string `json:"name"`
|
||||||
|
Kind string `json:"kind"`
|
||||||
|
Cron string `json:"cron,omitempty"`
|
||||||
|
Enabled bool `json:"enabled"`
|
||||||
|
LastRun string `json:"lastRun,omitempty"`
|
||||||
|
NextRun string `json:"nextRun,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// AdminUserRecord is the wire shape of one account/API key in the admin list.
|
||||||
|
type AdminUserRecord struct {
|
||||||
|
ID string `json:"id"`
|
||||||
|
Username string `json:"username"`
|
||||||
|
Role string `json:"role"`
|
||||||
|
CreatedAt string `json:"createdAt,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// AdminBackthinkItem is the wire shape of one background-think record in the
|
||||||
|
// admin monitor list.
|
||||||
|
type AdminBackthinkItem struct {
|
||||||
|
ID string `json:"id"`
|
||||||
|
Trigger string `json:"trigger"`
|
||||||
|
Conclusion string `json:"conclusion,omitempty"`
|
||||||
|
ShouldNotify bool `json:"shouldNotify"`
|
||||||
|
Pushed bool `json:"pushed"`
|
||||||
|
At int64 `json:"at"`
|
||||||
|
}
|
||||||
|
|
||||||
// adminListPersonas lists persona summaries (admin read).
|
// adminListPersonas lists persona summaries (admin read).
|
||||||
func (s *Server) adminListPersonas(w http.ResponseWriter, r *http.Request) {
|
func (s *Server) adminListPersonas(w http.ResponseWriter, r *http.Request) {
|
||||||
writeJSON(w, http.StatusOK, s.collectPersonas())
|
writeJSON(w, http.StatusOK, s.collectPersonas())
|
||||||
@@ -97,6 +153,104 @@ func (s *Server) adminListIoT(w http.ResponseWriter, r *http.Request) {
|
|||||||
writeJSON(w, http.StatusOK, out)
|
writeJSON(w, http.StatusOK, out)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// adminGetLlm returns the LLM provider/model configuration read from the live
|
||||||
|
// config. API keys are never returned (the wire shape carries no key field), so
|
||||||
|
// the handler leaks nothing sensitive.
|
||||||
|
func (s *Server) adminGetLlm(w http.ResponseWriter, r *http.Request) {
|
||||||
|
cfg := s.cfg.LLM
|
||||||
|
writeJSON(w, http.StatusOK, AdminLlmConfig{Providers: []AdminLlmProvider{
|
||||||
|
{
|
||||||
|
ID: "openai",
|
||||||
|
Name: "OpenAI",
|
||||||
|
BaseURL: cfg.BaseURL,
|
||||||
|
Model: cfg.Model,
|
||||||
|
Multimodal: cfg.VisionModel != "",
|
||||||
|
Enabled: cfg.Provider != "anthropic",
|
||||||
|
Fallback: cfg.Fallback,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
ID: "anthropic",
|
||||||
|
Name: "Anthropic",
|
||||||
|
BaseURL: cfg.AnthropicBaseURL,
|
||||||
|
Model: cfg.AnthropicModel,
|
||||||
|
Multimodal: false,
|
||||||
|
Enabled: cfg.Provider == "anthropic",
|
||||||
|
Fallback: cfg.AnthropicFallback,
|
||||||
|
},
|
||||||
|
}})
|
||||||
|
}
|
||||||
|
|
||||||
|
// adminListAutomations lists automations and reminders (admin read).
|
||||||
|
func (s *Server) adminListAutomations(w http.ResponseWriter, r *http.Request) {
|
||||||
|
out := make([]AdminAutomationRecord, 0)
|
||||||
|
if s.automations != nil {
|
||||||
|
for _, a := range s.automations.List() {
|
||||||
|
rec := AdminAutomationRecord{
|
||||||
|
ID: a.ID,
|
||||||
|
Name: a.Name,
|
||||||
|
Kind: a.Kind,
|
||||||
|
Cron: a.Cron,
|
||||||
|
Enabled: a.Enabled,
|
||||||
|
}
|
||||||
|
if !a.LastRun.IsZero() {
|
||||||
|
rec.LastRun = a.LastRun.Format(time.RFC3339)
|
||||||
|
}
|
||||||
|
if !a.NextRun.IsZero() {
|
||||||
|
rec.NextRun = a.NextRun.Format(time.RFC3339)
|
||||||
|
}
|
||||||
|
out = append(out, rec)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
writeJSON(w, http.StatusOK, out)
|
||||||
|
}
|
||||||
|
|
||||||
|
// adminListUsers lists accounts and API keys (admin read). Without a wired
|
||||||
|
// UserStore it reports the single configured admin account rather than
|
||||||
|
// fabricating a column of virtual users.
|
||||||
|
func (s *Server) adminListUsers(w http.ResponseWriter, r *http.Request) {
|
||||||
|
out := []AdminUserRecord{{
|
||||||
|
ID: "admin",
|
||||||
|
Username: s.cfg.Auth.AdminUser,
|
||||||
|
Role: string(auth.RoleOwner),
|
||||||
|
}}
|
||||||
|
if s.users != nil {
|
||||||
|
for _, u := range s.users.List() {
|
||||||
|
rec := AdminUserRecord{ID: u.ID, Username: u.Username, Role: string(u.Role)}
|
||||||
|
if u.CreatedAt != nil {
|
||||||
|
rec.CreatedAt = u.CreatedAt.Format(time.RFC3339)
|
||||||
|
}
|
||||||
|
out = append(out, rec)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
writeJSON(w, http.StatusOK, out)
|
||||||
|
}
|
||||||
|
|
||||||
|
// adminListBackthink lists the most recent background-think records (admin read).
|
||||||
|
func (s *Server) adminListBackthink(w http.ResponseWriter, r *http.Request) {
|
||||||
|
out := make([]AdminBackthinkItem, 0)
|
||||||
|
if s.backthink == nil {
|
||||||
|
writeJSON(w, http.StatusOK, out)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
records, err := s.backthink.List(r.Context(), backthinkListLimit)
|
||||||
|
if err != nil {
|
||||||
|
s.log.Error("list backthink", "err", err)
|
||||||
|
writeError(w, apperr.Wrap(apperr.ErrInternal, "backthink_failed", "failed to list backthink records"))
|
||||||
|
return
|
||||||
|
}
|
||||||
|
for _, rec := range records {
|
||||||
|
out = append(out, AdminBackthinkItem{
|
||||||
|
ID: strconv.FormatInt(rec.ID, 10),
|
||||||
|
Trigger: rec.Trigger,
|
||||||
|
Conclusion: rec.Conclusion,
|
||||||
|
ShouldNotify: rec.ShouldPush,
|
||||||
|
Pushed: rec.PushResult.Pushed,
|
||||||
|
At: rec.At.UnixMilli(),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
writeJSON(w, http.StatusOK, out)
|
||||||
|
}
|
||||||
|
|
||||||
// adminSetPolicy upserts the reachability markers for a capability. It is the
|
// adminSetPolicy upserts the reachability markers for a capability. It is the
|
||||||
// audited admin write example (owner-only, enforced by the route registration).
|
// audited admin write example (owner-only, enforced by the route registration).
|
||||||
func (s *Server) adminSetPolicy(w http.ResponseWriter, r *http.Request) {
|
func (s *Server) adminSetPolicy(w http.ResponseWriter, r *http.Request) {
|
||||||
|
|||||||
@@ -25,6 +25,7 @@ import (
|
|||||||
"git.yeij.top/AskaEth/Cyrene_v2/internal/runtime/loop"
|
"git.yeij.top/AskaEth/Cyrene_v2/internal/runtime/loop"
|
||||||
"git.yeij.top/AskaEth/Cyrene_v2/internal/runtime/memory"
|
"git.yeij.top/AskaEth/Cyrene_v2/internal/runtime/memory"
|
||||||
"git.yeij.top/AskaEth/Cyrene_v2/internal/runtime/model"
|
"git.yeij.top/AskaEth/Cyrene_v2/internal/runtime/model"
|
||||||
|
"git.yeij.top/AskaEth/Cyrene_v2/internal/runtime/multimodal/voice"
|
||||||
"git.yeij.top/AskaEth/Cyrene_v2/internal/runtime/orchestration"
|
"git.yeij.top/AskaEth/Cyrene_v2/internal/runtime/orchestration"
|
||||||
"git.yeij.top/AskaEth/Cyrene_v2/internal/runtime/orchestration/comm"
|
"git.yeij.top/AskaEth/Cyrene_v2/internal/runtime/orchestration/comm"
|
||||||
"git.yeij.top/AskaEth/Cyrene_v2/internal/runtime/task"
|
"git.yeij.top/AskaEth/Cyrene_v2/internal/runtime/task"
|
||||||
@@ -918,10 +919,10 @@ func TestHubRoute(t *testing.T) {
|
|||||||
}
|
}
|
||||||
hb.taskOutputs = &stubTaskOutputs{ev: task.OutputEvent{ID: "task-1", Status: task.StatusDone, Output: task.Output{Stdout: "out"}}}
|
hb.taskOutputs = &stubTaskOutputs{ev: task.OutputEvent{ID: "task-1", Status: task.StatusDone, Output: task.Output{Stdout: "out"}}}
|
||||||
|
|
||||||
// voice_input → publishes to bus (no frame enqueued, not an unknown error).
|
// voice_input with no voice service → clear error, never a crash or a drop.
|
||||||
c.route(wsInbound{Type: "voice_input", Data: json.RawMessage(`{"audio":"x"}`)})
|
c.route(wsInbound{Type: "voice_input", Data: json.RawMessage(`{"audio":"x"}`)})
|
||||||
if err := connHasNoFrame(c); err != nil {
|
if ev := readFakeFrame(t, c); ev.Type != "error" {
|
||||||
t.Fatalf("voice_input should not enqueue a frame directly: %v", err)
|
t.Fatalf("no-voice voice_input frame type = %q, want error", ev.Type)
|
||||||
}
|
}
|
||||||
// unknown → error (enqueueJSON).
|
// unknown → error (enqueueJSON).
|
||||||
c.route(wsInbound{Type: "nope"})
|
c.route(wsInbound{Type: "nope"})
|
||||||
@@ -935,6 +936,43 @@ func TestHubRoute(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// stubVoice is a deterministic VoiceService for voice route tests.
|
||||||
|
type stubVoice struct {
|
||||||
|
sttText string
|
||||||
|
sttErr error
|
||||||
|
tts voice.Audio
|
||||||
|
ttsErr error
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *stubVoice) STT(_ context.Context, _ voice.Audio) (string, error) { return s.sttText, s.sttErr }
|
||||||
|
func (s *stubVoice) TTS(_ context.Context, _ string) (voice.Audio, error) { return s.tts, s.ttsErr }
|
||||||
|
|
||||||
|
func TestHubVoiceInputTurn(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
hb := newTestHub(&stubRunner{res: &loop.Result{Text: "你好", Steps: 1, StopReason: "stop"}}, nil)
|
||||||
|
hb.voice = &stubVoice{sttText: "你好", tts: voice.Audio{Format: "wav", Data: []byte("audio-bytes")}}
|
||||||
|
c := newFakeConn(hb, 32)
|
||||||
|
hb.register(c)
|
||||||
|
|
||||||
|
// "aGVsbG8=" decodes to "hello" — a valid base64 audio payload.
|
||||||
|
c.route(wsInbound{Type: "voice_input", Data: json.RawMessage(`{"audio":"aGVsbG8=","format":"wav"}`)})
|
||||||
|
// runVoiceTurn runs in a goroutine; read with a deadline.
|
||||||
|
ev := readFakeFrame(t, c)
|
||||||
|
if ev.Type != "voice_output" {
|
||||||
|
t.Fatalf("voice frame type = %q, want voice_output (frame=%+v)", ev.Type, ev)
|
||||||
|
}
|
||||||
|
r, ok := ev.Data.(map[string]any)
|
||||||
|
if !ok || r["text"] != "你好" {
|
||||||
|
t.Fatalf("voice transcript = %+v, want 你好", ev.Data)
|
||||||
|
}
|
||||||
|
if r["reply"] != "你好" {
|
||||||
|
t.Fatalf("voice reply = %+v, want 你好", ev.Data)
|
||||||
|
}
|
||||||
|
if r["audio"] == "" || r["audio"] == nil {
|
||||||
|
t.Fatalf("voice audio missing from reply: %+v", ev.Data)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestHubBridgeDeviceUpdate(t *testing.T) {
|
func TestHubBridgeDeviceUpdate(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
b := bus.New()
|
b := bus.New()
|
||||||
|
|||||||
@@ -57,10 +57,13 @@ type loginResponse struct {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// chatRequest is the POST /api/v1/chat body. When Stream is true the gateway
|
// chatRequest is the POST /api/v1/chat body. When Stream is true the gateway
|
||||||
// triggers streaming over the WebSocket channel and returns immediately.
|
// triggers streaming over the WebSocket channel and returns immediately. Image is
|
||||||
|
// an optional base64 data: URI or http(s) URL attached for the vision path; the
|
||||||
|
// loop routes it through the vision/OCR tool or the main model (design/06 §0).
|
||||||
type chatRequest struct {
|
type chatRequest struct {
|
||||||
SessionID string `json:"sessionId"`
|
SessionID string `json:"sessionId"`
|
||||||
Text string `json:"text"`
|
Text string `json:"text"`
|
||||||
|
Image string `json:"image,omitempty"`
|
||||||
Stream bool `json:"stream"`
|
Stream bool `json:"stream"`
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -108,12 +111,12 @@ func (s *Server) handleChat(w http.ResponseWriter, r *http.Request) {
|
|||||||
writeError(w, apperr.Wrap(apperr.ErrBadRequest, "bad_request", "invalid request body"))
|
writeError(w, apperr.Wrap(apperr.ErrBadRequest, "bad_request", "invalid request body"))
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
if strings.TrimSpace(req.Text) == "" {
|
if strings.TrimSpace(req.Text) == "" && strings.TrimSpace(req.Image) == "" {
|
||||||
writeError(w, apperr.Wrap(apperr.ErrBadRequest, "bad_request", "text is required"))
|
writeError(w, apperr.Wrap(apperr.ErrBadRequest, "bad_request", "text or image is required"))
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
if req.Stream {
|
if req.Stream {
|
||||||
s.publishChatStream(r.Context(), req.SessionID, req.Text)
|
s.publishChatStream(r.Context(), req.SessionID, req.Text, req.Image)
|
||||||
writeJSON(w, http.StatusOK, map[string]any{"stream": true})
|
writeJSON(w, http.StatusOK, map[string]any{"stream": true})
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
@@ -124,7 +127,7 @@ func (s *Server) handleChat(w http.ResponseWriter, r *http.Request) {
|
|||||||
if req.SessionID == "" {
|
if req.SessionID == "" {
|
||||||
req.SessionID = "web"
|
req.SessionID = "web"
|
||||||
}
|
}
|
||||||
res, err := s.runner.Run(r.Context(), req.SessionID, []model.Message{model.TextMessage(model.RoleUser, req.Text)})
|
res, err := s.runner.Run(r.Context(), req.SessionID, userChatMessages(req.Text, req.Image))
|
||||||
if err != nil {
|
if err != nil {
|
||||||
s.log.Error("chat run", "err", err, "session", req.SessionID)
|
s.log.Error("chat run", "err", err, "session", req.SessionID)
|
||||||
writeError(w, apperr.Wrap(apperr.ErrInternal, "chat_failed", "chat failed"))
|
writeError(w, apperr.Wrap(apperr.ErrInternal, "chat_failed", "chat failed"))
|
||||||
@@ -185,6 +188,16 @@ func firstMode(modes []string) string {
|
|||||||
return modes[0]
|
return modes[0]
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// userChatMessages builds the user turn for a chat / stream request. When image is
|
||||||
|
// present the turn becomes an image message (with optional text); otherwise a
|
||||||
|
// plain text message. An empty text with an image yields an image-only turn.
|
||||||
|
func userChatMessages(text, image string) []model.Message {
|
||||||
|
if strings.TrimSpace(image) != "" {
|
||||||
|
return []model.Message{model.ImageMessage(model.RoleUser, text, image)}
|
||||||
|
}
|
||||||
|
return []model.Message{model.TextMessage(model.RoleUser, text)}
|
||||||
|
}
|
||||||
|
|
||||||
// summaryOfSession maps a session.Session to its wire shape.
|
// summaryOfSession maps a session.Session to its wire shape.
|
||||||
func summaryOfSession(sess session.Session) SessionSummary {
|
func summaryOfSession(sess session.Session) SessionSummary {
|
||||||
return SessionSummary{
|
return SessionSummary{
|
||||||
|
|||||||
@@ -40,6 +40,8 @@ import (
|
|||||||
"git.yeij.top/AskaEth/Cyrene_v2/internal/runtime/model"
|
"git.yeij.top/AskaEth/Cyrene_v2/internal/runtime/model"
|
||||||
"git.yeij.top/AskaEth/Cyrene_v2/internal/runtime/orchestration"
|
"git.yeij.top/AskaEth/Cyrene_v2/internal/runtime/orchestration"
|
||||||
"git.yeij.top/AskaEth/Cyrene_v2/internal/runtime/task"
|
"git.yeij.top/AskaEth/Cyrene_v2/internal/runtime/task"
|
||||||
|
"git.yeij.top/AskaEth/Cyrene_v2/internal/services/automations"
|
||||||
|
"git.yeij.top/AskaEth/Cyrene_v2/internal/services/backthink"
|
||||||
)
|
)
|
||||||
|
|
||||||
// source is the log provenance tag for every record this package emits.
|
// source is the log provenance tag for every record this package emits.
|
||||||
@@ -110,6 +112,26 @@ type TaskOutputStore interface {
|
|||||||
TaskOutput(ctx context.Context, id string) (task.OutputEvent, error)
|
TaskOutput(ctx context.Context, id string) (task.OutputEvent, error)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// AutomationStore enumerates the automations and reminders for the admin list.
|
||||||
|
// *automations.Service is the production implementation. The gateway maps each
|
||||||
|
// record to its wire shape.
|
||||||
|
type AutomationStore interface {
|
||||||
|
List() []automations.AutomationRecord
|
||||||
|
}
|
||||||
|
|
||||||
|
// UserStore enumerates the accounts and API keys visible to the admin panel.
|
||||||
|
// The production implementation reads the configured admin account plus the
|
||||||
|
// auth.APIKey table. The gateway maps each summary to its wire shape.
|
||||||
|
type UserStore interface {
|
||||||
|
List() []auth.UserSummary
|
||||||
|
}
|
||||||
|
|
||||||
|
// BackthinkStore enumerates the persisted background-think records for the
|
||||||
|
// M9/M10 monitor panel. *backthink.Backthink is the production implementation.
|
||||||
|
type BackthinkStore interface {
|
||||||
|
List(ctx context.Context, limit int) ([]backthink.Record, error)
|
||||||
|
}
|
||||||
|
|
||||||
// Deps carries the gateway's injected dependencies. Every field is nil-tolerant
|
// Deps carries the gateway's injected dependencies. Every field is nil-tolerant
|
||||||
// except Auth, which is required (authentication backs login and every RBAC
|
// except Auth, which is required (authentication backs login and every RBAC
|
||||||
// decision). Registry/Loop are satisfied by the concrete *capability.Registry and
|
// decision). Registry/Loop are satisfied by the concrete *capability.Registry and
|
||||||
@@ -141,6 +163,13 @@ type Deps struct {
|
|||||||
// Tasks queries task output for the task_output_query WS route.
|
// Tasks queries task output for the task_output_query WS route.
|
||||||
// Optional (task_output_query returns a clear error when unset).
|
// Optional (task_output_query returns a clear error when unset).
|
||||||
Tasks TaskOutputStore
|
Tasks TaskOutputStore
|
||||||
|
// Automations lists automations and reminders. Optional.
|
||||||
|
Automations AutomationStore
|
||||||
|
// Users lists accounts and API keys. Optional (falls back to the configured
|
||||||
|
// single admin account).
|
||||||
|
Users UserStore
|
||||||
|
// Backthink lists background-think records. Optional.
|
||||||
|
Backthink BackthinkStore
|
||||||
// Streaming marks the injected Loop as streaming-capable (its OnStream is
|
// Streaming marks the injected Loop as streaming-capable (its OnStream is
|
||||||
// wired to publish ws.stream_chunk). When set the hub skips its own single
|
// wired to publish ws.stream_chunk). When set the hub skips its own single
|
||||||
// final stream_chunk emission, since deltas already stream through OnStream.
|
// final stream_chunk emission, since deltas already stream through OnStream.
|
||||||
@@ -165,6 +194,10 @@ type Server struct {
|
|||||||
iot IoTStore
|
iot IoTStore
|
||||||
auditor SubagentAuditor
|
auditor SubagentAuditor
|
||||||
taskOutputs TaskOutputStore
|
taskOutputs TaskOutputStore
|
||||||
|
automations AutomationStore
|
||||||
|
users UserStore
|
||||||
|
backthink BackthinkStore
|
||||||
|
voice VoiceService
|
||||||
streaming bool
|
streaming bool
|
||||||
frontend fs.FS
|
frontend fs.FS
|
||||||
log *slog.Logger
|
log *slog.Logger
|
||||||
@@ -193,6 +226,9 @@ func New(cfg *config.Config, deps Deps) (*Server, error) {
|
|||||||
iot: deps.IoT,
|
iot: deps.IoT,
|
||||||
auditor: deps.Scheduler,
|
auditor: deps.Scheduler,
|
||||||
taskOutputs: deps.Tasks,
|
taskOutputs: deps.Tasks,
|
||||||
|
automations: deps.Automations,
|
||||||
|
users: deps.Users,
|
||||||
|
backthink: deps.Backthink,
|
||||||
streaming: deps.Streaming,
|
streaming: deps.Streaming,
|
||||||
frontend: deps.Frontend,
|
frontend: deps.Frontend,
|
||||||
log: sourcedLog(deps.Log),
|
log: sourcedLog(deps.Log),
|
||||||
@@ -229,6 +265,10 @@ func (s *Server) Handler() http.Handler {
|
|||||||
mux.Handle("GET /api/v1/admin/plugins", adminAuth(http.HandlerFunc(s.adminListPlugins)))
|
mux.Handle("GET /api/v1/admin/plugins", adminAuth(http.HandlerFunc(s.adminListPlugins)))
|
||||||
mux.Handle("GET /api/v1/admin/channels", adminAuth(http.HandlerFunc(s.adminListChannels)))
|
mux.Handle("GET /api/v1/admin/channels", adminAuth(http.HandlerFunc(s.adminListChannels)))
|
||||||
mux.Handle("GET /api/v1/admin/iot", adminAuth(http.HandlerFunc(s.adminListIoT)))
|
mux.Handle("GET /api/v1/admin/iot", adminAuth(http.HandlerFunc(s.adminListIoT)))
|
||||||
|
mux.Handle("GET /api/v1/admin/llm", adminAuth(http.HandlerFunc(s.adminGetLlm)))
|
||||||
|
mux.Handle("GET /api/v1/admin/automations", adminAuth(http.HandlerFunc(s.adminListAutomations)))
|
||||||
|
mux.Handle("GET /api/v1/admin/users", adminAuth(http.HandlerFunc(s.adminListUsers)))
|
||||||
|
mux.Handle("GET /api/v1/admin/backthink", adminAuth(http.HandlerFunc(s.adminListBackthink)))
|
||||||
mux.Handle("POST /api/v1/admin/plugins/policy", s.auth.Authorize(auth.RoleOwner)(http.HandlerFunc(s.adminSetPolicy)))
|
mux.Handle("POST /api/v1/admin/plugins/policy", s.auth.Authorize(auth.RoleOwner)(http.HandlerFunc(s.adminSetPolicy)))
|
||||||
|
|
||||||
var h http.Handler = mux
|
var h http.Handler = mux
|
||||||
|
|||||||
@@ -0,0 +1,163 @@
|
|||||||
|
package gateway
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/base64"
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"git.yeij.top/AskaEth/Cyrene_v2/internal/runtime/model"
|
||||||
|
"git.yeij.top/AskaEth/Cyrene_v2/internal/runtime/multimodal/voice"
|
||||||
|
)
|
||||||
|
|
||||||
|
// VoiceService runs speech-to-text and text-to-speech for the voice_input WS
|
||||||
|
// route. *voice.Chain (the degrading provider chain) and *voice.Service (the
|
||||||
|
// background worker pool) both satisfy it. It is nil-tolerant: the voice route
|
||||||
|
// replies with a clear error when no service is injected.
|
||||||
|
type VoiceService interface {
|
||||||
|
// STT transcribes audio to text.
|
||||||
|
STT(ctx context.Context, audio voice.Audio) (string, error)
|
||||||
|
// TTS synthesizes audio from text.
|
||||||
|
TTS(ctx context.Context, text string) (voice.Audio, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
// voiceInputRequest is the decoded payload of a voice_input frame. Audio is a
|
||||||
|
// base64 (or data: URI) audio blob; Format is its encoding ("wav"/"mp3"/"ogg");
|
||||||
|
// Text overrides STT when present (a client that already transcribed can skip
|
||||||
|
// recognition); SessionID defaults to "web".
|
||||||
|
type voiceInputRequest struct {
|
||||||
|
Audio string `json:"audio"`
|
||||||
|
Format string `json:"format"`
|
||||||
|
Text string `json:"text"`
|
||||||
|
SessionID string `json:"sessionId"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// SetVoice injects the voice STT/TTS service used by the voice_input WS route.
|
||||||
|
// It is the gateway's exposed injection point so the composition root can wire a
|
||||||
|
// *voice.Chain or *voice.Service without touching Deps. A nil value disables the
|
||||||
|
// route (it replies with a clear error).
|
||||||
|
func (s *Server) SetVoice(v VoiceService) {
|
||||||
|
s.voice = v
|
||||||
|
s.hub.voice = v
|
||||||
|
}
|
||||||
|
|
||||||
|
// handleVoiceInput routes one voice_input frame. Without an injected voice
|
||||||
|
// service it replies with a clear error (never a crash); otherwise it decodes the
|
||||||
|
// request and runs the STT→chat→TTS turn asynchronously so the read pump is never
|
||||||
|
// blocked by provider latency.
|
||||||
|
func (c *wsConn) handleVoiceInput(in wsInbound) {
|
||||||
|
if c.h.voice == nil {
|
||||||
|
c.enqueueJSON(wsEvent{Type: "error", Data: map[string]any{"message": "voice unavailable"}})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
var req voiceInputRequest
|
||||||
|
_ = json.Unmarshal(in.Data, &req)
|
||||||
|
if strings.TrimSpace(req.Audio) == "" && strings.TrimSpace(req.Text) == "" {
|
||||||
|
c.enqueueJSON(wsEvent{Type: "error", Data: map[string]any{"message": "voice audio or text required"}})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
go c.h.runVoiceTurn(c, req)
|
||||||
|
}
|
||||||
|
|
||||||
|
// runVoiceTurn drives one voice turn: STT (or a supplied transcript) → agent chat
|
||||||
|
// → TTS, then replies to the requesting connection with a voice_output frame
|
||||||
|
// carrying the base64 audio plus the recognized transcript and assistant reply.
|
||||||
|
func (h *hub) runVoiceTurn(c *wsConn, req voiceInputRequest) {
|
||||||
|
ctx := context.Background()
|
||||||
|
if req.SessionID == "" {
|
||||||
|
req.SessionID = "web"
|
||||||
|
}
|
||||||
|
transcript := strings.TrimSpace(req.Text)
|
||||||
|
if transcript == "" {
|
||||||
|
if strings.TrimSpace(req.Audio) == "" {
|
||||||
|
h.voiceError(c, "voice audio or text required")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
audio, err := decodeVoiceAudio(req.Audio, req.Format)
|
||||||
|
if err != nil {
|
||||||
|
h.log.Warn("voice audio decode", "err", err)
|
||||||
|
h.voiceError(c, "voice audio could not be decoded")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
stt, err := h.voice.STT(ctx, audio)
|
||||||
|
if err != nil {
|
||||||
|
h.log.Warn("voice stt", "err", err)
|
||||||
|
h.voiceError(c, "voice transcription failed")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
transcript = strings.TrimSpace(stt)
|
||||||
|
}
|
||||||
|
if transcript == "" {
|
||||||
|
h.voiceError(c, "no speech recognized")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if h.runner == nil {
|
||||||
|
h.voiceError(c, "chat service is not configured")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
res, err := h.runner.Run(ctx, req.SessionID, []model.Message{model.TextMessage(model.RoleUser, transcript)})
|
||||||
|
if err != nil {
|
||||||
|
h.log.Warn("voice chat run", "err", err, "session", req.SessionID)
|
||||||
|
h.voiceError(c, "voice chat failed")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
reply := res.Text
|
||||||
|
if strings.TrimSpace(reply) == "" {
|
||||||
|
reply = res.StopReason
|
||||||
|
}
|
||||||
|
audi, err := h.voice.TTS(ctx, reply)
|
||||||
|
if err != nil {
|
||||||
|
h.log.Warn("voice tts", "err", err)
|
||||||
|
h.voiceError(c, "voice synthesis failed")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
c.enqueueJSON(wsEvent{Type: wsVoiceOutput, Data: map[string]any{
|
||||||
|
"audio": base64.StdEncoding.EncodeToString(audi.Data),
|
||||||
|
"format": audi.Format,
|
||||||
|
"text": transcript,
|
||||||
|
"reply": reply,
|
||||||
|
}})
|
||||||
|
}
|
||||||
|
|
||||||
|
// voiceError replies with a typed error frame for a failed voice turn.
|
||||||
|
func (h *hub) voiceError(c *wsConn, message string) {
|
||||||
|
c.enqueueJSON(wsEvent{Type: "error", Data: map[string]any{"message": message}})
|
||||||
|
}
|
||||||
|
|
||||||
|
// decodeVoiceAudio turns a base64 (or data:) audio payload into voice.Audio. The
|
||||||
|
// format is taken from req when set, guessed from a data: URI mime otherwise, and
|
||||||
|
// defaulted to "wav".
|
||||||
|
func decodeVoiceAudio(src, format string) (voice.Audio, error) {
|
||||||
|
payload := src
|
||||||
|
if strings.HasPrefix(src, "data:") {
|
||||||
|
if comma := strings.IndexByte(src, ','); comma >= 0 {
|
||||||
|
if format == "" {
|
||||||
|
format = formatFromMime(src[:comma])
|
||||||
|
}
|
||||||
|
payload = src[comma+1:]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(payload) == "" {
|
||||||
|
return voice.Audio{}, fmt.Errorf("voice: empty audio")
|
||||||
|
}
|
||||||
|
data, err := base64.StdEncoding.DecodeString(payload)
|
||||||
|
if err != nil {
|
||||||
|
return voice.Audio{}, fmt.Errorf("voice: decode audio: %w", err)
|
||||||
|
}
|
||||||
|
if format == "" {
|
||||||
|
format = "wav"
|
||||||
|
}
|
||||||
|
return voice.Audio{Format: format, Data: data}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// formatFromMime guesses an audio format from a data: URI mime segment.
|
||||||
|
func formatFromMime(meta string) string {
|
||||||
|
lower := strings.ToLower(meta)
|
||||||
|
for _, f := range []string{"wav", "mp3", "ogg", "webm", "mpeg"} {
|
||||||
|
if strings.Contains(lower, f) {
|
||||||
|
return f
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return "wav"
|
||||||
|
}
|
||||||
@@ -28,6 +28,7 @@ const (
|
|||||||
wsChatSubject = "ws.chat"
|
wsChatSubject = "ws.chat"
|
||||||
wsStreamSubject = "ws.stream"
|
wsStreamSubject = "ws.stream"
|
||||||
wsVoiceInput = "ws.voice_input"
|
wsVoiceInput = "ws.voice_input"
|
||||||
|
wsVoiceOutput = "voice_output"
|
||||||
wsReadLimit = 4096
|
wsReadLimit = 4096
|
||||||
wsWriteWait = 10 * time.Second
|
wsWriteWait = 10 * time.Second
|
||||||
wsPongWait = 60 * time.Second
|
wsPongWait = 60 * time.Second
|
||||||
@@ -72,6 +73,7 @@ type hub struct {
|
|||||||
runner Chatter
|
runner Chatter
|
||||||
auditor SubagentAuditor
|
auditor SubagentAuditor
|
||||||
taskOutputs TaskOutputStore
|
taskOutputs TaskOutputStore
|
||||||
|
voice VoiceService
|
||||||
streaming bool
|
streaming bool
|
||||||
log *slog.Logger
|
log *slog.Logger
|
||||||
upgrader websocket.Upgrader
|
upgrader websocket.Upgrader
|
||||||
@@ -90,6 +92,7 @@ func newHub(s *Server) *hub {
|
|||||||
runner: s.runner,
|
runner: s.runner,
|
||||||
auditor: s.auditor,
|
auditor: s.auditor,
|
||||||
taskOutputs: s.taskOutputs,
|
taskOutputs: s.taskOutputs,
|
||||||
|
voice: s.voice,
|
||||||
streaming: s.streaming,
|
streaming: s.streaming,
|
||||||
log: s.log,
|
log: s.log,
|
||||||
origins: origins,
|
origins: origins,
|
||||||
@@ -213,14 +216,14 @@ func (h *hub) streamChat(ctx context.Context, payload any) {
|
|||||||
_ = json.Unmarshal(b, &req)
|
_ = json.Unmarshal(b, &req)
|
||||||
}
|
}
|
||||||
turnID := newTurnID()
|
turnID := newTurnID()
|
||||||
if h.runner == nil || strings.TrimSpace(req.Text) == "" {
|
if h.runner == nil || (strings.TrimSpace(req.Text) == "" && strings.TrimSpace(req.Image) == "") {
|
||||||
h.broadcast(wsEvent{Type: "stream_end", TurnID: turnID, Seq: int(h.streamSeq.Add(1)), Data: map[string]any{"stopReason": "idle"}})
|
h.broadcast(wsEvent{Type: "stream_end", TurnID: turnID, Seq: int(h.streamSeq.Add(1)), Data: map[string]any{"stopReason": "idle"}})
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
if req.SessionID == "" {
|
if req.SessionID == "" {
|
||||||
req.SessionID = "web"
|
req.SessionID = "web"
|
||||||
}
|
}
|
||||||
res, err := h.runner.Run(ctx, req.SessionID, []model.Message{model.TextMessage(model.RoleUser, req.Text)})
|
res, err := h.runner.Run(ctx, req.SessionID, userChatMessages(req.Text, req.Image))
|
||||||
if err != nil {
|
if err != nil {
|
||||||
h.log.Error("ws chat run", "err", err, "session", req.SessionID)
|
h.log.Error("ws chat run", "err", err, "session", req.SessionID)
|
||||||
h.broadcast(wsEvent{Type: "stream_end", TurnID: turnID, Seq: int(h.streamSeq.Add(1)), Data: map[string]any{"stopReason": "error", "error": err.Error()}})
|
h.broadcast(wsEvent{Type: "stream_end", TurnID: turnID, Seq: int(h.streamSeq.Add(1)), Data: map[string]any{"stopReason": "error", "error": err.Error()}})
|
||||||
@@ -256,6 +259,7 @@ func (h *hub) broadcastStream(payload any) {
|
|||||||
type chatStreamRequest struct {
|
type chatStreamRequest struct {
|
||||||
SessionID string `json:"sessionId"`
|
SessionID string `json:"sessionId"`
|
||||||
Text string `json:"text"`
|
Text string `json:"text"`
|
||||||
|
Image string `json:"image,omitempty"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// handleWS upgrades /ws/chat after authenticating the JWT from the query token or
|
// handleWS upgrades /ws/chat after authenticating the JWT from the query token or
|
||||||
@@ -382,11 +386,11 @@ func (c *wsConn) enqueueJSON(ev wsEvent) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// publishChatStream publishes a ws.chat streaming trigger (REST stream mode).
|
// publishChatStream publishes a ws.chat streaming trigger (REST stream mode).
|
||||||
func (s *Server) publishChatStream(ctx context.Context, sessionID, text string) {
|
func (s *Server) publishChatStream(ctx context.Context, sessionID, text, image string) {
|
||||||
if s.bus == nil {
|
if s.bus == nil {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
_ = s.bus.Publish(ctx, wsChatSubject, map[string]any{"sessionId": sessionID, "text": text})
|
_ = s.bus.Publish(ctx, wsChatSubject, map[string]any{"sessionId": sessionID, "text": text, "image": image})
|
||||||
}
|
}
|
||||||
|
|
||||||
// wsToken extracts the JWT from the ?token= query or the Authorization header.
|
// wsToken extracts the JWT from the ?token= query or the Authorization header.
|
||||||
|
|||||||
@@ -15,10 +15,11 @@ func (c *wsConn) route(in wsInbound) {
|
|||||||
case "chat":
|
case "chat":
|
||||||
c.publishPublic(wsChatSubject, rawData(in.Data))
|
c.publishPublic(wsChatSubject, rawData(in.Data))
|
||||||
case "voice_input":
|
case "voice_input":
|
||||||
// Voice is not wired to a live pipeline yet; surface the payload on the
|
// Voice routes through the injected STT→chat→TTS pipeline. Without a
|
||||||
// bus so an upper layer (or a later milestone) can consume it, rather
|
// voice service it replies with a clear error rather than dropping the
|
||||||
// than dropping it as unknown.
|
// frame. The audio reply is delivered as a voice_output event on the
|
||||||
c.publishPublic(wsVoiceInput, rawData(in.Data))
|
// requesting connection.
|
||||||
|
c.handleVoiceInput(in)
|
||||||
case "subagent_ask":
|
case "subagent_ask":
|
||||||
c.handleSubagentAsk(in)
|
c.handleSubagentAsk(in)
|
||||||
case "task_output_query":
|
case "task_output_query":
|
||||||
|
|||||||
Reference in new issue
Block a user