feat: SearXNG 搜索集成 + DevTools Docker + PG 备份 + 文档更新

- web_search 工具/插件接入自托管 SearXNG,支持百度/必应/搜狗/360搜索
- DevTools 加入 docker-compose.dev.yml,devtools/Dockerfile
- scripts/pg-backup.sh 数据库备份恢复脚本,docs/pg-backup-migration.md
- 后台思考 + datetime 插件时区默认 Asia/Shanghai
- docker-compose 对齐 volume 名称,清理 tool-engine 残留引用
- README.md / Deploy.md 更新至当前架构(移除简报/tool-engine,新增搜索/跨端同步/DevTools)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
2026-05-26 20:36:38 +08:00
parent 08687bb13d
commit b14d267642
15 changed files with 574 additions and 137 deletions
+4
View File
@@ -31,12 +31,16 @@ backend/cmd/
# ========== 运行时数据 ==========
logs/
backups/
*.log
*.pid
uploads/
backend/gateway/uploads/
data/
# ========== nginx 部署配置 (仅服务器端使用,不进仓库) ==========
nginx-ssl.conf
# ========== 环境与敏感配置 ==========
.env
backend/.env
+30 -19
View File
@@ -53,7 +53,7 @@ cp backend/.env.example backend/.env
./devtools.sh start --build
```
首次运行会编译全部后端 Go 服务(约 1-2 分钟),之后按依赖顺序启动全部 9 个服务,每步等待健康检查通过。
首次运行会编译全部后端 Go 服务(约 1-2 分钟),之后按依赖顺序启动全部 8 个服务,每步等待健康检查通过。
### 4. 打开控制台
@@ -83,28 +83,25 @@ docker compose -f docker-compose.dev.db.yml up -d
# 1) 记忆服务 (端口 8091)
cd backend/memory-service && go build -o main.exe ./cmd/main.go && ./main.exe
# 2) 工具引擎 (端口 8092)
cd backend/tool-engine && go build -o main.exe ./cmd/main.go && ./main.exe
# 3) 插件管理器 (端口 8094)
# 2) 插件管理器 (端口 8094)
cd backend/plugin-manager && go build -o main.exe ./cmd/main.go && ./main.exe
# 4) IoT 调试服务 (端口 8083)
# 3) IoT 调试服务 (端口 8083)
cd backend/iot-debug-service && go build -o main.exe ./cmd/main.go && ./main.exe
# 5) 语音服务 (端口 8093)
# 4) 语音服务 (端口 8093)
cd backend/voice-service && go build -o main.exe ./cmd/main.go && ./main.exe
# 6) AI-Core (端口 8081)
# 5) AI-Core (端口 8081)
cd backend/ai-core && go build -o main.exe ./cmd/main.go && ./main.exe
# 7) 多平台桥接 (端口 8095)
# 6) 多平台桥接 (端口 8095)
cd backend/platform-bridge && go build -o main.exe ./cmd/main.go && ./main.exe
# 8) Gateway (端口 8080)
# 7) Gateway (端口 8080)
cd backend/gateway && go build -o main.exe ./cmd/main.go && ./main.exe
# 9) 前端 (端口 5173)
# 8) 前端 (端口 5173)
cd frontend/web && npm install && npx vite --host 0.0.0.0
```
@@ -120,7 +117,7 @@ cd frontend/web && npm install && npx vite --host 0.0.0.0
docker compose -f docker-compose.dev.yml up -d
```
启动服务:postgres, redis, qdrant, minio, nats, memory-service, tool-engine, voice-service, iot-debug-service, ai-core, gateway。前端需本地启动。
启动服务:postgres, redis, qdrant, minio, nats, searxng, memory-service, voice-service, iot-debug-service, ai-core, gateway, devtools。前端需本地启动。
> plugin-manager 和 platform-bridge 目前不包含在 docker-compose 中,需本地启动。
@@ -144,18 +141,19 @@ Cyrene/
│ ├── ai-core/ # AI 推理核心 (LLM 编排、人设注入、工具调用、后台思考)
│ ├── gateway/ # API 网关 (JWT、路由、限流、WebSocket Hub)
│ ├── memory-service/ # 记忆服务 (CRUD、语义检索、衰减、自动提取)
│ ├── tool-engine/ # 工具引擎 (13+ 内置工具:计算器/HTTP/IoT/文件等)
│ ├── voice-service/ # 语音服务 (DashScope STT + Edge-TTS)
│ ├── iot-debug-service/ # IoT 调试服务 (8 个模拟智能家居设备)
│ ├── plugin-manager/ # 插件管理器 (14 个内置插件)
│ ├── plugin-manager/ # 插件管理器 (管理 API,插件逻辑在 pkg/plugins)
│ ├── platform-bridge/ # 多平台桥接 (QQ / Telegram / Discord / Webhook)
│ └── pkg/ # 共享包 (logger)
│ └── pkg/ # 共享包 (logger, plugins — 15 个通用插件/工具)
├── devtools/ # DevTools 管理面板 (Express + WebSocket)
├── scripts/ # 辅助脚本 (migrate / tunnel / whisper-setup)
├── scripts/ # 辅助脚本 (migrate / tunnel / whisper-setup / pg-backup)
├── searxng/ # SearXNG 搜索引擎配置
├── backups/ # 数据库备份文件
├── test/ # E2E 测试
├── docs/ # 文档
├── docker-compose.dev.db.yml # 开发基础设施
├── docker-compose.dev.yml # 开发环境 (DB + 后端)
├── docker-compose.dev.yml # 开发环境 (DB + 后端 + DevTools + SearXNG)
├── docker-compose.yml # 生产环境 (+ Caddy)
└── devtools.sh # DevTools CLI
```
@@ -171,7 +169,7 @@ Cyrene/
| 8081 | AI-Core | 否 |
| 8083 | IoT Debug | 否 |
| 8091 | Memory Service | 否 |
| 8092 | Tool Engine | 否 |
| 8088 | SearXNG | 否 |
| 8093 | Voice Service | 否 |
| 8094 | Plugin Manager | 否 |
| 8095 | Platform Bridge | 否 |
@@ -233,7 +231,6 @@ Cyrene/
| 变量 | 默认值 |
|------|--------|
| `MEMORY_SERVICE_URL` | `http://localhost:8091` |
| `TOOL_ENGINE_URL` | `http://localhost:8092` |
| `VOICE_SERVICE_URL` | `http://localhost:8093` |
| `IOT_SERVICE_URL` | `http://localhost:8083` |
@@ -267,6 +264,20 @@ docker exec -it cyrene_postgres psql -U cyrene -d cyrene_ai
---
## 数据库备份
```bash
# 备份数据库
./scripts/pg-backup.sh backup
# 从最新备份恢复
./scripts/pg-backup.sh restore
```
备份文件保存在 `backups/` 目录,自动保留最近 7 个。详见 [docs/pg-backup-migration.md](docs/pg-backup-migration.md)。
---
## 平台迁移
从 Linux 迁移到 Windows 的详细指南见 [Migration.md](Migration.md)。
+27 -21
View File
@@ -16,20 +16,22 @@
│ Gateway (Go/Gin) │
│ localhost:8080 │
│ JWT Auth · Rate Limit · WS Hub · API 路由 │
└──┬───────┬────────┬────────┬────────┬────────┬────────┬─────────┘
│ │ │ │ │ │ │
▼ ▼ ▼ ▼ ▼ ▼ ▼
┌─────┐┌─────┐┌──────┐┌──────┐┌──────┐┌──────┐┌──────┐┌──────────┐
│AI ││Mem- ││Tool ││Voice ││IoT ││Plugin││Plat- ││ Infra │
│Core ││ory ││Engine││Svc ││Debug ││Mgr ││form ││ │
│:8081││:8091││:8092 ││:8093 ││:8083 ││:8094 ││Bridge││ PG:5432 │
│ ││ ││ ││ ││ ││ ││:8095 ││ Redis │
│LLM ││CRUD ││工具 ││STT/ ││模拟 ││插件 ││QQ/ ││ :6379 │
│编排 ││检索 ││调用 ││TTS ││设备 ││托管 ││TG/ ││ Qdrant │
│人设 ││衰减 ││链 ││ ││管理 ││沙箱 ││DC/ ││ :6333 │
│后台 ││ ││ ││ ││ ││ ││Webhk ││ MinIO │
│思考 ││ ││ ││ ││ ││ ││ ││ :9000 │
└─────┘└─────┘└──────┘└──────┘└──────┘└──────┘└──────┘└──────────┘
└──┬───────┬────────┬────────┬────────┬────────┬──────────┘
│ │ │ │ │ │ │
▼ ▼ ▼ ▼ ▼ ▼ ▼
┌─────┐┌─────┐┌──────┐┌──────┐┌──────┐┌──────┐┌──────────┐
│AI ││Mem- ││Voice ││IoT ││Plugin││Plat- ││ Infra │
│Core ││ory ││Svc ││Debug ││Mgr ││form ││ │
│:8081││:8091││:8093 ││:8083 ││:8094 ││Bridge││ PG:5432 │
│ ││ ││ ││ ││ ││:8095 ││ Redis │
│LLM ││CRUD ││STT/ ││模拟 ││插件 ││QQ/ ││ :6379 │
│编排 ││检索 ││TTS ││设备 ││托管 ││TG/ ││ Qdrant │
│人设 ││衰减 ││ ││管理 ││沙箱 ││DC/ ││ :6333 │
│后台 ││ ││ ││ ││ ││Webhk ││ MinIO │
│思考 ││ ││ ││ ││ ││ ││ :9000 │
│ ││ ││ ││ ││ ││ ││ SearXNG │
│ ││ ││ ││ ││ ││ ││ :8088 │
└─────┘└─────┘└──────┘└──────┘└──────┘└──────┘└──────────┘
```
**客户端只需连接 Gateway (8080)**。所有后端服务不直接对外暴露。
@@ -42,13 +44,14 @@
- **IoT 操控** — 8 个模拟智能家居设备(灯/空调/窗帘/传感器/门锁),语音/文本控制
- **记忆管理** — LLM 驱动的长期记忆提取、存储、语义检索、衰减(pgvector)
- **自动化** — 规则引擎 + 场景执行(定时/条件触发/Webhook)
- **每日简报** — 定时生成当日汇总并推送
- **提醒** — 创建/管理定时提醒,到期 WebSocket 推送
- **知识库** — 文档管理 + 向量语义检索
- **文件管理** — 上传/下载/缩略图/图片 AI 分析
- **语音交互** — 服务端 DashScope STT + Edge-TTS,支持实时流式语音
- **WebSocket** — 实时消息推送、IoT 状态广播、通知、流式响应
- **后台思考** — AI 在对话间隙自主反思和记忆整理
- **跨端消息同步** — 多设备实时消息广播、会话隔离与去重
- **互联网搜索** — 自托管 SearXNG 搜索引擎,支持百度/必应/搜狗/360
- **PWA** — 可安装为桌面/移动应用
- **多平台桥接** — QQ / Telegram / Discord / Webhook 第三方平台接入
- **插件系统** — 14 个内置插件(计算器/HTTP/加密/搜索/IoT 等),沙箱隔离
@@ -92,7 +95,7 @@ docker compose -f docker-compose.dev.db.yml up -d
devtools.bat start --build
```
按依赖顺序编译并启动全部 9 个服务:memory → tool-engine → plugin-manager → iot-debug → voice → ai-core → platform-bridge → gateway → frontend。
按依赖顺序编译并启动全部 8 个服务:memory → plugin-manager → iot-debug → voice → ai-core → platform-bridge → gateway → frontend。
启动后访问:
@@ -127,17 +130,18 @@ Cyrene/
│ ├── ai-core/ # AI 推理核心 (LLM 编排、人设注入、工具调用、后台思考)
│ ├── gateway/ # API 网关 (JWT 认证、路由、限流、WebSocket Hub)
│ ├── memory-service/ # 记忆服务 (CRUD、语义检索、衰减、LLM 提取)
│ ├── tool-engine/ # 工具引擎 (13+ 内置工具,支持工具调用链)
│ ├── voice-service/ # 语音服务 (DashScope STT + Edge-TTS)
│ ├── iot-debug-service/ # IoT 调试服务 (8 个模拟智能家居设备)
│ ├── plugin-manager/ # 插件管理器 (14 个内置插件、沙箱隔离)
│ ├── plugin-manager/ # 插件管理器 (管理 API,插件逻辑在 pkg/plugins)
│ ├── platform-bridge/ # 多平台桥接 (QQ / Telegram / Discord / Webhook)
│ └── pkg/ # 共享包 (logger)
│ └── pkg/ # 共享包 (logger, plugins — 15 个通用插件/工具)
├── devtools/ # DevTools 管理面板 (Express + WebSocket)
├── scripts/ # 辅助脚本 (migrate / tunnel / whisper-setup)
├── scripts/ # 辅助脚本 (migrate / tunnel / whisper-setup / pg-backup)
├── backups/ # 数据库备份文件 (.gitignore)
├── test/ # E2E 测试
├── docs/ # 文档与调试记录
│ └── api/ # API 文档
├── searxng/ # SearXNG 搜索引擎配置
├── docker-compose.dev.db.yml # 开发基础设施 (仅 DB)
├── docker-compose.dev.yml # 开发环境一键启动
├── docker-compose.yml # 生产环境 (含 Caddy)
@@ -157,7 +161,7 @@ Cyrene/
| 8081 | AI-Core | 否 |
| 8083 | IoT Debug | 否 |
| 8091 | Memory Service | 否 |
| 8092 | Tool Engine | 否 |
| 8088 | SearXNG | 否 |
| 8093 | Voice Service | 否 |
| 8094 | Plugin Manager | 否 |
| 8095 | Platform Bridge | 否 |
@@ -184,6 +188,7 @@ Cyrene/
| 向量库 | Qdrant |
| 对象存储 | MinIO |
| 消息队列 | NATS |
| 搜索 | SearXNG (自托管元搜索引擎) |
| 语音 | DashScope STT / Edge-TTS / Whisper.cpp |
| 反向代理 | Caddy (生产环境) |
@@ -198,6 +203,7 @@ Cyrene/
| [docs/api/devtools.md](docs/api/devtools.md) | DevTools CLI + Web 控制台文档 |
| [docs/api/backend-services/](docs/api/backend-services/) | 后端服务 API 文档 |
| [docs/dev_must_read.md](docs/dev_must_read.md) | 开发者必读 |
| [docs/pg-backup-migration.md](docs/pg-backup-migration.md) | PG 备份与迁移指南 |
---
+6 -1
View File
@@ -178,7 +178,12 @@ func main() {
registerPluginTools(toolRegistry, &pluginJSON.JSONPlugin{})
registerPluginTools(toolRegistry, pluginFile.NewFilePlugin(dataDir))
registerPluginTools(toolRegistry, pluginHTTP.NewHTTPPlugin())
registerPluginTools(toolRegistry, pluginWS.NewWebSearchPlugin())
searxngURL := getEnv("SEARXNG_URL", "")
if searxngURL != "" {
registerPluginTools(toolRegistry, pluginWS.NewWebSearchPluginWithURL(searxngURL))
} else {
registerPluginTools(toolRegistry, pluginWS.NewWebSearchPlugin())
}
registerPluginTools(toolRegistry, pluginWF.NewWebFetchPlugin())
// ai-core 专属工具 — 通过 sdk.Tool 适配器注册
+35 -4
View File
@@ -125,6 +125,9 @@ type Thinker struct {
userOnline bool
lastOnlineChange time.Time
userSessionID string // 当前活跃的 session ID (用于重连)
// 时区设置 (默认 Asia/Shanghai,可通过 TZ 环境变量覆盖)
timeLocation *time.Location
}
// AutonomousToolPolicy 自主思考工具调用安全策略
@@ -247,6 +250,17 @@ func NewThinker(
adminSessionID string,
memClient *memory.Client,
) *Thinker {
// 加载时区配置
tzName := os.Getenv("TZ")
if tzName == "" {
tzName = "Asia/Shanghai"
}
loc, err := time.LoadLocation(tzName)
if err != nil {
log.Printf("[后台思考] 无效时区 '%s',回退到 Asia/Shanghai: %%v", tzName, err)
loc, _ = time.LoadLocation("Asia/Shanghai")
}
return &Thinker{
enabled: cfg.Enabled,
personaLoader: personaLoader,
@@ -261,6 +275,7 @@ func NewThinker(
minThinkGap: cfg.MinThinkGap,
offlineThinkGap: cfg.OfflineThinkGap,
memoryStore: memoryStore,
timeLocation: loc,
toolRegistry: toolRegistry,
convStore: convStore,
@@ -863,14 +878,30 @@ func (t *Thinker) buildThinkingUserPrompt(
var sb strings.Builder
// 注入当前现实时间,让模型对时间有感知
now := time.Now()
now := time.Now().In(t.timeLocation)
weekdayNames := []string{"周日", "周一", "周二", "周三", "周四", "周五", "周六"}
sb.WriteString(fmt.Sprintf("🕐 现在是 %s %s %02d:%02d。\n",
hour := now.Hour()
minute := now.Minute()
ampm := ""
if hour >= 0 && hour < 6 {
ampm = "凌晨"
} else if hour < 9 {
ampm = "早上"
} else if hour < 12 {
ampm = "上午"
} else if hour < 14 {
ampm = "中午"
} else if hour < 18 {
ampm = "下午"
} else {
ampm = "晚上"
}
sb.WriteString(fmt.Sprintf("🕐 现在是 %s %s %s%d:%02d (%s)。\n",
now.Format("2006年1月2日"),
weekdayNames[now.Weekday()],
now.Hour(), now.Minute()))
ampm, hour, minute,
t.timeLocation.String()))
// 根据触发原因使用不同的开场白
switch triggerReason {
case "post_chat":
sb.WriteString("开拓者刚和你聊完天。你想自然地在心里回味一下刚才的对话……\n")
@@ -303,11 +303,15 @@ func (t *DateTimeTool) handleTimezoneList() (*ToolResult, error) {
}, nil
}
// getTimezone extracts the timezone from arguments, defaulting to local.
// getTimezone extracts the timezone from arguments, defaulting to Asia/Shanghai.
func (t *DateTimeTool) getTimezone(arguments map[string]interface{}) (*time.Location, error) {
tzName, _ := arguments["timezone"].(string)
if tzName == "" {
return time.Local, nil
loc, err := time.LoadLocation("Asia/Shanghai")
if err != nil {
return time.Local, nil
}
return loc, nil
}
loc, err := time.LoadLocation(tzName)
if err != nil {
+102 -38
View File
@@ -11,10 +11,11 @@ import (
"time"
)
// WebSearchTool 网页搜索工具 - 基于 DuckDuckGo Instant Answer API
// WebSearchTool 网页搜索工具 - 基于 SearXNG (或 DuckDuckGo fallback)
type WebSearchTool struct {
client *http.Client
timeout time.Duration
client *http.Client
timeout time.Duration
searxngURL string
}
// NewWebSearchTool 创建网页搜索工具
@@ -27,6 +28,17 @@ func NewWebSearchTool() *WebSearchTool {
}
}
// NewWebSearchToolWithURL 使用 SearXNG 创建搜索工具
func NewWebSearchToolWithURL(searxngURL string) *WebSearchTool {
return &WebSearchTool{
client: &http.Client{
Timeout: 10 * time.Second,
},
timeout: 10 * time.Second,
searxngURL: strings.TrimRight(searxngURL, "/"),
}
}
// Definition 返回工具定义
func (t *WebSearchTool) Definition() ToolDefinition {
return ToolDefinition{
@@ -78,66 +90,124 @@ func (t *WebSearchTool) Execute(ctx context.Context, arguments map[string]interf
}, nil
}
// 使用 DuckDuckGo Instant Answer API
if t.searxngURL != "" {
return t.searchViaSearXNG(ctx, query)
}
return t.searchViaDuckDuckGo(ctx, query)
}
func (t *WebSearchTool) searchViaSearXNG(ctx context.Context, query string) (*ToolResult, error) {
apiURL := fmt.Sprintf("%s/search?format=json&engines=baidu,sogou,360search,bing&q=%s",
t.searxngURL, url.QueryEscape(query))
req, err := http.NewRequestWithContext(ctx, "GET", apiURL, nil)
if err != nil {
return &ToolResult{ToolName: "web_search", Success: false, Error: fmt.Sprintf("创建请求失败: %v", err)}, nil
}
resp, err := t.client.Do(req)
if err != nil {
return &ToolResult{ToolName: "web_search", Success: false, Error: fmt.Sprintf("SearXNG 请求失败: %v", err)}, nil
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return &ToolResult{ToolName: "web_search", Success: false, Error: fmt.Sprintf("SearXNG HTTP %d", resp.StatusCode)}, nil
}
var sr searxngAPIResponse
if err := json.NewDecoder(resp.Body).Decode(&sr); err != nil {
return &ToolResult{ToolName: "web_search", Success: false, Error: fmt.Sprintf("SearXNG 解析失败: %v", err)}, nil
}
var result strings.Builder
result.WriteString(fmt.Sprintf("搜索关键词: %s (共%d条结果)\n\n", query, sr.NumberOrResults))
for _, answer := range sr.Answers {
result.WriteString(fmt.Sprintf("📌 %s\n\n", answer))
}
count := 0
for _, r := range sr.Results {
if count >= 5 {
break
}
if r.Title == "" || r.URL == "" {
continue
}
snippet := cleanSnippet(r.Content)
result.WriteString(fmt.Sprintf("%d. %s\n %s\n %s\n\n", count+1, r.Title, r.URL, snippet))
count++
}
if result.Len() == 0 {
result.WriteString("未找到相关结果。")
}
return &ToolResult{ToolName: "web_search", Success: true, Data: result.String()}, nil
}
// searxngAPIResponse SearXNG JSON 响应
type searxngAPIResponse struct {
NumberOrResults int `json:"number_of_results"`
Results []searxngResult `json:"results"`
Answers []string `json:"answers"`
}
type searxngResult struct {
Title string `json:"title"`
URL string `json:"url"`
Content string `json:"content"`
Score float64 `json:"score"`
}
func cleanSnippet(s string) string {
text := stripHTML(s)
runes := []rune(text)
if len(runes) > 200 {
return string(runes[:200]) + "..."
}
return text
}
func (t *WebSearchTool) searchViaDuckDuckGo(ctx context.Context, query string) (*ToolResult, error) {
apiURL := fmt.Sprintf("https://api.duckduckgo.com/?q=%s&format=json&no_html=1&skip_disambig=1",
url.QueryEscape(query))
req, err := http.NewRequestWithContext(ctx, "GET", apiURL, nil)
if err != nil {
return &ToolResult{
ToolName: "web_search",
Success: false,
Error: fmt.Sprintf("创建请求失败: %v", err),
}, nil
return &ToolResult{ToolName: "web_search", Success: false, Error: fmt.Sprintf("创建请求失败: %v", err)}, nil
}
req.Header.Set("User-Agent", "Mozilla/5.0 (compatible; CyreneBot/1.0)")
resp, err := t.client.Do(req)
if err != nil {
return &ToolResult{
ToolName: "web_search",
Success: false,
Error: fmt.Sprintf("请求失败: %v", err),
}, nil
return &ToolResult{ToolName: "web_search", Success: false, Error: fmt.Sprintf("请求失败: %v", err)}, nil
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return &ToolResult{
ToolName: "web_search",
Success: false,
Error: fmt.Sprintf("HTTP %d", resp.StatusCode),
}, nil
return &ToolResult{ToolName: "web_search", Success: false, Error: fmt.Sprintf("HTTP %d", resp.StatusCode)}, nil
}
body, err := io.ReadAll(io.LimitReader(resp.Body, 500*1024))
if err != nil {
return &ToolResult{
ToolName: "web_search",
Success: false,
Error: fmt.Sprintf("读取响应失败: %v", err),
}, nil
return &ToolResult{ToolName: "web_search", Success: false, Error: fmt.Sprintf("读取响应失败: %v", err)}, nil
}
var ddg duckDuckGoResponse
if err := json.Unmarshal(body, &ddg); err != nil {
return &ToolResult{
ToolName: "web_search",
Success: false,
Error: fmt.Sprintf("解析响应失败: %v", err),
}, nil
return &ToolResult{ToolName: "web_search", Success: false, Error: fmt.Sprintf("解析响应失败: %v", err)}, nil
}
var result strings.Builder
result.WriteString(fmt.Sprintf("搜索关键词: %s\n\n", query))
// 1. 如果有即时答案
if ddg.Answer != "" {
result.WriteString(fmt.Sprintf("📌 即时答案: %s\n\n", ddg.Answer))
}
// 2. 摘要
if ddg.AbstractText != "" {
abstract := ddg.AbstractText
if len([]rune(abstract)) > 500 {
@@ -151,10 +221,8 @@ func (t *WebSearchTool) Execute(ctx context.Context, arguments map[string]interf
result.WriteString("\n")
}
// 3. 相关话题
topics := ddg.RelatedTopics
if len(ddg.Results) > 0 {
// 优先用 Results
count := 0
for _, r := range ddg.Results {
if count >= 5 {
@@ -198,11 +266,7 @@ func (t *WebSearchTool) Execute(ctx context.Context, arguments map[string]interf
result.WriteString("未找到相关结果。")
}
return &ToolResult{
ToolName: "web_search",
Success: true,
Data: result.String(),
}, nil
return &ToolResult{ToolName: "web_search", Success: true, Data: result.String()}, nil
}
// stripHTML 去除 HTML 标签
+5 -1
View File
@@ -126,7 +126,11 @@ func (t *DatetimeTool) Execute(_ context.Context, args map[string]interface{}) (
func parseLocation(tz string) (*time.Location, error) {
if tz == "" {
return time.UTC, nil
loc, err := time.LoadLocation("Asia/Shanghai")
if err != nil {
return time.UTC, nil
}
return loc, nil
}
return time.LoadLocation(tz)
}
+114 -11
View File
@@ -14,33 +14,62 @@ import (
type WebSearchPlugin struct {
sdk.BasePlugin
client *http.Client
client *http.Client
searxngURL string
}
func NewWebSearchPlugin() *WebSearchPlugin {
return &WebSearchPlugin{client: &http.Client{Timeout: 10 * time.Second}}
}
func NewWebSearchPluginWithURL(searxngURL string) *WebSearchPlugin {
return &WebSearchPlugin{
client: &http.Client{Timeout: 10 * time.Second},
searxngURL: strings.TrimRight(searxngURL, "/"),
}
}
func (p *WebSearchPlugin) Metadata() sdk.PluginMetadata {
return sdk.PluginMetadata{
Name: "web_search", DisplayName: "Web Search", Version: "1.0.0",
Description: "Search the internet via DuckDuckGo Instant Answer API",
Name: "web_search", DisplayName: "Web Search", Version: "1.1.0",
Description: "Search the internet via SearXNG (or DuckDuckGo fallback)",
Category: "network", Author: sdk.PluginAuthor{Name: "Cyrene Team"},
}
}
func (p *WebSearchPlugin) Tools() []sdk.Tool { return []sdk.Tool{&WebSearchTool{client: p.client}} }
func (p *WebSearchPlugin) Tools() []sdk.Tool {
return []sdk.Tool{&WebSearchTool{client: p.client, searxngURL: p.searxngURL}}
}
type WebSearchTool struct {
sdk.BaseTool
client *http.Client
client *http.Client
searxngURL string
}
// ---- SearXNG response types ----
type searxngResponse struct {
Query string `json:"query"`
NumberOrResults int `json:"number_of_results"`
Results []searxngResult `json:"results"`
Answers []string `json:"answers"`
Suggestions []string `json:"suggestions"`
}
type searxngResult struct {
Title string `json:"title"`
URL string `json:"url"`
Content string `json:"content"`
Engine string `json:"engine"`
Score float64 `json:"score"`
}
// ---- DuckDuckGo response types (fallback) ----
type ddgResponse struct {
Abstract string `json:"Abstract"`
AbstractText string `json:"AbstractText"`
Answer string `json:"Answer"`
Heading string `json:"Heading"`
Abstract string `json:"Abstract"`
AbstractText string `json:"AbstractText"`
Answer string `json:"Answer"`
Heading string `json:"Heading"`
Results []ddgTopic `json:"Results"`
RelatedTopics []ddgTopic `json:"RelatedTopics"`
}
@@ -53,7 +82,7 @@ type ddgTopic struct {
func (t *WebSearchTool) Definition() sdk.ToolDefinition {
return sdk.ToolDefinition{
ID: "web_search", Name: "web_search", DisplayName: "Web Search",
Description: "Search the internet using DuckDuckGo Instant Answer API. Returns up to 5 results.",
Description: "Search the internet. SearXNG backend with DuckDuckGo fallback. Returns up to 5 results.",
Category: "network", Complexity: sdk.ComplexitySimple,
Parameters: map[string]interface{}{
"type": "object",
@@ -72,6 +101,71 @@ func (t *WebSearchTool) Validate(args map[string]interface{}) error {
func (t *WebSearchTool) Execute(_ context.Context, args map[string]interface{}) (*sdk.ToolResult, error) {
query, _ := args["query"].(string)
if query == "" {
return &sdk.ToolResult{ToolName: "web_search", Success: false, Error: "empty query"}, nil
}
if t.searxngURL != "" {
return t.searchViaSearXNG(query)
}
return t.searchViaDuckDuckGo(query)
}
// China-accessible SearXNG engines (baidu, sogou, 360search, bing all work from China)
const searxngEngines = "baidu,sogou,360search,bing"
func (t *WebSearchTool) searchViaSearXNG(query string) (*sdk.ToolResult, error) {
apiURL := fmt.Sprintf("%s/search?format=json&engines=%s&q=%s",
t.searxngURL, searxngEngines, url.QueryEscape(query))
resp, err := t.client.Get(apiURL)
if err != nil {
return &sdk.ToolResult{ToolName: "web_search", Success: false,
Error: fmt.Sprintf("SearXNG request failed: %v", err)}, nil
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return &sdk.ToolResult{ToolName: "web_search", Success: false,
Error: fmt.Sprintf("SearXNG returned HTTP %d", resp.StatusCode)}, nil
}
var result searxngResponse
if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
return &sdk.ToolResult{ToolName: "web_search", Success: false,
Error: fmt.Sprintf("SearXNG parse error: %v", err)}, nil
}
var out strings.Builder
out.WriteString(fmt.Sprintf("搜索: %s (共%d条结果)\n\n", query, result.NumberOrResults))
// 优先显示答案(如 Wikipedia infobox
for _, answer := range result.Answers {
out.WriteString(fmt.Sprintf("📌 %s\n\n", answer))
}
// 搜索结果(最多5条,按score排序)
count := 0
for _, r := range result.Results {
if count >= 5 {
break
}
if r.Title == "" || r.URL == "" {
continue
}
content := cleanSnippet(r.Content)
out.WriteString(fmt.Sprintf("%d. **%s**\n %s\n %s\n\n", count+1, r.Title, r.URL, content))
count++
}
if out.Len() == 0 {
return &sdk.ToolResult{ToolName: "web_search", Success: true,
Output: fmt.Sprintf("未找到与「%s」相关的结果。", query)}, nil
}
return &sdk.ToolResult{ToolName: "web_search", Success: true, Output: out.String()}, nil
}
func (t *WebSearchTool) searchViaDuckDuckGo(query string) (*sdk.ToolResult, error) {
apiURL := fmt.Sprintf("https://api.duckduckgo.com/?q=%s&format=json&no_html=1", url.QueryEscape(query))
resp, err := t.client.Get(apiURL)
if err != nil {
@@ -111,11 +205,20 @@ func (t *WebSearchTool) Execute(_ context.Context, args map[string]interface{})
count++
}
if out.Len() == 0 {
return &sdk.ToolResult{ToolName: "web_search", Success: true, Output: "No results found for: " + query}, nil
return &sdk.ToolResult{ToolName: "web_search", Success: true,
Output: "No results found for: " + query}, nil
}
return &sdk.ToolResult{ToolName: "web_search", Success: true, Output: out.String()}, nil
}
func cleanSnippet(s string) string {
runes := []rune(strings.TrimSpace(s))
if len(runes) > 200 {
return string(runes[:200]) + "..."
}
return string(runes)
}
func stripHTML(s string) string {
result := make([]rune, 0, len([]rune(s)))
inTag := false
+16
View File
@@ -0,0 +1,16 @@
FROM node:22-slim
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci --omit=dev
COPY src/ ./src/
COPY public/ ./public/
ENV DEVTOOLS_PORT=9090
ENV NODE_ENV=development
EXPOSE 9090
CMD ["node", "src/index.js"]
+52 -27
View File
@@ -5,6 +5,7 @@ services:
# ========== 基础设施 ==========
postgres:
image: pgvector/pgvector:pg16
container_name: cyrene_postgres
environment:
POSTGRES_USER: cyrene
POSTGRES_PASSWORD: cyrene_pass
@@ -12,7 +13,7 @@ services:
ports:
- "5432:5432"
volumes:
- pg_data:/var/lib/postgresql/data
- cyrene_pg_data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U cyrene -d cyrene_ai"]
interval: 10s
@@ -21,10 +22,11 @@ services:
redis:
image: redis:7-alpine
container_name: cyrene_redis
ports:
- "6379:6379"
volumes:
- redis_data:/data
- cyrene_redis_data:/data
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 10s
@@ -33,14 +35,16 @@ services:
qdrant:
image: qdrant/qdrant:latest
container_name: cyrene_qdrant
ports:
- "6333:6333"
- "6334:6334"
volumes:
- qdrant_data:/qdrant/storage
- cyrene_qdrant_data:/qdrant/storage
minio:
image: minio/minio:latest
container_name: cyrene_minio
command: server /data --console-address ":9001"
environment:
MINIO_ROOT_USER: minioadmin
@@ -49,16 +53,51 @@ services:
- "9000:9000"
- "9001:9001"
volumes:
- minio_data:/data
- cyrene_minio_data:/data
nats:
image: nats:2-alpine
container_name: cyrene_nats
ports:
- "4222:4222"
- "8222:8222"
# ========== 搜索引擎 ==========
searxng:
image: searxng/searxng:latest
container_name: cyrene_searxng
volumes:
- ./searxng/settings.yml:/etc/searxng/settings.yml:ro
environment:
SEARXNG_SETTINGS_PATH: /etc/searxng/settings.yml
ports:
- "8088:8080"
restart: unless-stopped
# ========== 开发调试工具 ==========
devtools:
container_name: cyrene_devtools
build:
context: ./devtools
dockerfile: Dockerfile
environment:
DEVTOOLS_PORT: "9090"
GATEWAY_URL: http://gateway:8080
AI_CORE_URL: http://ai-core:8081
MEMORY_SERVICE_URL: http://memory-service:8091
VOICE_SERVICE_URL: http://voice-service:8093
PLATFORM_BRIDGE_URL: http://platform-bridge:8095
ADMIN_USERNAME: admin
ADMIN_PASSWORD: cyrene-dev-admin
ports:
- "9090:9090"
depends_on:
- gateway
restart: unless-stopped
# ========== 后端服务 ==========
memory-service:
container_name: cyrene_memory_service
build:
context: ./backend/memory-service
dockerfile: Dockerfile
@@ -76,23 +115,8 @@ services:
condition: service_healthy
restart: unless-stopped
tool-engine:
build:
context: ./backend/tool-engine
dockerfile: Dockerfile
environment:
PORT: "8092"
IOT_SERVICE_URL: http://iot-debug-service:8083
DATA_DIR: /tmp/cyrene_data
DB_URL: "postgres://cyrene:cyrene_pass@postgres:5432/cyrene_ai?sslmode=disable"
ports:
- "8092:8092"
depends_on:
postgres:
condition: service_healthy
restart: unless-stopped
voice-service:
container_name: cyrene_voice_service
build:
context: ./backend/voice-service
dockerfile: Dockerfile
@@ -106,6 +130,7 @@ services:
restart: unless-stopped
ai-core:
container_name: cyrene_ai_core
build:
context: ./backend/ai-core
dockerfile: Dockerfile
@@ -122,6 +147,7 @@ services:
POSTGRES_PASSWORD: cyrene_pass
POSTGRES_DB: cyrene_ai
IOT_DEBUG_SERVICE_URL: http://iot-debug-service:8083
SEARXNG_URL: http://searxng:8080
ENABLE_BACKGROUND_THINKING: ${ENABLE_BACKGROUND_THINKING:-true}
ports:
- "8081:8081"
@@ -133,6 +159,7 @@ services:
restart: unless-stopped
iot-debug-service:
container_name: cyrene_iot_debug_service
build:
context: ./backend/iot-debug-service
dockerfile: Dockerfile
@@ -143,6 +170,7 @@ services:
restart: unless-stopped
gateway:
container_name: cyrene_gateway
build:
context: ./backend/gateway
dockerfile: Dockerfile
@@ -153,7 +181,6 @@ services:
JWT_EXPIRY_HOURS: "720"
AI_CORE_URL: http://ai-core:8081
MEMORY_SERVICE_URL: http://memory-service:8091
TOOL_ENGINE_URL: http://tool-engine:8092
VOICE_SERVICE_URL: http://voice-service:8093
IOT_DEBUG_SERVICE_URL: http://iot-debug-service:8083
POSTGRES_HOST: postgres
@@ -174,14 +201,12 @@ services:
condition: service_started
memory-service:
condition: service_started
tool-engine:
condition: service_started
voice-service:
condition: service_started
restart: unless-stopped
volumes:
pg_data:
redis_data:
qdrant_data:
minio_data:
cyrene_pg_data:
cyrene_redis_data:
cyrene_qdrant_data:
cyrene_minio_data:
+10 -13
View File
@@ -2,6 +2,15 @@
version: '3.8'
services:
# ========== 搜索引擎 ==========
searxng:
image: searxng/searxng:latest
volumes:
- ./searxng/settings.yml:/etc/searxng/settings.yml:ro
environment:
SEARXNG_SETTINGS_PATH: /etc/searxng/settings.yml
restart: unless-stopped
# ========== 反向代理 ==========
caddy:
image: caddy:2-alpine
@@ -25,7 +34,6 @@ services:
JWT_EXPIRY_HOURS: "720"
AI_CORE_URL: http://ai-core:8081
MEMORY_SERVICE_URL: http://memory-service:8091
TOOL_ENGINE_URL: http://tool-engine:8092
VOICE_SERVICE_URL: http://voice-service:8093
IOT_DEBUG_SERVICE_URL: http://iot-debug-service:8083
POSTGRES_HOST: postgres
@@ -52,6 +60,7 @@ services:
LLM_API_KEY: ${LLM_API_KEY}
LLM_MODEL: ${LLM_MODEL:-gpt-4o}
LLM_FALLBACK_MODEL: ${LLM_FALLBACK_MODEL:-gpt-4o-mini}
SEARXNG_URL: http://searxng:8080
POSTGRES_HOST: postgres
POSTGRES_PORT: "5432"
POSTGRES_USER: ${POSTGRES_USER:-cyrene}
@@ -76,18 +85,6 @@ services:
condition: service_healthy
restart: unless-stopped
tool-engine:
build: ./backend/tool-engine
environment:
PORT: "8092"
IOT_SERVICE_URL: http://iot-debug-service:8083
DATA_DIR: /tmp/cyrene_data
DB_URL: "postgres://${POSTGRES_USER:-cyrene}:${POSTGRES_PASSWORD}@postgres:5432/${POSTGRES_DB:-cyrene_ai}?sslmode=disable"
depends_on:
postgres:
condition: service_healthy
restart: unless-stopped
voice-service:
build: ./backend/voice-service
environment:
+99
View File
@@ -0,0 +1,99 @@
# PostgreSQL 备份与迁移
## 备份脚本
`scripts/pg-backup.sh` 封装了 `pg_dump` / `psql`,跨任何 PG 版本兼容。
```bash
./scripts/pg-backup.sh backup # 备份
./scripts/pg-backup.sh restore # 从最新备份恢复
./scripts/pg-backup.sh restore <文件路径> # 恢复指定备份
```
备份文件写入 `backups/cyrene_YYYYMMDD_HHMMSS.sql`,自动保留最近 7 个。
## PG 大版本升级(16 → 17 等)
生产环境用 `docker-compose.yml`,开发环境用 `docker-compose.dev.yml`
```bash
# === 步骤 1:备份 ===
./scripts/pg-backup.sh backup
# === 步骤 2:停服并清理 ===
docker compose -f docker-compose.dev.yml down
# 删旧 volume,确保新 PG 从干净状态初始化(可选但推荐)
docker volume rm cyrene_cyrene_pg_data
# === 步骤 3:修改镜像版本 ===
# docker-compose.dev.yml 中:
# image: pgvector/pgvector:pg16 → image: pgvector/pgvector:pg17
# === 步骤 4:启动新 PG ===
docker compose -f docker-compose.dev.yml up -d postgres
# 等 healthcheck 通过
docker ps --filter name=cyrene_postgres
# === 步骤 5:恢复数据 ===
./scripts/pg-backup.sh restore
# === 步骤 6:启动全栈 ===
docker compose -f docker-compose.dev.yml up -d
```
## 版本间切换(dev.db.yml ↔ dev.yml
两个文件的 volume 名和容器名已对齐,直接切换即可,数据自动继承:
```bash
# 从轻量版切到全服务版
docker compose -f docker-compose.dev.db.yml down
docker compose -f docker-compose.dev.yml up -d
# 反向切回
docker compose -f docker-compose.dev.yml down
docker compose -f docker-compose.dev.db.yml up -d
```
原有的 `cyrene_*_data` volume 会被新容器直接挂载,无需恢复。
## 跨服务器迁移
```bash
# 源服务器
./scripts/pg-backup.sh backup
scp backups/cyrene_*.sql user@target:/path/to/backups/
# 目标服务器
./scripts/pg-backup.sh restore backups/cyrene_*.sql
```
## 常见问题
### 新版 PG 启动后不接受旧 volume
`pg_upgrade` 要求 in-place 升级必须用 PG 自带的升级工具,不能直接挂旧数据目录。如果没删旧 volume 导致启动失败:
```bash
docker logs cyrene_postgres # 会提示 data directory incompatible
docker compose down
docker volume rm cyrene_cyrene_pg_data
docker compose up -d
./scripts/pg-backup.sh restore
```
### 备份文件过大
`pg_dump` 产生的 `.sql` 文件包含全部数据,如果数据库过大可以考虑:
```bash
# 仅导出结构,不导出日志/缓存表
docker exec cyrene_postgres pg_dump -U cyrene -d cyrene_ai \
--schema-only > schema.sql
# 按表分别导出
docker exec cyrene_postgres pg_dump -U cyrene -d cyrene_ai \
-t memories -t sessions -t users > core_tables.sql
```
+54
View File
@@ -0,0 +1,54 @@
#!/bin/bash
# Cyrene PostgreSQL 备份/恢复
# 用法: ./pg-backup.sh backup # 备份到 ./backups/
# ./pg-backup.sh restore # 从最新备份恢复
set -euo pipefail
CONTAINER="cyrene_postgres"
DB="cyrene_ai"
USER="cyrene"
BACKUP_DIR="$(cd "$(dirname "$0")/../backups" && pwd)"
backup() {
mkdir -p "$BACKUP_DIR"
local file="$BACKUP_DIR/cyrene_$(date +%Y%m%d_%H%M%S).sql"
echo "正在备份 $DB$file ..."
docker exec "$CONTAINER" pg_dump -U "$USER" -d "$DB" \
--no-owner --no-acl --clean --if-exists > "$file"
ls -lh "$file"
echo "备份完成: $file"
# 只保留最近 7 个备份
local count=$(ls -1 "$BACKUP_DIR"/*.sql 2>/dev/null | wc -l)
if [ "$count" -gt 7 ]; then
ls -1t "$BACKUP_DIR"/*.sql | tail -n +8 | xargs rm -f
echo "已清理旧备份 (保留最近 7 个)"
fi
}
restore() {
local file
if [ -n "${1:-}" ]; then
file="$1"
else
file=$(ls -1t "$BACKUP_DIR"/*.sql 2>/dev/null | head -1)
fi
if [ -z "$file" ] || [ ! -f "$file" ]; then
echo "错误: 找不到备份文件" >&2
exit 1
fi
echo "正在从 $file 恢复..."
docker exec -i "$CONTAINER" psql -U "$USER" -d "$DB" < "$file"
echo "恢复完成"
}
case "${1:-}" in
backup) backup ;;
restore) restore "${2:-}" ;;
*)
echo "用法: $0 backup|restore [文件路径]"
echo " backup 备份到 $BACKUP_DIR"
echo " restore [file] 恢复备份 (默认最新)"
exit 1
;;
esac
+14
View File
@@ -0,0 +1,14 @@
# SearXNG settings for Cyrene AI
use_default_settings: true
server:
secret_key: "cyrene-searxng-09dc71af7e2b4f6d"
limiter: false
search:
formats:
- html
- json
redis:
url: redis://redis:6379/1