feat(model): LLM token 预算可配置化 — models.json每模型maxTokens + LLM_FALLBACK_MAX_TOKENS回退兜底
- config.go: LLMConfig.FallbackMaxTokens(LLM_FALLBACK_MAX_TOKENS, 默认8192) - runtimeconfig/models.go: Group.MaxTokens(models.json每模型独立预算, 负值拒绝) - model/fallback.go: Chain.WithTokenBudget + applyBudget; runtime.go buildSnapshot接入(active.MaxTokens, env.FallbackMaxTokens) - 优先级: 显式Request.MaxTokens > 组maxTokens > 仅多步链末端的LLM_FALLBACK_MAX_TOKENS > 适配器默认8192; 单步链不动env兜底(避免OpenAI主模型行为回归) - gateway admin: AdminLlmProvider.MaxTokens 透传/回显/负值400 - .env.example + .env(LLM_FALLBACK_MAX_TOKENS=8192); Docs/api-client v0.27 + api-reference + design/04 - 新增测试(config/runtimeconfig/budget_test/admin_write), build/vet/test绿+3平台交叉编译
This commit is contained in:
1 parent
f347681034
commit
53002f1e11
13 files changed
+384
-15
No files matched your search
@@ -445,9 +445,10 @@ curl -s http://<host>:8080/api/v1/admin/iot -H "Authorization: Bearer $TOKEN" -H
|
||||
|
||||
### 4.9 GET /api/v1/admin/llm
|
||||
- **响应**:`200 {"data":{"providers":[<AdminLlmProvider>...]}}`(`AdminLlmConfig`)。
|
||||
- `AdminLlmProvider`:`{id, name, provider, baseUrl, model, multimodal, enabled, fallback?}`。
|
||||
- `AdminLlmProvider`:`{id, name, provider, baseUrl, model, multimodal, enabled, fallback?, maxTokens?}`。
|
||||
- `provider` 协议 ∈ `openai|anthropic`;`name` 为友好标签(OpenAI/Anthropic 或组 id)。
|
||||
- `enabled`:index 0 的组为生效组 = `true`。
|
||||
- `maxTokens`:该组的 token 预算(`omitempty`,缺省/0 = 未设);缺省时链末端回退步回落 `LLM_FALLBACK_MAX_TOKENS`(`.env`,默认 8192),再回落适配器默认(anthropic 8192)。
|
||||
- **API key 永不返回**(wire 无 key 字段)。
|
||||
- 数据源:`models.json` 各组(index 0 = 生效组);无 `models.json` 时按 `.env` 合成单组。curl:
|
||||
|
||||
@@ -676,7 +677,7 @@ curl -s -X DELETE http://<host>:8080/api/v1/admin/automations/<id> -H "Authoriza
|
||||
```
|
||||
|
||||
### 5.13 POST /api/v1/admin/llm
|
||||
- **Body**(`llmProviderRequest`):`{"id?","name?","provider?","baseUrl","apiKey?","model","multimodal?","fallback?"}`。`baseUrl`+`model` 必填;`provider` 空默认 `openai`(仅允许 `openai|anthropic`)。
|
||||
- **Body**(`llmProviderRequest`):`{"id?","name?","provider?","baseUrl","apiKey?","model","multimodal?","fallback?","maxTokens?"}`。`baseUrl`+`model` 必填;`provider` 空默认 `openai`(仅允许 `openai|anthropic`);`maxTokens` 缺省/`0` = 未设,负数回 `400`。
|
||||
- **响应**:`201 {"data":{"provider":<AdminLlmProvider>,"persisted":true,"path":"<models.json 绝对路径>"}}`。**真实持久化**:读-改-写 `models.json`,保存组**提升为生效组**(index 0),并**热重载**运行中的 LLM stack(无需重启)。
|
||||
- **错误**:`400`(缺 baseUrl/model / invalid provider);`500`(LLM 服务未注入 / 写盘失败 / 热重载失败)。**API key 写入文件但永不回显、不落日志**。
|
||||
- **审计**:`llm.addProvider`。curl:
|
||||
|
||||
Reference in new issue
Block a user