feat(config): 新增 LLM_MULTIMODAL —— 让 env-only 部署能声明「主模型自身能收图」(默认 false,行为不变)
- 起因(Android 会话质询):其测到生效配置 multimodal:false 却与 DeepSeek 视觉文档矛盾。核实结论: **既非设计选择也非漏配,而是 env 路径的表达力限制** —— `data/models.json` 不存在 ⇒ 走 `GroupFromEnv`, 而它此前硬编 false(理由:不得声称端点可能不具备的能力);该标志的正式载体只有 models.json 组(面板写入)。 后果:env-only 部署里图像**恒走 KindVisionTool**(每回合多一次完整模型调用 + 模型只见文字描述=有损)。 - 修法:`LLMConfig.Multimodal`(`env:"LLM_MULTIMODAL" default:"false"`)+ `GroupFromEnv` 两个 provider 分叉都接入; **只对 env 合成组生效**(有有效 models.json 时组文件优先,与 LLM_REASONING_KIND 同口径);非 bool **启动报错**不静默 - 复核:config 包测试绿(5 条:默认不变 / 显式 true 生效且组仍 Validate / 进程 env 端到端 / 非法值 fail-fast / 组文件优先两方向);**变异 3 条**(分叉写死 false、default 改 true、env 覆盖组文件)均已还原 - 交付给主人的动作:**在 `.env` 写 `LLM_MULTIMODAL=true`** 即可让图直进主模型上下文(改完需重启后端生效)
This commit is contained in:
1 parent
6c164f920b
commit
4adfc9a2df
5 files changed
+214
-21
No files matched your search
@@ -300,13 +300,17 @@ func loadFile(path string) ([]Group, error) {
|
||||
// anthropic block when LLM_PROVIDER=anthropic, otherwise the flat OpenAI-compatible
|
||||
// block. It is the .env fallback used when models.json is absent or invalid.
|
||||
//
|
||||
// Multimodal stays false here: the flat env block has no way to declare whether
|
||||
// the main model accepts image input, and claiming a capability the endpoint may
|
||||
// not have is worse than routing the image through the vision/OCR tool (which is
|
||||
// exactly what the tool exists for). The declaration lives on the models.json
|
||||
// group's `multimodal` flag, which the provider panel writes (design/06 §0 首条;
|
||||
// 裁决 14.4.1). LLM_VISION_MODEL configures the vision *tool's* model, not the
|
||||
// main model's ability.
|
||||
// Multimodal 来自 LLM_MULTIMODAL,默认 false。语义(裁决 14.4.1):它是本组
|
||||
// `multimodal` 字段的 **env 声明**——「主模型自身能否收图」,而不是「本部署有没有
|
||||
// 视觉能力」。默认 false 与引入本键之前的行为逐字节一致:
|
||||
// 扁平 env 块只有这一个键能声明收图能力,而**声称端点可能不具备的能力,比把图
|
||||
// 交给 vision/OCR 工具更糟**——工具至少会返回一段文本描述(design/06 §0 首条)。
|
||||
// 因此只有显式写 `LLM_MULTIMODAL=true` 才让主模型直接收图,**绝不凭 baseURL /
|
||||
// 模型名猜测**(与 reasoning.kind 的口径一致:声明才生效,不推断)。LLM_VISION_MODEL
|
||||
// / LLM_OCR_MODEL 配的是**视觉/OCR 工具调用的模型**,与本组能否收图互不代表。
|
||||
//
|
||||
// 非法值(不是 bool,如 `maybe`)在 config.Load 解码阶段**直接报错**,启动失败,
|
||||
// 不静默回退成 false——静默的「配了当没配」正是本键要消灭的东西。
|
||||
//
|
||||
// Reasoning.Kind is carried through so an env-only deployment can still declare
|
||||
// its endpoint's switch shape (LLM_REASONING_KIND); its env default is blank,
|
||||
@@ -314,23 +318,25 @@ func loadFile(path string) ([]Group, error) {
|
||||
func GroupFromEnv(env config.LLMConfig) []Group {
|
||||
if env.Provider == ProviderAnthropic {
|
||||
return []Group{{
|
||||
ID: ProviderAnthropic,
|
||||
Provider: ProviderAnthropic,
|
||||
BaseURL: env.AnthropicBaseURL,
|
||||
APIKey: env.AnthropicAPIKey,
|
||||
Model: env.AnthropicModel,
|
||||
Fallback: env.AnthropicFallback,
|
||||
Reasoning: ReasoningOptions{Kind: strings.TrimSpace(env.ReasoningKind)},
|
||||
ID: ProviderAnthropic,
|
||||
Provider: ProviderAnthropic,
|
||||
BaseURL: env.AnthropicBaseURL,
|
||||
APIKey: env.AnthropicAPIKey,
|
||||
Model: env.AnthropicModel,
|
||||
Fallback: env.AnthropicFallback,
|
||||
Multimodal: env.Multimodal,
|
||||
Reasoning: ReasoningOptions{Kind: strings.TrimSpace(env.ReasoningKind)},
|
||||
}}
|
||||
}
|
||||
return []Group{{
|
||||
ID: ProviderOpenAI,
|
||||
Provider: ProviderOpenAI,
|
||||
BaseURL: env.BaseURL,
|
||||
APIKey: env.APIKey,
|
||||
Model: env.Model,
|
||||
Fallback: env.Fallback,
|
||||
Reasoning: ReasoningOptions{Kind: strings.TrimSpace(env.ReasoningKind)},
|
||||
ID: ProviderOpenAI,
|
||||
Provider: ProviderOpenAI,
|
||||
BaseURL: env.BaseURL,
|
||||
APIKey: env.APIKey,
|
||||
Model: env.Model,
|
||||
Fallback: env.Fallback,
|
||||
Multimodal: env.Multimodal,
|
||||
Reasoning: ReasoningOptions{Kind: strings.TrimSpace(env.ReasoningKind)},
|
||||
}}
|
||||
}
|
||||
|
||||
|
||||
Reference in new issue
Block a user