feat(model): LLM 多供应商适配(Adapter/OpenAI 非流式+SSE 流式/per-role 路由/主备降级/多模态检测/调用记录)
This commit is contained in:
1 parent
51b2bdfba2
commit
d09ccc7570
9 files changed
+1285
No files matched your search
@@ -0,0 +1,99 @@
|
||||
package model
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
|
||||
"git.yeij.top/AskaEth/Cyrene_v2/internal/config"
|
||||
)
|
||||
|
||||
// Step binds a model name to an Adapter for one entry in a fallback chain. An
|
||||
// empty Model keeps the incoming Request.Model.
|
||||
type Step struct {
|
||||
Model string
|
||||
Adapter Adapter
|
||||
}
|
||||
|
||||
// Chain is an Adapter that tries its steps in order and returns the first
|
||||
// successful response. It implements Adapter, so it can be used anywhere a
|
||||
// single provider is expected.
|
||||
type Chain struct {
|
||||
steps []Step
|
||||
}
|
||||
|
||||
// NewChain builds a fallback chain from the given steps.
|
||||
func NewChain(steps ...Step) *Chain {
|
||||
return &Chain{steps: steps}
|
||||
}
|
||||
|
||||
// NewOpenAIChatChain builds a fallback chain from a single supplier config: the
|
||||
// primary model first, then cfg.Fallback (when set and distinct), both served
|
||||
// by an OpenAIChatAdapter against the same OpenAI-compatible endpoint. It
|
||||
// returns ErrNoModel when no base URL or primary model is configured.
|
||||
func NewOpenAIChatChain(cfg config.LLMConfig, opts ...OpenAIOption) (*Chain, error) {
|
||||
if cfg.BaseURL == "" || cfg.Model == "" {
|
||||
return nil, ErrNoModel
|
||||
}
|
||||
steps := []Step{{Model: cfg.Model, Adapter: NewOpenAIChat(cfg, opts...)}}
|
||||
if cfg.Fallback != "" && cfg.Fallback != cfg.Model {
|
||||
steps = append(steps, Step{Model: cfg.Fallback, Adapter: NewOpenAIChat(cfg, opts...)})
|
||||
}
|
||||
return NewChain(steps...), nil
|
||||
}
|
||||
|
||||
// Name returns the chain identifier.
|
||||
func (c *Chain) Name() string { return "fallback" }
|
||||
|
||||
// Chat tries each step in order and returns the first success. If every step
|
||||
// fails, the last error is returned wrapped as ErrUpstream.
|
||||
func (c *Chain) Chat(ctx context.Context, req Request, opts Options) (Response, error) {
|
||||
if len(c.steps) == 0 {
|
||||
return Response{}, ErrNoSteps
|
||||
}
|
||||
var lastErr error
|
||||
for _, s := range c.steps {
|
||||
stepReq := req
|
||||
if s.Model != "" {
|
||||
stepReq.Model = s.Model
|
||||
}
|
||||
resp, err := s.Adapter.Chat(ctx, stepReq, opts)
|
||||
if err == nil {
|
||||
return resp, nil
|
||||
}
|
||||
lastErr = err
|
||||
}
|
||||
return Response{}, fmt.Errorf("model: %w: %v", ErrUpstream, lastErr)
|
||||
}
|
||||
|
||||
// ChatStream tries each step in order. If a step fails before delivering any
|
||||
// delta, the chain moves to the next step; once a step has delivered output it
|
||||
// is committed, so a mid-stream failure surfaces as an error rather than mixing
|
||||
// content across providers.
|
||||
func (c *Chain) ChatStream(ctx context.Context, req Request, onChunk func(delta string)) (Response, error) {
|
||||
if len(c.steps) == 0 {
|
||||
return Response{}, ErrNoSteps
|
||||
}
|
||||
var lastErr error
|
||||
for _, s := range c.steps {
|
||||
stepReq := req
|
||||
if s.Model != "" {
|
||||
stepReq.Model = s.Model
|
||||
}
|
||||
delivered := false
|
||||
wrapped := func(delta string) {
|
||||
delivered = true
|
||||
if onChunk != nil {
|
||||
onChunk(delta)
|
||||
}
|
||||
}
|
||||
resp, err := s.Adapter.ChatStream(ctx, stepReq, wrapped)
|
||||
if err == nil {
|
||||
return resp, nil
|
||||
}
|
||||
if delivered {
|
||||
return Response{}, fmt.Errorf("model: %w: stream step %q failed after partial output: %v", ErrUpstream, s.Model, err)
|
||||
}
|
||||
lastErr = err
|
||||
}
|
||||
return Response{}, fmt.Errorf("model: %w: %v", ErrUpstream, lastErr)
|
||||
}
|
||||
Reference in new issue
Block a user