 Aska
|
886fcf2216
|
fix(memory): rewrite dedup with n-gram keyword overlap + embedder
- Keyword extraction: 2-3 char sliding window + segment splitting
- Dedup: keyword overlap (>25%) OR Jaccard (>0.4) triggers merge
- SearchByNGrams: pgvector GIN index on keywords array
- Embedder: n-gram FNV hash (zero dep), OpenAI API with auto-fallback
- GIN index on keywords column for fast overlap search
- All tests pass
|
2026-08-12 18:24:04 +08:00 |
|