修复知识库搜索两个bug:
1. 否定式查询惩罚: 查询'是否X'时不含核心名词section×0.3,在reranking后执行 2. 子串回退: token交集失败时用2-6字子串匹配(解决jieba碎片token问题)
This commit is contained in:
+16
-1
@@ -117,7 +117,22 @@ class AutoReply:
|
||||
if has_heading_hit or has_body_hit:
|
||||
return True
|
||||
|
||||
# 3. 高匹配度但无共同词:可能是TF-IDF误匹配,拒绝
|
||||
# 3. 子串回退:jieba分词可能产生"包是""还是"等碎片token,
|
||||
# 导致token交集为空但文档确实包含查询核心词(如"资源包")。
|
||||
# 检查查询中任意2~6字子串是否出现在文档文本中。
|
||||
full_text = doc_text + " " + body[:500]
|
||||
stripped_query = re.sub(r'[??!!。,,\s]', '', query)
|
||||
if len(stripped_query) >= 4:
|
||||
for qlen in range(min(len(stripped_query), 6), 1, -1):
|
||||
for start in range(len(stripped_query) - qlen + 1):
|
||||
phrase = stripped_query[start:start + qlen]
|
||||
if len(phrase) >= 2 and phrase in full_text:
|
||||
logger.debug(
|
||||
f"子串回退命中: phrase={phrase!r} in doc={doc.get('heading','')[:30]!r}"
|
||||
)
|
||||
return True
|
||||
|
||||
# 4. 高匹配度但无共同词:可能是TF-IDF误匹配,拒绝
|
||||
# 用户问"PVP击杀播报",KB返回"json格式说明"(匹配度9.53)→ 应拒绝
|
||||
if score >= 10.0 and not has_heading_hit and not has_body_hit:
|
||||
return False
|
||||
|
||||
Reference in New Issue
Block a user