Compare commits

...

12 Commits

Author SHA1 Message Date
kplam 35b7decd49 feat: 前端页面优化与入库任务重试/删除
- 概览与检索合并为单一概览页
- 类目/文档管理/入库整合为树形目录知识库页(Library),支持文档搜索
- 入库进度改为右侧 Drawer,支持任务详情、重试、删除
- 后端新增任务重试与删除接口
2026-08-04 12:58:11 +08:00
kplam f0fc20a9b6 feat: 前端补齐批量上传、重新摘要入库与入库进度页
- api/documents.js: 新增 uploadBatch / reingest / taskList 封装
- Ingest.vue: 新增「批量上传」标签页,多文件逐项异步入库并展示结果
- Documents.vue: 文档操作列新增「重新摘要」按钮(读原文件→删旧→新任务)
- 新增 Tasks.vue 入库进度页:近期任务列表 + 2s 自动轮询 + 状态徽标
- 路由与菜单新增「入库进度」入口
2026-08-04 11:52:14 +08:00
kplam bdd30f0a88 fix: 修复会话鉴权「登录后 token 无效」并补齐 NAS 部署流程
- deps.py: _create_redis_client 增加同步 ping 校验,Redis 不可达时正确降级为内存模式
- main.py: lifespan 复用 deps 的 UserStore 单例,避免 admin 与 API 请求实例不一致
- 前端: 强制改密弹窗(must_change_password 用户)、兼容新旧登录返回格式、markPasswordChanged
- 新增 NAS SSH 部署脚本与批处理测试;gitignore 前端构建产物
2026-08-04 11:38:28 +08:00
kplam 884cf78033 fix: 恢复被误删的 5 个运行时依赖
364fede 重构时误将 python-multipart/pypdf/python-docx/pypdfium2/
rapidocr-onnxruntime 从 pyproject 删除,导致:
- python-multipart 缺失使 app 启动即崩(Form 登录依赖)
- 其余为文档解析/OCR 必需(file_parser.py lazy import)
恢复为 92b062c 已验证可稳定运行的依赖基线。
2026-08-03 10:55:01 +08:00
kplam 364fede1b2 build: 精简 Dockerfile,移除镜像内前端构建阶段
前端采用挂载式部署(compose 将 ./frontend/dist 挂到 /app/app/static/admin),
镜像内烘焙前端纯属冗余且导致 NAS 构建时 npm ci 访问 registry 失败。
改为镜像只负责 Python 运行时+依赖,前端由本地 npm run build 后 rsync。
2026-08-01 21:48:41 +08:00
kplam c5d0f4c63e fix: 修复会话鉴权重构导致的测试回归
- 移除 conftest 中对 app.api.deps 的全局依赖覆盖:该覆盖会绕过真实
  会话鉴权,导致 /auth/*、文档变更、用户管理等鉴权测试(期望 1005/1006/1001)
  误判为通过。鉴权测试现走真实 UserStore/SessionStore。
- 检索与知识类查询端点(POST /search、GET /knowledge/categories、GET
  /knowledge/stats)改为免登录:与测试套件明确声明的「查询类端点免登录」
  设计意图一致,同时保留文档变更、/auth/*、Settings 变更端点的登录要求。

变更文件:tests/conftest.py、app/api/v1/search.py、app/api/v1/knowledge.py
2026-08-01 00:48:42 +08:00
kplam a1b80566bb feat: 升级 Ollama 本地模型 qwen2.5:1.5b → qwen3:1.7b
- config.py / .env.example / docker-compose.yml:默认模型改为 qwen3:1.7b(备选 qwen3:4b),compose 同步拉取
- 经 runtime_settings 注入 summarize/query/classify 三用途,改默认值即全局生效
- README / CLAUDE.md / 前端 Settings.vue 占位符同步更新
2026-08-01 00:28:50 +08:00
kplam 83eb8d0dd0 feat: 新增 QMDSearch Agent Skill 包与下载入口
- 新增 skills/qmdsearch-agent(SKILL.md + references/api-examples.md),文档化
  AI Agent 鉴权、文本/文件上传入库(异步轮询)与分层检索调用方式
- 打包 QMDSearch-Agent-Skill.zip 至 app/static/agent-skill/
- 后端新增 GET /agent-skill 免登录下载路由(随 ./app 卷挂载,restart 即生效)
- 前端 API 说明页新增「AI Agent Skill 下载」卡片,链接至 /agent-skill
2026-08-01 00:11:05 +08:00
kplam 6ae91679e2 feat: 接入 BGE-M3 本地嵌入默认配置 + 新增 Qwen3-Reranker 重排
- .env.example 嵌入默认改为本地 BGE-M3(EMBEDDING_PROVIDER=local, dim=1024),NAS 全新部署即用
- 新增 app/services/reranker.py:基于 Ollama /api/rerank 的 cross-encoder 重排服务
- retriever 在 chunk 候选阶段接入语义精排,调用失败优雅降级为 RRF 顺序
- docker-compose 拉取 qwen3-reranker:0.6b,OLLAMA_MAX_LOADED_MODELS 提至 3
- .gitignore 排除 .workbuddy/ 与 .trae/(防止误提交项目记忆与 IDE spec)
- 新增 reranker 单元与集成测试,全量测试 529 passed
2026-08-01 00:08:38 +08:00
kplam 6c6f690788 chore: 完成全量功能迭代与部署准备
- 移除冗余依赖包
- 新增账号禁用校验与用户管理能力
- 新增文档下载与管理页面文件展示
- 新增API文档页面与用户管理前端页面
- 重构时区处理与docker-compose部署配置
- 完善测试用例与项目文档
2026-08-01 00:02:42 +08:00
kplam f92eff6f65 feat: add document file download API and related features
1. add `/api/v1/documents/{doc_id}/file` endpoint for downloading original document files
2. add file field to document detail API response based on metadata
3. add comprehensive tests for file download API and metadata integration
4. update API documentation in CLAUDE.md
2026-07-31 22:46:50 +08:00
kplam e2e8e6829d feat: add document file download and admin page file display
1. 新增将文档元数据存入L1向量库的功能
2. 新增文档文件下载API接口,支持路径安全校验
3. 后端文档详情接口新增文件信息返回字段
4. 管理后台页面新增原始文件信息展示与下载链接
2026-07-31 22:46:13 +08:00
63 changed files with 7070 additions and 1273 deletions
+12 -5
View File
@@ -7,12 +7,14 @@ APP_PORT=8000
LOG_LEVEL=info LOG_LEVEL=info
# --- 嵌入模型 --- # --- 嵌入模型 ---
# openai | local # openai | local。默认走本地 Ollama 的 BGE-M3(多语言榜首、dense+sparse 一体,零 API 成本)
EMBEDDING_PROVIDER=openai EMBEDDING_PROVIDER=local
# 以下为 EMBEDDING_PROVIDER=openai 时启用的 OpenAI 兼容配置(本地模式可留空)
OPENAI_API_KEY=sk-xxx OPENAI_API_KEY=sk-xxx
OPENAI_BASE_URL=https://api.openai.com/v1 OPENAI_BASE_URL=https://api.openai.com/v1
EMBEDDING_MODEL=text-embedding-3-small EMBEDDING_MODEL=text-embedding-3-small
EMBEDDING_DIMENSION=1536 # 维度需与所选嵌入模型一致:BGE-M3 为 1024;切换 OpenAI 时改回 1536
EMBEDDING_DIMENSION=1024
# --- Qdrant --- # --- Qdrant ---
QDRANT_PORT=6333 QDRANT_PORT=6333
@@ -23,11 +25,16 @@ REDIS_PORT=6379
# --- Ollama 本地模型(文档三级总结)--- # --- Ollama 本地模型(文档三级总结)---
OLLAMA_PORT=11434 OLLAMA_PORT=11434
OLLAMA_MODEL=qwen2.5:1.5b OLLAMA_MODEL=qwen3:1.7b
# 备选模型: qwen2.5:3b(更好的总结质量,需更多内存) # 备选模型: qwen3:4b(更好的总结质量,需更多内存)
# EMBEDDING_PROVIDER=local 时使用的 Ollama 嵌入模型 # EMBEDDING_PROVIDER=local 时使用的 Ollama 嵌入模型
OLLAMA_EMBEDDING_MODEL=bge-m3 OLLAMA_EMBEDDING_MODEL=bge-m3
# --- 重排模型(cross-encoder,经 Ollama /api/rerank 对 chunk 候选精排)---
# 启用后显著提升最终 Top-K 相关性;需 Ollama 已拉取 RERANKER_MODEL(见 docker-compose
RERANKER_ENABLED=true
RERANKER_MODEL=qwen3-reranker:0.6b
# --- NAS 持久化 --- # --- NAS 持久化 ---
NAS_DATA_DIR=./data NAS_DATA_DIR=./data
+7
View File
@@ -9,6 +9,9 @@ build/
# Frontend # Frontend
node_modules/ node_modules/
# 前端构建产物(本地残留的 Vue 构建 / 符号链接,勿提交;线上由 frontend/dist 挂载)
app/static/admin
# Env # Env
.env .env
.env.local .env.local
@@ -36,3 +39,7 @@ logs/
.pytest_cache/ .pytest_cache/
.coverage .coverage
htmlcov/ htmlcov/
# Project memory & Trae IDE spec (切勿提交)
.workbuddy/
.trae/
+12 -6
View File
@@ -12,7 +12,7 @@ QMDSearch 是面向 AI Agent 的分层信息检索服务,支持多层级知识
- **缓存**: Redis (Docker) - **缓存**: Redis (Docker)
- **关系数据库**: PostgreSQL (Docker, 可选) - **关系数据库**: PostgreSQL (Docker, 可选)
- **嵌入模型**: OpenAI / 本地模型 (通过配置切换) - **嵌入模型**: OpenAI / 本地模型 (通过配置切换)
- **本地推理模型**: Ollama (Docker) — 用于文档三级总结,默认 qwen2.5:1.5b - **本地推理模型**: Ollama (Docker) — 用于文档三级总结,默认 qwen3:1.7b
- **部署**: Docker Compose on NAS - **部署**: Docker Compose on NAS
## 项目结构 ## 项目结构
@@ -90,11 +90,15 @@ QMDSearch/
| POST | `/api/v1/search` | 分层检索 | 免登录 | | POST | `/api/v1/search` | 分层检索 | 免登录 |
| POST | `/api/v1/documents` | 文档入库(202 异步入库,返回 task_id | Bearer | | POST | `/api/v1/documents` | 文档入库(202 异步入库,返回 task_id | Bearer |
| POST | `/api/v1/documents/upload` | multipart 文件上传入库(202 异步,支持 .txt/.md/.html/.htm/.pdf/.docx | Bearer | | POST | `/api/v1/documents/upload` | multipart 文件上传入库(202 异步,支持 .txt/.md/.html/.htm/.pdf/.docx | Bearer |
| POST | `/api/v1/documents/upload-batch` | 批量文件上传入库(多文件,202 异步) | Bearer |
| GET | `/api/v1/documents/tasks` | 入库任务列表(近期,按时间降序) | Bearer |
| GET | `/api/v1/documents/tasks/{task_id}` | 入库任务状态查询(done 附 resultfailed 附 error | 免登录 | | GET | `/api/v1/documents/tasks/{task_id}` | 入库任务状态查询(done 附 resultfailed 附 error | 免登录 |
| GET | `/api/v1/knowledge/categories` | 知识分类类目集 | 免登录 | | GET | `/api/v1/knowledge/categories` | 知识分类类目集 | 免登录 |
| GET | `/api/v1/knowledge/stats` | 统计(四层点数 + 类目分布 + uncategorized 数) | 免登录 | | GET | `/api/v1/knowledge/stats` | 统计(四层点数 + 类目分布 + uncategorized 数) | 免登录 |
| GET | `/api/v1/documents` | 文档列表(limit/offset 分页) | 免登录 | | GET | `/api/v1/documents` | 文档列表(limit/offset 分页) | 免登录 |
| GET | `/api/v1/documents/{doc_id}` | 文档详情 | 免登录 | | GET | `/api/v1/documents/{doc_id}` | 文档详情 | 免登录 |
| GET | `/api/v1/documents/{doc_id}/file` | 下载关联的原始文件(免登录) | 免登录 |
| POST | `/api/v1/documents/{doc_id}/reingest` | 重新摘要入库(读原文件→删旧→重跑流水线) | Bearer |
| DELETE | `/api/v1/documents/{doc_id}` | 删除文档(幂等) | Bearer | | DELETE | `/api/v1/documents/{doc_id}` | 删除文档(幂等) | Bearer |
| POST | `/api/v1/auth/login` | 用户名密码登录,签发 session tokenTTL 12h | 免登录 | | POST | `/api/v1/auth/login` | 用户名密码登录,签发 session tokenTTL 12h | 免登录 |
| POST | `/api/v1/auth/logout` | 退出登录(删除当前 session | Bearer | | POST | `/api/v1/auth/logout` | 退出登录(删除当前 session | Bearer |
@@ -102,26 +106,28 @@ QMDSearch/
| GET | `/api/v1/auth/me` | 当前登录用户信息(脱敏) | Bearer | | GET | `/api/v1/auth/me` | 当前登录用户信息(脱敏) | Bearer |
| GET | `/api/v1/auth/users` | 用户列表(脱敏) | Bearer + admin | | GET | `/api/v1/auth/users` | 用户列表(脱敏) | Bearer + admin |
| POST | `/api/v1/auth/users` | 创建用户(重名/非法用户名/弱密码 1001 | Bearer + admin | | POST | `/api/v1/auth/users` | 创建用户(重名/非法用户名/弱密码 1001 | Bearer + admin |
| PATCH | `/api/v1/auth/users/{username}` | 更新用户角色/启用状态(仅 admin | Bearer + admin |
| POST | `/api/v1/auth/users/{username}/password` | 重置指定用户密码(成功后清除其全部 session | Bearer + admin | | POST | `/api/v1/auth/users/{username}/password` | 重置指定用户密码(成功后清除其全部 session | Bearer + admin |
| DELETE | `/api/v1/auth/users/{username}` | 删除用户(清除其 session;禁删自己/最后一个 admin | Bearer + admin | | DELETE | `/api/v1/auth/users/{username}` | 删除用户(清除其 session;禁删自己/最后一个 admin | Bearer + admin |
| GET | `/admin` | 管理页面 | 页面登录门禁 | | GET | `/admin` | 管理页面 | 页面登录门禁 |
「Bearer」指请求头 `Authorization: Bearer <token>`token 经 `/api/v1/auth/login` 获取;变更类文档端点(POST /documents、POST /documents/upload、DELETE /documents/{id}需 Bearer tokenadmin/user 角色均可),查询类端点免登录。 「Bearer」指请求头 `Authorization: Bearer <token>`token 经 `/api/v1/auth/login` 获取;变更类文档端点(POST /documents、POST /documents/upload、POST /documents/upload-batch、POST /documents/{id}/reingest、DELETE /documents/{id})与 GET /documents/tasks 任务列表需 Bearer tokenadmin/user 角色均可),其余查询类端点免登录。
入库任务状态持久化在 Redis(key: `ingest_task:{task_id}`):进行中与 done 保留 24hfailed 保留 7 天;Redis 不可用时降级为纯内存。 入库任务状态持久化在 Redis(key: `ingest_task:{task_id}`):进行中与 done 保留 24hfailed 保留 7 天;Redis 不可用时降级为纯内存。
## 管理页面 ## 管理页面
浏览器访问 `/admin`,页面带登录门禁(未登录/token 失效自动回登录卡片;must_change_password 用户先强制改密后方可进入)。单页面含个区块:概览、文档管理(列表/详情/删除)、文档入库(文本 + 文件上传)、检索测试台、类目列表、API 指南(端点清单 + 在线测试台)、用户管理(仅 admin 角色挂载,含创建/重置密码/删除)。 浏览器访问 `/admin`,页面带登录门禁(未登录/token 失效自动回登录卡片;must_change_password 用户先强制改密后方可进入)。单页面含个区块(admin 视角;user 角色无用户管理区块):概览、个人中心(user 角色默认进入且可见,含账号信息/改密/退出)、文档管理(列表/详情/删除)、文档入库(文本 + 文件上传,文件上传支持批量多选)、入库进度(展示近期任务与状态,2s 轮询自动刷新)、检索测试台、类目列表、API 指南(端点清单 + 在线测试台)、用户管理(仅 admin 角色挂载,含搜索筛选/行内编辑角色与启用状态/弹窗式重置密码删除)。
## 认证与用户 ## 认证与用户
会话制认证:登录签发 session tokenRedis 持久化,TTL 12h;Redis 不可用时降级为进程内存,重启失效),请求经 `Authorization: Bearer <token>` 携带,鉴权依赖见 `app/api/deps.py` 会话制认证:登录签发 session tokenRedis 持久化,TTL 12h;Redis 不可用时降级为进程内存,重启失效),请求经 `Authorization: Bearer <token>` 携带,鉴权依赖见 `app/api/deps.py`
- **角色与权限边界**: `admin` 拥有全部权限(含 /auth/users* 用户管理);`user` 可登录并调用变更类文档端点(入库/上传/删除),访问用户管理端点返回 1006 - **角色与权限边界**: `admin` 拥有全部权限(含 /auth/users* 用户管理);`user` 可登录并调用变更类文档端点(入库/上传/删除),访问用户管理端点返回 1006
- **启用状态与 PATCH 端点**: 用户记录含 `enabled` 字段(默认 true);admin 可经 `PATCH /auth/users/{username}` 更新角色或启用状态(请求体至少一项字段,空 body 1001;role 非法 1001;用户不存在 1004)。禁用用户时清除其全部 session(旧 token 立即失效 1005),被禁用用户登录拒绝 1005("账号已禁用");最后一个 admin 禁止降级 role 或被禁用(1001)。角色变更不清 session,同一 token 即时反映新角色
- **初始 admin 引导**: 空库启动时 `bootstrap_admin` 自动创建 admin 账号,随机明文密码仅在启动日志中打印一次(must_change_password=true),首次登录后须先经 POST /auth/password 改密,改密前访问其他端点返回 1006 - **初始 admin 引导**: 空库启动时 `bootstrap_admin` 自动创建 admin 账号,随机明文密码仅在启动日志中打印一次(must_change_password=true),首次登录后须先经 POST /auth/password 改密,改密前访问其他端点返回 1006
- **免登录端点**: 查询类端点(POST /search、GET /documents*、GET /knowledge/*、GET /health)不需要 token - **免登录端点**: 查询类端点(POST /search、GET /documents*、GET /knowledge/*、GET /health)不需要 token
- **相关错误码**: 1005 未认证或凭证无效,1006 权限不足/首次登录须先改密 - **相关错误码**: 1005 未认证或凭证无效/账号已禁用,1006 权限不足/首次登录须先改密
## 编码规范 ## 编码规范
@@ -165,8 +171,8 @@ QMDSearch/
### 本地模型 (Ollama) ### 本地模型 (Ollama)
- 服务: Ollama 容器,默认模型 `qwen2.5:1.5b`(约 1GB,适合 NAS 低资源环境) - 服务: Ollama 容器,默认模型 `qwen3:1.7b`(约 1.3GB,适合 NAS 低资源环境)
- 备选模型: `qwen2.5:3b`(更好的总结质量,约 2GB - 备选模型: `qwen3:4b`(更好的总结质量,约 2.5GB
- 模型通过环境变量 `OLLAMA_MODEL` 配置 - 模型通过环境变量 `OLLAMA_MODEL` 配置
- 首次启动时自动拉取模型,需 NAS 可访问外网 - 首次启动时自动拉取模型,需 NAS 可访问外网
+8 -18
View File
@@ -1,17 +1,10 @@
# ---------- Stage 1: 前端构建 ---------- # ---------- Python 后端运行时镜像 ----------
FROM node:20-slim AS frontend-builder # 部署模型说明:
# 前端采用「挂载式」部署——docker-compose 将宿主机的 ./frontend/dist
WORKDIR /frontend # 挂到容器的 /app/app/static/admin,运行时直接覆盖镜像内同名目录。
# 因此镜像【不再构建前端】,只负责 Python 运行时 + 依赖;前端由本地
# 依赖层(利用 Docker 层缓存) # `npm run build` 后 rsync 到 NAS,或挂载既有 dist。
COPY frontend/package.json frontend/package-lock.json ./ # 仅当 pyproject.toml / uv.lock 变化时才需 `docker build -t qmdsearch-app .` 一次。
RUN npm ci
# 源码 + 构建
COPY frontend/ ./
RUN npm run build
# ---------- Stage 2: Python 后端 ----------
FROM python:3.12-slim AS base FROM python:3.12-slim AS base
WORKDIR /app WORKDIR /app
@@ -19,7 +12,7 @@ WORKDIR /app
# 安装 uv(放到 PATH 中,供 uv sync 与 CMD 的 uv run 使用) # 安装 uv(放到 PATH 中,供 uv sync 与 CMD 的 uv run 使用)
COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv
# 依赖层 # 依赖层(利用 Docker 层缓存)
COPY pyproject.toml uv.lock ./ COPY pyproject.toml uv.lock ./
RUN uv sync --frozen --no-dev RUN uv sync --frozen --no-dev
@@ -27,9 +20,6 @@ RUN uv sync --frozen --no-dev
COPY app/ app/ COPY app/ app/
COPY scripts/ scripts/ COPY scripts/ scripts/
# 前端构建产物 → 后端静态目录(SPA 由 /admin 路由服务)
COPY --from=frontend-builder /frontend/dist /app/app/static/admin
EXPOSE 8000 EXPOSE 8000
HEALTHCHECK --interval=30s --timeout=5s --retries=3 \ HEALTHCHECK --interval=30s --timeout=5s --retries=3 \
+3 -3
View File
@@ -22,7 +22,7 @@
| 向量数据库 | QdrantDocker | | 向量数据库 | QdrantDocker |
| 缓存 | RedisDocker | | 缓存 | RedisDocker |
| 嵌入模型 | OpenAI / Ollama 本地模型(可切换) | | 嵌入模型 | OpenAI / Ollama 本地模型(可切换) |
| 本地推理 | Ollama + qwen2.5:1.5b(文档总结) | | 本地推理 | Ollama + qwen3:1.7b(文档总结) |
| OCR | rapidocr-onnxruntime + pypdfium2(扫描件 PDF | | OCR | rapidocr-onnxruntime + pypdfium2(扫描件 PDF |
| 部署 | Docker Compose on NAS | | 部署 | Docker Compose on NAS |
@@ -123,7 +123,7 @@ docker compose up -d
### 4. 拉取 Ollama 模型(首次启动后) ### 4. 拉取 Ollama 模型(首次启动后)
```bash ```bash
docker exec qmdsearch-ollama ollama pull qwen2.5:1.5b docker exec qmdsearch-ollama ollama pull qwen3:1.7b
``` ```
### 5. 验证服务 ### 5. 验证服务
@@ -232,7 +232,7 @@ curl -X POST http://localhost:8000/api/v1/documents/upload \
| `EMBEDDING_PROVIDER` | 嵌入模型提供商(openai / local | `openai` | | `EMBEDDING_PROVIDER` | 嵌入模型提供商(openai / local | `openai` |
| `OPENAI_API_KEY` | OpenAI API Key | - | | `OPENAI_API_KEY` | OpenAI API Key | - |
| `EMBEDDING_MODEL` | 嵌入模型名称 | `text-embedding-3-small` | | `EMBEDDING_MODEL` | 嵌入模型名称 | `text-embedding-3-small` |
| `OLLAMA_MODEL` | Ollama 本地模型 | `qwen2.5:1.5b` | | `OLLAMA_MODEL` | Ollama 本地模型 | `qwen3:1.7b` |
| `RETRIEVAL_TOP_K` | 检索召回数 | `20` | | `RETRIEVAL_TOP_K` | 检索召回数 | `20` |
| `RETRIEVAL_FINAL_K` | 最终返回数 | `5` | | `RETRIEVAL_FINAL_K` | 最终返回数 | `5` |
| `INGEST_MAX_CONCURRENCY` | 入库并发上限 | `2` | | `INGEST_MAX_CONCURRENCY` | 入库并发上限 | `2` |
+16 -3
View File
@@ -8,6 +8,7 @@ UserStore/SessionStore 为模块级懒加载单例:Redis 客户端创建失败
import structlog import structlog
from fastapi import Depends, Header from fastapi import Depends, Header
from redis import asyncio as redis_async from redis import asyncio as redis_async
from redis import Redis as RedisSync
from app.api.response import ApiError from app.api.response import ApiError
from app.config import settings from app.config import settings
@@ -22,11 +23,20 @@ _session_store: SessionStore | None = None
def _create_redis_client() -> redis_async.Redis | None: def _create_redis_client() -> redis_async.Redis | None:
"""创建 redis.asyncio 客户端(decode_responses=True);失败返回 None 走内存降级""" """创建 redis.asyncio 客户端(decode_responses=True);连接失败返回 None 走内存降级
注意:redis.asyncio.from_url() 仅构造客户端对象,不会实际连接 Redis,
因此需要用同步客户端 ping 一次确认连接可用,否则后续操作才会报错,
导致 UserStore/SessionStore 无法降级为内存模式。
"""
try: try:
# 同步 ping 确认 Redis 可达(from_url 不会实际连接)
sync_client = RedisSync.from_url(settings.redis_url, decode_responses=True)
sync_client.ping()
sync_client.close()
return redis_async.from_url(settings.redis_url, decode_responses=True) return redis_async.from_url(settings.redis_url, decode_responses=True)
except Exception: except Exception:
logger.warning("Redis 客户端创建失败,认证存储降级为内存模式", exc_info=True) logger.warning("Redis 连接失败,认证存储降级为内存模式", exc_info=True)
return None return None
@@ -80,9 +90,12 @@ async def _resolve_user(authorization: str | None) -> tuple[UserRecord, str]:
async def get_current_user(authorization: str | None = Header(None)) -> UserRecord: async def get_current_user(authorization: str | None = Header(None)) -> UserRecord:
"""鉴权依赖:校验 Bearer token 并返回当前用户记录 """鉴权依赖:校验 Bearer token 并返回当前用户记录
must_change_password 用户被拦截(1006),须先经 POST /auth/password 改密。 禁用用户被拦截(1005);must_change_password 用户被拦截(1006),须先经 POST /auth/password 改密。
注:logout/password 端点经 _resolve_user 自解析,不在此拦截范围内(改密是已登录用户唯一可用接口)。
""" """
user, _ = await _resolve_user(authorization) user, _ = await _resolve_user(authorization)
if not user.enabled:
raise ApiError(1005, "账号已禁用")
if user.must_change_password: if user.must_change_password:
raise ApiError(1006, "首次登录须先修改密码") raise ApiError(1006, "首次登录须先修改密码")
return user return user
+58 -2
View File
@@ -10,12 +10,12 @@ from typing import Any, Literal
import structlog import structlog
from fastapi import APIRouter, Depends, Header from fastapi import APIRouter, Depends, Header
from pydantic import BaseModel from pydantic import BaseModel, model_validator
from app.api import deps from app.api import deps
from app.api.response import ApiError, ok from app.api.response import ApiError, ok
from app.core.sessions import SessionStoreError from app.core.sessions import SessionStoreError
from app.core.users import UserExistsError, UserRecord, UserStoreError from app.core.users import UserExistsError, UserNotFoundError, UserRecord, UserStoreError
logger = structlog.get_logger() logger = structlog.get_logger()
@@ -50,12 +50,26 @@ class PasswordResetRequest(BaseModel):
new_password: str new_password: str
class UserUpdateRequest(BaseModel):
"""管理员更新用户角色/启用状态(至少一项)"""
role: Literal["admin", "user"] | None = None
enabled: bool | None = None
@model_validator(mode="after")
def _at_least_one(self) -> "UserUpdateRequest":
if self.role is None and self.enabled is None:
raise ValueError("至少需要提供一项更新字段(role 或 enabled")
return self
def _public_user(user: UserRecord) -> dict[str, Any]: def _public_user(user: UserRecord) -> dict[str, Any]:
"""用户记录脱敏:剔除 password_hash/salt""" """用户记录脱敏:剔除 password_hash/salt"""
return { return {
"username": user.username, "username": user.username,
"role": user.role, "role": user.role,
"must_change_password": user.must_change_password, "must_change_password": user.must_change_password,
"enabled": user.enabled,
"created_at": user.created_at, "created_at": user.created_at,
} }
@@ -69,6 +83,8 @@ async def login(req: LoginRequest) -> dict[str, Any]:
raise ApiError(2001, "认证服务暂不可用") from exc raise ApiError(2001, "认证服务暂不可用") from exc
if user is None: if user is None:
raise ApiError(1005, "用户名或密码错误") raise ApiError(1005, "用户名或密码错误")
if not user.enabled:
raise ApiError(1005, "账号已禁用")
try: try:
token = await deps._get_session_store().create(user.username, user.role) token = await deps._get_session_store().create(user.username, user.role)
except SessionStoreError as exc: except SessionStoreError as exc:
@@ -151,6 +167,46 @@ async def create_user(
return ok(_public_user(user)) return ok(_public_user(user))
@router.patch("/users/{username}")
async def update_user(
username: str,
req: UserUpdateRequest,
admin: UserRecord = Depends(deps.require_admin),
) -> dict[str, Any]:
"""更新用户角色或启用状态
用户不存在 1004;role 非法 1001;最后一个 admin 降级 role 或被禁用 1001
禁用用户时清除其全部 session(已登录态立即失效)。
"""
store = deps._get_user_store()
try:
target = await store.get(username)
if target is None:
raise ApiError(1004, "用户不存在")
# 最后 admin 保护:目标当前是唯一 admin 且本次会使其失去 admin 身份或被禁用
if target.role == "admin" and await store.count_admins() <= 1:
will_lose_admin = (req.role is not None and req.role != "admin") or (req.enabled is False)
if will_lose_admin:
raise ApiError(1001, "禁止降级或禁用最后一个管理员")
updated = await store.update_user(username, role=req.role, enabled=req.enabled)
except UserStoreError as exc:
raise ApiError(2001, "认证服务暂不可用") from exc
except UserNotFoundError as exc:
raise ApiError(1004, "用户不存在") from exc
except ValueError as exc:
raise ApiError(1001, str(exc)) from exc
# 禁用用户 → 清除其全部 session(已登录态立即失效)
if req.enabled is False:
try:
await deps._get_session_store().delete_by_username(username)
except SessionStoreError as exc:
raise ApiError(2001, "认证服务暂不可用") from exc
logger.info(
"管理员更新用户", username=username, operator=admin.username, role=req.role, enabled=req.enabled
)
return ok(_public_user(updated))
@router.post("/users/{username}/password") @router.post("/users/{username}/password")
async def reset_user_password( async def reset_user_password(
username: str, req: PasswordResetRequest, admin: UserRecord = Depends(deps.require_admin) username: str, req: PasswordResetRequest, admin: UserRecord = Depends(deps.require_admin)
+193 -1
View File
@@ -8,7 +8,7 @@ from typing import Any
import structlog import structlog
from fastapi import APIRouter, Depends, File, Form, Query, UploadFile from fastapi import APIRouter, Depends, File, Form, Query, UploadFile
from fastapi.responses import JSONResponse from fastapi.responses import FileResponse, JSONResponse
from app.api.deps import get_current_user from app.api.deps import get_current_user
from app.api.response import ApiError, ok from app.api.response import ApiError, ok
@@ -183,6 +183,149 @@ async def upload_document(
) )
async def _process_single_upload(file: UploadFile) -> dict[str, Any]:
"""处理单个上传文件:校验 → 提取文本 → 落盘 → 提交入库
供批量上传复用:返回成功 {"filename", "task_id"} 或失败 {"filename", "error"}。
title/source 使用文件名默认值,不接受用户传入的额外表单字段。
"""
original_filename = file.filename or "unnamed"
ext = Path(original_filename).suffix.lower()
# 1. 扩展名校验
allowed = _allowed_extensions()
if ext not in allowed:
return {"filename": original_filename, "error": f"不支持的文件类型: {ext or '(无扩展名)'}"}
# 2. 读取字节并校验大小
try:
content = await file.read()
except Exception as exc:
return {"filename": original_filename, "error": f"文件读取失败: {exc}"}
max_bytes = settings.upload_max_size_mb * 1024 * 1024
if len(content) > max_bytes:
return {"filename": original_filename, "error": f"文件超过大小上限: {settings.upload_max_size_mb}MB"}
# 3. 提取文本
try:
text = parse_file(original_filename, content)
except ValueError as exc:
return {"filename": original_filename, "error": str(exc)}
if not text.strip():
return {"filename": original_filename, "error": "无法从文件提取文本"}
# 4. 落盘(按 YYYY/MM 日期分片;失败仅 warning,不阻塞入库)
doc_id = uuid.uuid4().hex
metadata_dict: dict[str, str] = {}
try:
upload_dir = Path(settings.upload_dir).resolve()
shard_subdir = datetime.now(UTC).strftime("%Y/%m")
target_dir = upload_dir / shard_subdir
target_dir.mkdir(parents=True, exist_ok=True)
target = target_dir / f"{doc_id}_{original_filename}"
target.write_bytes(content)
metadata_dict.update(
{
"raw_file_path": str(target),
"original_filename": original_filename,
"original_size_bytes": str(len(content)),
}
)
logger.info(
"上传文件已落盘", doc_id=doc_id, saved_path=str(target), size=len(content)
)
except Exception:
logger.warning(
"上传文件落盘失败,仅做文本入库",
doc_id=doc_id,
filename=original_filename,
exc_info=True,
)
# 5. 默认 title / source
title = Path(original_filename).stem
source = f"file:{original_filename}"
# 6. 提交入库流水线
doc_input = DocumentInput(text=text, title=title, source=source, metadata=metadata_dict)
task_id = await _get_task_manager().submit(doc_input)
return {"filename": original_filename, "task_id": task_id}
@router.post("/documents/upload-batch")
async def upload_batch(
files: list[UploadFile] = File(...),
user: UserRecord = Depends(get_current_user),
) -> JSONResponse:
"""批量文件上传入库:逐文件复用单文件逻辑,收集成功与失败结果
返回 202 + {tasks: [{filename, task_id}], failed: [{filename, error}]}
空文件列表返回 1001。
"""
if not files:
raise ApiError(1001, "未提供任何文件")
tasks: list[dict[str, Any]] = []
failed: list[dict[str, Any]] = []
for file in files:
result = await _process_single_upload(file)
if "task_id" in result:
tasks.append({"filename": result["filename"], "task_id": result["task_id"]})
else:
failed.append({"filename": result["filename"], "error": result["error"]})
return JSONResponse(status_code=202, content=ok({"tasks": tasks, "failed": failed}))
@router.get("/documents/tasks")
async def list_ingest_tasks(
limit: int = Query(default=20, ge=1, le=100),
user: UserRecord = Depends(get_current_user),
) -> dict[str, Any]:
"""列出近期入库任务(合并内存与 Redis 镜像,按 updated_at 降序)"""
tasks = await _get_task_manager().list_tasks(limit=limit)
return ok({"items": tasks, "total": len(tasks)})
@router.post("/documents/{doc_id}/reingest")
async def reingest_document(
doc_id: str, user: UserRecord = Depends(get_current_user)
) -> JSONResponse:
"""重新入库:读原始文件 → 删旧数据 → 提交新入库任务
仅对有原始文件落盘记录的文档可重新入库;重新入库会生成新 doc_id。
"""
meta = await _get_qdrant().get_l1_metadata(doc_id)
if meta is None:
raise ApiError(1004, "文档不存在")
raw_file_path = meta.get("raw_file_path", "")
if not raw_file_path:
raise ApiError(1001, "该文档无原始文件,无法重新入库")
path = Path(raw_file_path)
if not path.is_file():
raise ApiError(1004, "文件不存在")
# 读原文件并重新提取文本
try:
text = parse_file(meta.get("original_filename", path.name), path.read_bytes())
except ValueError as exc:
raise ApiError(1001, str(exc)) from exc
if not text.strip():
raise ApiError(1001, "无法从文件提取文本")
# 删旧数据(四层集合按 doc_id 清除,幂等)
await _get_qdrant().delete_by_doc_id(doc_id)
# 构造新文档输入(保留原 metadatatitle/source 从 meta 取或文件名回退)
original_filename = meta.get("original_filename", path.name)
title = meta.get("title") or Path(original_filename).stem
source = meta.get("source") or f"file:{original_filename}"
doc_input = DocumentInput(text=text, title=title, source=source, metadata=dict(meta))
task_id = await _get_task_manager().submit(doc_input)
return JSONResponse(
status_code=202, content=ok({"task_id": task_id, "status": "pending"})
)
@router.get("/documents/tasks/{task_id}") @router.get("/documents/tasks/{task_id}")
async def get_ingest_task(task_id: str) -> dict[str, Any]: async def get_ingest_task(task_id: str) -> dict[str, Any]:
"""查询入库任务状态:含 task_id/status/created_at/updated_atdone 附 resultfailed 附 error""" """查询入库任务状态:含 task_id/status/created_at/updated_atdone 附 resultfailed 附 error"""
@@ -192,6 +335,28 @@ async def get_ingest_task(task_id: str) -> dict[str, Any]:
return ok(task) return ok(task)
@router.post("/documents/tasks/{task_id}/retry")
async def retry_ingest_task(
task_id: str, user: UserRecord = Depends(get_current_user)
) -> JSONResponse:
"""重试入库任务:用原 source 重新提交一个新任务,返回新 task_id"""
new_task_id = await _get_task_manager().retry(task_id)
if new_task_id is None:
raise ApiError(1004, "任务不存在或缺少重试所需的源信息")
return JSONResponse(
status_code=202, content=ok({"task_id": new_task_id, "status": "pending"})
)
@router.delete("/documents/tasks/{task_id}")
async def delete_ingest_task(
task_id: str, user: UserRecord = Depends(get_current_user)
) -> dict[str, Any]:
"""删除入库任务(内存注册表 + Redis 镜像),幂等"""
deleted = await _get_task_manager().delete(task_id)
return ok({"task_id": task_id, "deleted": deleted})
@router.get("/documents") @router.get("/documents")
async def list_documents( async def list_documents(
limit: int = Query(default=20, ge=1, le=100), limit: int = Query(default=20, ge=1, le=100),
@@ -219,6 +384,33 @@ async def get_document(doc_id: str) -> dict[str, Any]:
return ok(detail) return ok(detail)
@router.get("/documents/{doc_id}/file")
async def download_document_file(doc_id: str) -> FileResponse:
"""下载文档关联的原始文件(免登录)
从 L1 metadata 读取 raw_file_path,校验路径位于 upload_dir 之内后返回 FileResponse
文档不存在 / 无关联文件 / 文件缺失 / 路径越界统一返回 1004。
"""
meta = await _get_qdrant().get_l1_metadata(doc_id)
if not meta:
raise ApiError(1004, "文档不存在或未关联文件")
raw_path = meta.get("raw_file_path", "")
if not raw_path:
raise ApiError(1004, "文档未关联文件")
path = Path(raw_path).resolve()
# 路径越界校验:只允许读取 upload_dir 下的文件
try:
upload_dir = Path(settings.upload_dir).resolve()
path.relative_to(upload_dir)
except ValueError:
logger.warning("文件路径越界", doc_id=doc_id, raw_path=raw_path)
raise ApiError(1004, "文件不存在") from None
if not path.is_file():
raise ApiError(1004, "文件不存在")
filename = meta.get("original_filename", path.name)
return FileResponse(path, filename=filename)
@router.delete("/documents/{doc_id}") @router.delete("/documents/{doc_id}")
async def delete_document( async def delete_document(
doc_id: str, user: UserRecord = Depends(get_current_user) doc_id: str, user: UserRecord = Depends(get_current_user)
+3 -4
View File
@@ -4,11 +4,10 @@ from functools import lru_cache
from typing import Any from typing import Any
import structlog import structlog
from fastapi import APIRouter, Depends from fastapi import APIRouter
from app.api.response import ApiError, ok from app.api.response import ApiError, ok
from app.config import settings from app.config import settings
from app.core.auth import AuthUser, get_current_user
from app.models.knowledge import UNCATEGORIZED, TaxonomyCategory, load_taxonomy from app.models.knowledge import UNCATEGORIZED, TaxonomyCategory, load_taxonomy
from app.services.qdrant import ALL_COLLECTIONS, COLLECTION_L1, QdrantService from app.services.qdrant import ALL_COLLECTIONS, COLLECTION_L1, QdrantService
@@ -37,14 +36,14 @@ def _get_taxonomy() -> list[TaxonomyCategory]:
@router.get("/knowledge/categories") @router.get("/knowledge/categories")
async def list_categories(user: AuthUser = Depends(get_current_user)) -> dict[str, Any]: async def list_categories() -> dict[str, Any]:
"""返回完整知识分类类目集""" """返回完整知识分类类目集"""
categories = _get_taxonomy() categories = _get_taxonomy()
return ok({"categories": [c.model_dump() for c in categories], "count": len(categories)}) return ok({"categories": [c.model_dump() for c in categories], "count": len(categories)})
@router.get("/knowledge/stats") @router.get("/knowledge/stats")
async def knowledge_stats(user: AuthUser = Depends(get_current_user)) -> dict[str, Any]: async def knowledge_stats() -> dict[str, Any]:
"""返回四层集合规模与 L1 类目分布统计""" """返回四层集合规模与 L1 类目分布统计"""
service = _get_qdrant() service = _get_qdrant()
try: try:
+2 -3
View File
@@ -4,10 +4,9 @@ from hashlib import sha256
from typing import Any from typing import Any
import structlog import structlog
from fastapi import APIRouter, Depends from fastapi import APIRouter
from app.api.response import ApiError, ok from app.api.response import ApiError, ok
from app.core.auth import AuthUser, get_current_user
from app.core.retriever import Retriever from app.core.retriever import Retriever
from app.models.search import SearchRequest from app.models.search import SearchRequest
from app.services.redis import get_cache from app.services.redis import get_cache
@@ -34,7 +33,7 @@ def _cache_key(request: SearchRequest) -> str:
@router.post("/search") @router.post("/search")
async def search(request: SearchRequest, user: AuthUser = Depends(get_current_user)) -> dict[str, Any]: async def search(request: SearchRequest) -> dict[str, Any]:
"""分层检索入口,返回统一包装的 SearchResponse """分层检索入口,返回统一包装的 SearchResponse
先查 Redis 缓存:命中直接返回缓存的响应;未命中走检索流程并回写缓存。 先查 Redis 缓存:命中直接返回缓存的响应;未命中走检索流程并回写缓存。
+6 -5
View File
@@ -11,7 +11,8 @@ from fastapi import APIRouter, Depends
from pydantic import BaseModel from pydantic import BaseModel
from app.api.response import ApiError, ok from app.api.response import ApiError, ok
from app.core.auth import AuthUser, get_current_user, require_admin from app.api.deps import get_current_user, require_admin
from app.core.users import UserRecord
from app.core.dedup import invalidate_dedup_strategy_cache from app.core.dedup import invalidate_dedup_strategy_cache
from app.core.file_parser import ( from app.core.file_parser import (
list_docx_plugins, list_docx_plugins,
@@ -46,7 +47,7 @@ class SettingsUpdateRequest(BaseModel):
@router.get("/settings") @router.get("/settings")
async def get_settings(user: AuthUser = Depends(get_current_user)) -> dict[str, Any]: async def get_settings(user: UserRecord = Depends(get_current_user)) -> dict[str, Any]:
"""返回当前 RuntimeSettings(任何登录用户可读)""" """返回当前 RuntimeSettings(任何登录用户可读)"""
cfg = get_runtime_settings() cfg = get_runtime_settings()
return ok(cfg.model_dump(mode="json")) return ok(cfg.model_dump(mode="json"))
@@ -55,7 +56,7 @@ async def get_settings(user: AuthUser = Depends(get_current_user)) -> dict[str,
@router.put("/settings") @router.put("/settings")
async def update_settings( async def update_settings(
body: SettingsUpdateRequest, body: SettingsUpdateRequest,
user: AuthUser = Depends(require_admin), user: UserRecord = Depends(require_admin),
) -> dict[str, Any]: ) -> dict[str, Any]:
"""部分更新 RuntimeSettings(仅 admin """部分更新 RuntimeSettings(仅 admin
@@ -82,7 +83,7 @@ async def update_settings(
@router.get("/settings/schema") @router.get("/settings/schema")
async def get_settings_schema( async def get_settings_schema(
user: AuthUser = Depends(get_current_user), user: UserRecord = Depends(get_current_user),
) -> dict[str, Any]: ) -> dict[str, Any]:
"""返回可选插件与策略列表(前端 Settings 页渲染选项用)""" """返回可选插件与策略列表(前端 Settings 页渲染选项用)"""
return ok( return ok(
@@ -98,7 +99,7 @@ async def get_settings_schema(
@router.post("/settings/reset") @router.post("/settings/reset")
async def reset_settings( async def reset_settings(
user: AuthUser = Depends(require_admin), user: UserRecord = Depends(require_admin),
) -> dict[str, Any]: ) -> dict[str, Any]:
"""重置 RuntimeSettings 为默认值(仅 admin),同时清缓存""" """重置 RuntimeSettings 为默认值(仅 admin),同时清缓存"""
try: try:
+7 -1
View File
@@ -18,9 +18,15 @@ class Settings(BaseSettings):
# Ollama 本地模型(用于文档三级总结) # Ollama 本地模型(用于文档三级总结)
ollama_base_url: str = "http://localhost:11434" ollama_base_url: str = "http://localhost:11434"
ollama_model: str = "qwen2.5:1.5b" # 备选: qwen2.5:3b ollama_model: str = "qwen3:1.7b" # 备选: qwen3:4b(更好的总结质量,需更多内存)
ollama_embedding_model: str = "bge-m3" # embedding_provider=local 时使用的嵌入模型 ollama_embedding_model: str = "bge-m3" # embedding_provider=local 时使用的嵌入模型
# 重排模型(cross-encoder,经 Ollama /api/rerank,对 chunk 候选做最终精排)
# 关闭时检索链路退化为纯 RRF 融合结果,行为不变
reranker_enabled: bool = False
reranker_model: str = "qwen3-reranker:0.6b"
reranker_timeout: float = 30.0
# Qdrant # Qdrant
qdrant_host: str = "localhost" qdrant_host: str = "localhost"
qdrant_port: int = 6333 qdrant_port: int = 6333
+4 -4
View File
@@ -4,7 +4,7 @@
鉴权(get_current_user)以 JWT 自包含信息为主,Redis 不可用时降级可用。 鉴权(get_current_user)以 JWT 自包含信息为主,Redis 不可用时降级可用。
""" """
from datetime import UTC, datetime, timedelta from datetime import datetime, timedelta, timezone
import bcrypt import bcrypt
import jwt import jwt
@@ -67,7 +67,7 @@ def verify_password(password: str, hashed: str) -> bool:
def create_access_token(username: str, role: str) -> tuple[str, int]: def create_access_token(username: str, role: str) -> tuple[str, int]:
"""签发 JWT,返回 (token, expires_in_seconds)""" """签发 JWT,返回 (token, expires_in_seconds)"""
now = datetime.now(UTC) now = datetime.now(timezone.utc)
expire = now + timedelta(minutes=settings.jwt_expire_minutes) expire = now + timedelta(minutes=settings.jwt_expire_minutes)
payload = { payload = {
"sub": username, "sub": username,
@@ -136,7 +136,7 @@ class UserStore:
user = StoredUser( user = StoredUser(
username=username, username=username,
role=role, role=role,
created_at=datetime.now(UTC), created_at=datetime.now(timezone.utc),
hashed_password=hash_password(password), hashed_password=hash_password(password),
) )
try: try:
@@ -203,7 +203,7 @@ async def get_current_user(
) )
# Redis 不可用或用户不存在(可能已删除):降级用 JWT payload # Redis 不可用或用户不存在(可能已删除):降级用 JWT payload
logger.warning("用户存储查询未命中,降级使用 JWT payload", username=username) logger.warning("用户存储查询未命中,降级使用 JWT payload", username=username)
return AuthUser(username=username, role=role, created_at=datetime.now(UTC)) return AuthUser(username=username, role=role, created_at=datetime.now(timezone.utc))
async def require_admin(user: AuthUser = Depends(get_current_user)) -> AuthUser: async def require_admin(user: AuthUser = Depends(get_current_user)) -> AuthUser:
+120 -1
View File
@@ -20,7 +20,7 @@ from typing import Any
import structlog import structlog
from app.config import Settings from app.config import Settings
from app.core.dedup import DEDUP_KEY_PREFIX, get_dedup_strategy from app.core.dedup import get_dedup_strategy
from app.core.ingestion import Ingester, IngestionError from app.core.ingestion import Ingester, IngestionError
from app.models.document import DocumentInput from app.models.document import DocumentInput
from app.services.redis import RedisCache from app.services.redis import RedisCache
@@ -79,6 +79,8 @@ class IngestTaskManager:
""" """
task_id = uuid.uuid4().hex task_id = uuid.uuid4().hex
now = _utc_now_iso() now = _utc_now_iso()
filename = self._extract_filename(doc)
source = self._extract_source(doc)
dedup = get_dedup_strategy(self._redis) dedup = get_dedup_strategy(self._redis)
# 1. 去重命中:直接置 done,复用旧结果,不调 _run # 1. 去重命中:直接置 done,复用旧结果,不调 _run
@@ -93,6 +95,8 @@ class IngestTaskManager:
"updated_at": now, "updated_at": now,
"result": result_dict, "result": result_dict,
"error": None, "error": None,
"filename": filename,
"source": source,
} }
self._schedule_mirror(task_id) self._schedule_mirror(task_id)
logger.info( logger.info(
@@ -110,6 +114,8 @@ class IngestTaskManager:
"updated_at": now, "updated_at": now,
"result": None, "result": None,
"error": None, "error": None,
"filename": filename,
"source": source,
} }
self._schedule_mirror(task_id) self._schedule_mirror(task_id)
background = asyncio.create_task(self._run(task_id, doc, dedup)) background = asyncio.create_task(self._run(task_id, doc, dedup))
@@ -135,6 +141,119 @@ class IngestTaskManager:
) )
return None return None
async def list_tasks(self, limit: int = 20) -> list[dict[str, Any]]:
"""合并内存注册表与 Redis 镜像,去重,按 updated_at 降序,截断 limit
每项提取 {task_id, status, filename, created_at, updated_at, doc_id}
doc_id 从 done 任务的 result.document_id 提取,其余状态为 None。
"""
records: dict[str, dict[str, Any]] = {}
# 1. 内存注册表(主)
for task_id, record in self._tasks.items():
records[task_id] = record
# 2. Redis 镜像补充内存中没有的(内存未命中的任务,如重启后只存在 Redis 的历史记录)
for task_id, record in await self._scan_redis_tasks():
records.setdefault(task_id, record)
# 3. 按 updated_at 降序,截断 limit
sorted_records = sorted(
records.values(),
key=lambda r: r.get("updated_at") or "",
reverse=True,
)[:limit]
# 4. 提取展示字段
return [self._to_list_item(r) for r in sorted_records]
async def _scan_redis_tasks(self) -> list[tuple[str, dict[str, Any]]]:
"""扫描 Redis 中 ingest_task:* 键,返回 (task_id, record) 列表
RedisCache 未暴露公开扫描接口,借道底层 _get_client().scan_iter
任何异常降级为空列表,不影响 list_tasks 主流程。
"""
if self._redis is None:
return []
get_client = getattr(self._redis, "_get_client", None)
if not callable(get_client):
return []
try:
client = get_client()
tasks: list[tuple[str, dict[str, Any]]] = []
async for key in client.scan_iter(match=f"{REDIS_KEY_PREFIX}*"):
key_str = key if isinstance(key, str) else key.decode("utf-8", "replace")
task_id = key_str[len(REDIS_KEY_PREFIX) :]
record = await self._redis.get_json(key_str)
if isinstance(record, dict):
tasks.append((task_id, record))
return tasks
except Exception:
logger.warning("扫描 Redis 任务键失败", exc_info=True)
return []
@staticmethod
def _extract_filename(doc: DocumentInput) -> str | None:
"""从文档输入提取展示用文件名:优先 metadata.original_filename,其次 title"""
return doc.metadata.get("original_filename") or doc.title or None
@staticmethod
def _extract_source(doc: DocumentInput) -> dict[str, Any]:
"""从文档输入提取重试所需的源信息(供 retry 复用,含文件路径/大小)"""
return {
"text": doc.text,
"title": doc.title,
"source": doc.source,
"metadata": dict(doc.metadata),
}
async def retry(self, task_id: str) -> str | None:
"""重试失败/已完成的入库任务:用原 source 重新提交一个新任务
返回新 task_id;任务不存在或缺少 source 时返回 None。
"""
record = await self.get(task_id)
if record is None:
return None
source = record.get("source")
if not isinstance(source, dict) or not source.get("text"):
return None
doc = DocumentInput(
text=source["text"],
title=source.get("title") or "",
source=source.get("source") or "",
metadata=source.get("metadata") or {},
)
return await self.submit(doc)
async def delete(self, task_id: str) -> bool:
"""删除入库任务(内存注册表 + Redis 镜像),不存在返回 False"""
existed = self._tasks.pop(task_id, None) is not None
if self._redis is not None:
try:
get_client = getattr(self._redis, "_get_client", None)
if callable(get_client):
await get_client().delete(f"{REDIS_KEY_PREFIX}{task_id}")
existed = True
except Exception:
logger.warning("删除 Redis 任务镜像失败", task_id=task_id, exc_info=True)
return existed
@staticmethod
def _to_list_item(record: dict[str, Any]) -> dict[str, Any]:
"""从完整任务记录提取列表展示字段"""
result = record.get("result")
doc_id = result.get("document_id") if isinstance(result, dict) else None
source = record.get("source") or {}
metadata = source.get("metadata") or {}
return {
"task_id": record.get("task_id"),
"status": record.get("status"),
"filename": record.get("filename"),
"title": source.get("title"),
"source": source.get("source"),
"size_bytes": metadata.get("original_size_bytes"),
"created_at": record.get("created_at"),
"updated_at": record.get("updated_at"),
"doc_id": doc_id,
}
async def wait_done(self, task_id: str, timeout: float = 30.0) -> dict[str, Any]: async def wait_done(self, task_id: str, timeout: float = 30.0) -> dict[str, Any]:
"""轮询内存注册表直到任务进入终态(done/failed)或超时 """轮询内存注册表直到任务进入终态(done/failed)或超时
+1
View File
@@ -262,6 +262,7 @@ class Ingester:
tags=category.tags, tags=category.tags,
dense_vector=l1_vector, dense_vector=l1_vector,
sparse_vector=l1_sparse, sparse_vector=l1_sparse,
metadata=doc.metadata,
) )
# L2/L3 大纲节点(为空时跳过对应集合的 upsert) # L2/L3 大纲节点(为空时跳过对应集合的 upsert)
+26 -3
View File
@@ -16,7 +16,7 @@ from qdrant_client import models
from app.config import settings from app.config import settings
from app.core.embeddings import EmbeddingService, create_embedding_service from app.core.embeddings import EmbeddingService, create_embedding_service
from app.core.query_parser import QueryParser, RouteDecision from app.core.query_parser import QueryParser, RouteDecision
from app.core.ranker import finalize, rrf_fuse from app.core.ranker import rrf_fuse
from app.core.result_summarizer import ResultSummarizer from app.core.result_summarizer import ResultSummarizer
from app.core.sparse import SparseEncoder from app.core.sparse import SparseEncoder
from app.models.knowledge import load_taxonomy from app.models.knowledge import load_taxonomy
@@ -30,6 +30,7 @@ from app.services.qdrant import (
SPARSE_COLLECTIONS, SPARSE_COLLECTIONS,
QdrantService, QdrantService,
) )
from app.services.reranker import RerankerService, create_reranker_service
logger = structlog.get_logger() logger = structlog.get_logger()
@@ -49,6 +50,7 @@ class Retriever:
embedding: EmbeddingService | None = None, embedding: EmbeddingService | None = None,
sparse_encoder: SparseEncoder | None = None, sparse_encoder: SparseEncoder | None = None,
result_summarizer: ResultSummarizer | None = None, result_summarizer: ResultSummarizer | None = None,
reranker: RerankerService | None = None,
) -> None: ) -> None:
self.qdrant = qdrant or QdrantService() self.qdrant = qdrant or QdrantService()
self.query_parser = query_parser or QueryParser( self.query_parser = query_parser or QueryParser(
@@ -58,6 +60,7 @@ class Retriever:
self.embedding = embedding or create_embedding_service() self.embedding = embedding or create_embedding_service()
self.sparse_encoder = sparse_encoder or SparseEncoder() self.sparse_encoder = sparse_encoder or SparseEncoder()
self.result_summarizer = result_summarizer or ResultSummarizer() self.result_summarizer = result_summarizer or ResultSummarizer()
self.reranker = reranker if reranker is not None else create_reranker_service()
async def search(self, request: SearchRequest) -> SearchResponse: async def search(self, request: SearchRequest) -> SearchResponse:
"""分层检索主流程""" """分层检索主流程"""
@@ -77,7 +80,7 @@ class Retriever:
# L1 无候选文档 → 全库 chunk 兜底 # L1 无候选文档 → 全库 chunk 兜底
chunk_hits = await self._search_collection(COLLECTION_CHUNKS, dense, sparse, settings.retrieval_top_k, None) chunk_hits = await self._search_collection(COLLECTION_CHUNKS, dense, sparse, settings.retrieval_top_k, None)
logger.info("L1 无命中,全库 chunk 兜底", hits=len(chunk_hits)) logger.info("L1 无命中,全库 chunk 兜底", hits=len(chunk_hits))
hits = self._to_hits(finalize(chunk_hits, self._final_k(request))) hits = self._to_hits(await self._rerank(query_text, chunk_hits, self._final_k(request)))
return await self._build_response(request, route, hits, fallback=True) return await self._build_response(request, route, hits, fallback=True)
doc_ids = _unique((p.payload or {}).get("doc_id") for p in l1_hits) doc_ids = _unique((p.payload or {}).get("doc_id") for p in l1_hits)
@@ -123,7 +126,7 @@ class Retriever:
) )
logger.info("chunk 检索完成", hits=len(chunk_hits)) logger.info("chunk 检索完成", hits=len(chunk_hits))
final_points = finalize(rrf_fuse([chunk_hits]), self._final_k(request)) final_points = await self._rerank(query_text, chunk_hits, self._final_k(request))
hits = self._to_hits(final_points) hits = self._to_hits(final_points)
return await self._build_response(request, route, hits, fallback=route.fallback) return await self._build_response(request, route, hits, fallback=route.fallback)
@@ -145,6 +148,26 @@ class Retriever:
"""最终返回数:请求指定优先,否则用配置默认值""" """最终返回数:请求指定优先,否则用配置默认值"""
return request.top_k or settings.retrieval_final_k return request.top_k or settings.retrieval_final_k
async def _rerank(
self, query_text: str, points: list[models.ScoredPoint], final_k: int
) -> list[models.ScoredPoint]:
"""对 chunk 候选池做语义精排(cross-encoder
重排未启用、候选为空或调用异常时,退化为 RRF 融合的原始顺序(取前 final_k),
保证检索链路在任何情况下都不因重排失败而中断。
"""
if self.reranker is None or not points:
return points[:final_k]
documents = [(p.payload or {}).get("text", "") for p in points]
try:
scores = await self.reranker.rerank(query_text, documents)
except Exception:
logger.warning("重排调用失败,退化为 RRF 融合顺序", exc_info=True)
return points[:final_k]
# 按相关性分数降序重排,写回 score 字段为相关性分数
ordered = sorted(range(len(points)), key=lambda i: scores[i], reverse=True)
return [points[i].model_copy(update={"score": scores[i]}) for i in ordered[:final_k]]
@staticmethod @staticmethod
def _to_hits(points: list[models.ScoredPoint]) -> list[SearchHit]: def _to_hits(points: list[models.ScoredPoint]) -> list[SearchHit]:
"""chunk 点组装为 SearchHit,字段取自 chunk payloaddoc_summary 仅上下文标注)""" """chunk 点组装为 SearchHit,字段取自 chunk payloaddoc_summary 仅上下文标注)"""
+44 -4
View File
@@ -11,7 +11,7 @@ import hmac
import json import json
import re import re
import secrets import secrets
from dataclasses import asdict, dataclass from dataclasses import asdict, dataclass, field
from datetime import UTC, datetime from datetime import UTC, datetime
from redis import asyncio as redis_async from redis import asyncio as redis_async
@@ -41,6 +41,10 @@ class UserStoreError(Exception):
"""用户存储后端读写异常""" """用户存储后端读写异常"""
class UserNotFoundError(Exception):
"""用户不存在"""
@dataclass @dataclass
class UserRecord: class UserRecord:
"""用户记录(不含明文密码)""" """用户记录(不含明文密码)"""
@@ -50,7 +54,9 @@ class UserRecord:
password_hash: str # hex password_hash: str # hex
salt: str # hex, 16 字节 salt: str # hex, 16 字节
must_change_password: bool must_change_password: bool
created_at: str # UTC ISO8601 enabled: bool = True
# created_at 紧随 enabled 之后;二者均有默认值以满足 dataclass 字段顺序约束
created_at: str = field(default_factory=lambda: datetime.now(UTC).isoformat())
class UserStore: class UserStore:
@@ -76,6 +82,7 @@ class UserStore:
password: str, password: str,
role: str = "user", role: str = "user",
must_change_password: bool = False, must_change_password: bool = False,
enabled: bool = True,
) -> UserRecord: ) -> UserRecord:
"""创建用户 """创建用户
@@ -94,6 +101,7 @@ class UserStore:
password_hash=self.hash_password(password, salt), password_hash=self.hash_password(password, salt),
salt=salt.hex(), salt=salt.hex(),
must_change_password=must_change_password, must_change_password=must_change_password,
enabled=enabled,
created_at=datetime.now(UTC).isoformat(), created_at=datetime.now(UTC).isoformat(),
) )
await self._write(record) await self._write(record)
@@ -104,7 +112,7 @@ class UserStore:
raw = await self._read_raw(username) raw = await self._read_raw(username)
if raw is None: if raw is None:
return None return None
return UserRecord(**json.loads(raw)) return self._from_raw(raw)
async def list(self) -> list[UserRecord]: async def list(self) -> list[UserRecord]:
"""列出全部用户(扫描 user:* 键)""" """列出全部用户(扫描 user:* 键)"""
@@ -115,7 +123,15 @@ class UserStore:
raws = [await self._redis.get(key) async for key in self._redis.scan_iter(match=f"{_USER_KEY_PREFIX}*")] raws = [await self._redis.get(key) async for key in self._redis.scan_iter(match=f"{_USER_KEY_PREFIX}*")]
except Exception as e: except Exception as e:
raise UserStoreError("列出用户失败") from e raise UserStoreError("列出用户失败") from e
return [UserRecord(**json.loads(raw)) for raw in raws if raw is not None] return [self._from_raw(raw) for raw in raws if raw is not None]
@staticmethod
def _from_raw(raw: str) -> UserRecord:
"""从 JSON 反序列化 UserRecord;存量记录无 enabled 字段时按 True 兼容"""
data = json.loads(raw)
if "enabled" not in data:
data["enabled"] = True
return UserRecord(**data)
async def delete(self, username: str) -> bool: async def delete(self, username: str) -> bool:
"""删除用户,返回是否删除成功(幂等:不存在返回 False)""" """删除用户,返回是否删除成功(幂等:不存在返回 False)"""
@@ -158,6 +174,30 @@ class UserStore:
"""统计 admin 角色用户数""" """统计 admin 角色用户数"""
return sum(1 for record in await self.list() if record.role == "admin") return sum(1 for record in await self.list() if record.role == "admin")
async def update_user(
self,
username: str,
*,
role: str | None = None,
enabled: bool | None = None,
) -> UserRecord:
"""更新用户角色或启用状态
用户不存在抛 UserNotFoundErrorrole 非 admin/user 抛 ValueError。
role/enabled 为 None 表示不修改对应字段。更新后写回并返回最新记录。
"""
record = await self.get(username)
if record is None:
raise UserNotFoundError(f"用户不存在: {username}")
if role is not None and role not in ("admin", "user"):
raise ValueError(f"非法角色: {role!r}(须为 admin/user")
if role is not None:
record.role = role
if enabled is not None:
record.enabled = enabled
await self._write(record)
return record
async def _read_raw(self, username: str) -> str | None: async def _read_raw(self, username: str) -> str | None:
if self._redis is None: if self._redis is None:
return self._memory.get(username) return self._memory.get(username)
+25 -8
View File
@@ -6,7 +6,6 @@ import structlog
from fastapi import FastAPI, Request from fastapi import FastAPI, Request
from fastapi.exceptions import RequestValidationError from fastapi.exceptions import RequestValidationError
from fastapi.responses import FileResponse, JSONResponse, RedirectResponse from fastapi.responses import FileResponse, JSONResponse, RedirectResponse
from redis import asyncio as redis_async
from app.api.response import ApiError, error from app.api.response import ApiError, error
from app.api.v1.auth import router as auth_router from app.api.v1.auth import router as auth_router
@@ -15,7 +14,7 @@ from app.api.v1.knowledge import router as knowledge_router
from app.api.v1.search import router as search_router from app.api.v1.search import router as search_router
from app.api.v1.settings import router as settings_router from app.api.v1.settings import router as settings_router
from app.config import settings from app.config import settings
from app.core.users import UserStore, bootstrap_admin from app.core.users import bootstrap_admin
from app.services.qdrant import QdrantService from app.services.qdrant import QdrantService
logger = structlog.get_logger() logger = structlog.get_logger()
@@ -33,13 +32,13 @@ async def lifespan(_: FastAPI) -> AsyncIterator[None]:
except Exception: except Exception:
logger.error("Qdrant 集合初始化失败,跳过初始化继续启动") logger.error("Qdrant 集合初始化失败,跳过初始化继续启动")
try: try:
# Redis 客户端创建失败(如 URL 非法)时传 None,UserStore 降级为内存模式 # 复用 deps 的 UserStore 单例,确保 lifespan 创建的 admin 与 API 请求用的是同一实例
redis_client = redis_async.from_url(settings.redis_url, decode_responses=True) # _get_user_store 内部会检测 Redis 可达性,不可达时降级为内存模式
except Exception: from app.api.deps import _get_user_store
redis_client = None
try: user_store = _get_user_store()
# 空库引导默认管理员;明文密码由 bootstrap_admin 内部 warning 打印一次 # 空库引导默认管理员;明文密码由 bootstrap_admin 内部 warning 打印一次
await bootstrap_admin(UserStore(redis_client), logger) await bootstrap_admin(user_store, logger)
except Exception: except Exception:
logger.warning("默认管理员初始化失败,跳过", exc_info=True) logger.warning("默认管理员初始化失败,跳过", exc_info=True)
try: try:
@@ -144,3 +143,21 @@ async def admin_spa(rest: str):
return FileResponse(file_path) return FileResponse(file_path)
# SPA 客户端路由兜底 # SPA 客户端路由兜底
return FileResponse(spa_dir / "index.html", media_type="text/html") return FileResponse(spa_dir / "index.html", media_type="text/html")
# AI Agent Skill 压缩包(用于 Agent 上传文档与检索),随 ./app 卷挂载,restart 即生效
_AGENT_SKILL_ZIP = _STATIC_DIR / "agent-skill" / "QMDSearch-Agent-Skill.zip"
@app.get("/agent-skill", include_in_schema=False, response_model=None)
async def agent_skill_download():
"""下载 AI Agent Skill 压缩包(qmdsearch-agent skill,含 SKILL.md 与示例)"""
if not _AGENT_SKILL_ZIP.is_file():
return JSONResponse(
status_code=404, content={"code": 1002, "message": "Skill 包不存在"}
)
return FileResponse(
_AGENT_SKILL_ZIP,
media_type="application/zip",
filename="QMDSearch-Agent-Skill.zip",
)
+53 -3
View File
@@ -9,6 +9,7 @@
""" """
import uuid import uuid
from pathlib import Path
from typing import Any from typing import Any
import structlog import structlog
@@ -103,18 +104,50 @@ class QdrantService:
tags: list[str], tags: list[str],
dense_vector: list[float], dense_vector: list[float],
sparse_vector: SparseVectorTuple | None = None, sparse_vector: SparseVectorTuple | None = None,
metadata: dict[str, str] | None = None,
) -> None: ) -> None:
"""写入 L1 文档总结,payload 含 doc_id/title/category/tags/text(=summary)""" """写入 L1 文档总结,payload 含 doc_id/title/category/tags/text(=summary)/metadata
metadata 默认 None 时存空 dict,保证字段始终存在;存量旧文档读取时按缺失处理。
"""
vector: dict[str, Any] = {VECTOR_DENSE: dense_vector} vector: dict[str, Any] = {VECTOR_DENSE: dense_vector}
if sparse_vector is not None: if sparse_vector is not None:
vector[VECTOR_SPARSE] = models.SparseVector(indices=sparse_vector[0], values=sparse_vector[1]) vector[VECTOR_SPARSE] = models.SparseVector(indices=sparse_vector[0], values=sparse_vector[1])
point = models.PointStruct( point = models.PointStruct(
id=_point_id(f"{doc_id}:l1"), id=_point_id(f"{doc_id}:l1"),
vector=vector, vector=vector,
payload={"doc_id": doc_id, "title": title, "category": category, "tags": tags, "text": summary}, payload={
"doc_id": doc_id,
"title": title,
"category": category,
"tags": tags,
"text": summary,
"metadata": metadata or {},
},
) )
await self._client.upsert(collection_name=COLLECTION_L1, points=[point]) await self._client.upsert(collection_name=COLLECTION_L1, points=[point])
async def get_l1_metadata(self, doc_id: str) -> dict[str, str] | None:
"""按 doc_id 查 L1 点,返回其 payload.metadata
文档不存在或 payload 无 metadata 字段时返回 None。
"""
doc_filter = self.build_filter(doc_ids=[doc_id])
records, _ = await self._client.scroll(
collection_name=COLLECTION_L1,
scroll_filter=doc_filter,
limit=1,
with_payload=True,
with_vectors=False,
)
if not records:
return None
payload = records[0].payload or {}
meta = payload.get("metadata")
if not isinstance(meta, dict):
return None
return meta
async def upsert_nodes(self, collection: str, nodes: list[dict[str, Any]]) -> None: async def upsert_nodes(self, collection: str, nodes: list[dict[str, Any]]) -> None:
"""批量写入 L2/L3 大纲节点 """批量写入 L2/L3 大纲节点
@@ -252,7 +285,12 @@ class QdrantService:
return items, str(next_offset) if next_offset is not None else None return items, str(next_offset) if next_offset is not None else None
async def get_doc_detail(self, doc_id: str) -> dict[str, Any] | None: async def get_doc_detail(self, doc_id: str) -> dict[str, Any] | None:
"""获取文档详情:L1 记录 + L2/L3 全部节点 + chunks 数量,文档不存在返回 None""" """获取文档详情:L1 记录 + L2/L3 全部节点 + chunks 数量 + file 文件信息
file 字段从 L1 payload.metadata 提取(raw_file_path/original_filename/original_size_bytes):
有 raw_file_path 时返回 {filename, size_bytes, url},否则 None。
文档不存在返回 None。
"""
doc_filter = self.build_filter(doc_ids=[doc_id]) doc_filter = self.build_filter(doc_ids=[doc_id])
l1_records, _ = await self._client.scroll( l1_records, _ = await self._client.scroll(
collection_name=COLLECTION_L1, collection_name=COLLECTION_L1,
@@ -271,11 +309,23 @@ class QdrantService:
count_filter=doc_filter, count_filter=doc_filter,
exact=True, exact=True,
) )
# 从 L1 metadata 提取原始文件信息,无 raw_file_path 时 file=None
meta = (l1_records[0].payload or {}).get("metadata") or {}
raw_path = meta.get("raw_file_path", "")
file_info: dict[str, Any] | None = None
if raw_path:
file_info = {
"filename": meta.get("original_filename", Path(raw_path).name),
"size_bytes": int(meta.get("original_size_bytes", "0") or "0"),
"url": f"/api/v1/documents/{doc_id}/file",
}
return { return {
"l1": l1_records[0].payload or {}, "l1": l1_records[0].payload or {},
"l2_nodes": l2_nodes, "l2_nodes": l2_nodes,
"l3_nodes": l3_nodes, "l3_nodes": l3_nodes,
"chunks_count": chunks_count.count, "chunks_count": chunks_count.count,
"file": file_info,
} }
async def delete_by_doc_id(self, doc_id: str) -> dict[str, int]: async def delete_by_doc_id(self, doc_id: str) -> dict[str, int]:
+79
View File
@@ -0,0 +1,79 @@
"""语义重排服务(cross-encoder reranker
调用 Ollama 的 `/api/rerank` 端点(Qwen3-Reranker 等重排模型),对检索召回的
chunk 候选池做精排,提升最终 Top-K 的相关性。相比纯向量/RRF 融合,cross-encoder
以 (query, doc) 联合编码,能捕捉字词不匹配的语义关联,典型带来约 10% 的精度提升。
工厂 `create_reranker_service()` 按 `settings.reranker_enabled` 返回实例或 None
关闭时上层检索链路退化为原有的 RRF 融合结果,行为完全不变。
"""
from typing import Protocol, runtime_checkable
import httpx
import structlog
from app.config import settings
logger = structlog.get_logger()
@runtime_checkable
class RerankerService(Protocol):
"""重排服务统一接口"""
async def rerank(self, query: str, documents: list[str]) -> list[float]:
"""对候选文档按与 query 的相关性打分
Args:
query: 查询文本
documents: 候选文档文本列表(按输入顺序对齐)
Returns:
与 documents 等长的相关性分数列表(分数越高越相关)
"""
...
class OllamaRerankerService:
"""基于 Ollama /api/rerank 的重排服务"""
def __init__(self, base_url: str, model: str, timeout: float = 30.0) -> None:
self.base_url = base_url.rstrip("/")
self.model = model
self.timeout = timeout
async def rerank(self, query: str, documents: list[str]) -> list[float]:
if not documents:
return []
url = f"{self.base_url}/api/rerank"
payload = {"model": self.model, "query": query, "documents": documents}
async with httpx.AsyncClient(timeout=self.timeout) as client:
resp = await client.post(url, json=payload)
resp.raise_for_status()
data = resp.json()
results = data.get("results", [])
# 按 Ollama 返回的 index 映射 relevance_score;缺失项补 0,保证输出与输入等长
scores: list[float] = [0.0] * len(documents)
for item in results:
idx = item.get("index")
score = float(item.get("relevance_score", 0.0))
if idx is not None and 0 <= idx < len(documents):
scores[idx] = score
logger.debug("重排完成", model=self.model, candidates=len(documents))
return scores
def create_reranker_service() -> RerankerService | None:
"""按 settings.reranker_enabled 创建重排服务实例
关闭时返回 None,上层据此跳过精排、退化为 RRF 融合结果。
"""
if not settings.reranker_enabled:
return None
return OllamaRerankerService(
base_url=settings.ollama_base_url,
model=settings.reranker_model,
timeout=settings.reranker_timeout,
)
+835 -81
View File
File diff suppressed because it is too large Load Diff
Binary file not shown.
+31 -5
View File
@@ -1,6 +1,23 @@
# ============================================================
# QMDSearch — 部署 compose(相对路径,可任意目录部署)
# ------------------------------------------------------------
# 数据库数据 / 配置 / 代码 / 前端 全部从本地目录挂载,
# 日常更新(改代码或前端)只需改文件 + `docker compose restart app`
# 无需重新 build 镜像。
#
# 说明:
# - 相对路径相对于本 compose 文件所在目录(即项目根目录)解析,
# 因此直接把整个项目目录放到 NAS(或任意主机)即可部署。
# - app 服务复用已构建的 qmdsearch-app 镜像作为「带依赖的运行时」
# (Python 依赖装在镜像内,代码与前端构建产物走挂载)。
# - 仅当 Python 依赖(pyproject.toml)变化时,才需 `docker build -t qmdsearch-app .` 一次。
# - 数据目录默认 ./data,可通过 .env 的 NAS_DATA_DIR 覆盖到其他磁盘。
# ============================================================
services: services:
app: app:
build: . # 不再 build 项目代码,改用已含依赖的运行时镜像;代码与前端均走下方挂载
image: qmdsearch-app:latest
container_name: qmdsearch-app container_name: qmdsearch-app
restart: unless-stopped restart: unless-stopped
ports: ports:
@@ -13,7 +30,7 @@ services:
# 针对 16 线程 / 61GB 内存的 NAS 调优:放宽入库并发 # 针对 16 线程 / 61GB 内存的 NAS 调优:放宽入库并发
- INGEST_MAX_CONCURRENCY=${INGEST_MAX_CONCURRENCY:-4} - INGEST_MAX_CONCURRENCY=${INGEST_MAX_CONCURRENCY:-4}
env_file: env_file:
- .env - ./.env
depends_on: depends_on:
qdrant: qdrant:
condition: service_started condition: service_started
@@ -22,6 +39,14 @@ services:
ollama: ollama:
condition: service_started condition: service_started
volumes: volumes:
# ---- 代码(本地,改后 restart 即生效,免 build----
- ./app:/app/app
- ./scripts:/app/scripts
# ---- 前端构建产物(本地 frontend/dist,改前端后 build 到此目录 + restart----
- ./frontend/dist:/app/app/static/admin
# ---- 配置(显式文件映射,只读)----
- ./.env:/app/.env:ro
# ---- 数据 / 日志(持久化)----
- ${NAS_DATA_DIR:-./data}/logs:/app/logs - ${NAS_DATA_DIR:-./data}/logs:/app/logs
- ${NAS_DATA_DIR:-./data}/uploads:/app/uploads - ${NAS_DATA_DIR:-./data}/uploads:/app/uploads
networks: networks:
@@ -68,10 +93,10 @@ services:
# 针对 Ryzen 9 7940HS16 线程)的 CPU 推理调优: # 针对 Ryzen 9 7940HS16 线程)的 CPU 推理调优:
# 并行推理任务数、常驻模型数、单请求线程上限、KV 缓存量化以省内存 # 并行推理任务数、常驻模型数、单请求线程上限、KV 缓存量化以省内存
- OLLAMA_NUM_PARALLEL=4 - OLLAMA_NUM_PARALLEL=4
- OLLAMA_MAX_LOADED_MODELS=2 - OLLAMA_MAX_LOADED_MODELS=3
- OLLAMA_NUM_THREADS=16 - OLLAMA_NUM_THREADS=16
- OLLAMA_KV_CACHE_TYPE=q8_0 - OLLAMA_KV_CACHE_TYPE=q8_0
# 首次启动自动拉取所需模型(qwen2.5:1.5b 总结 + bge-m3 嵌入), # 首次启动自动拉取所需模型(qwen3:1.7b 总结 + bge-m3 嵌入 + qwen3-reranker 重排),
# 下载完成后转交常驻 ollama serve。已存在时仅做健康检查。 # 下载完成后转交常驻 ollama serve。已存在时仅做健康检查。
entrypoint: /bin/bash entrypoint: /bin/bash
command: command:
@@ -80,8 +105,9 @@ services:
ollama serve & ollama serve &
SERVE_PID=$$! SERVE_PID=$$!
sleep 6 sleep 6
ollama pull qwen2.5:1.5b ollama pull qwen3:1.7b
ollama pull bge-m3 ollama pull bge-m3
ollama pull qwen3-reranker:0.6b
wait $$SERVE_PID wait $$SERVE_PID
networks: networks:
- qmdsearch - qmdsearch
+58
View File
@@ -17,3 +17,61 @@ export function login(username, password) {
export function me() { export function me() {
return http.get('/api/v1/auth/me') return http.get('/api/v1/auth/me')
} }
/**
* 列出全部用户(admin 专用)
* @returns {Promise<Array<{username:string, role:string, enabled:boolean, must_change_password:boolean, created_at:string}>>}
*/
export function listUsers() {
return http.get('/api/v1/auth/users')
}
/**
* 创建用户(admin 专用)
* @param {string} username
* @param {string} password
* @param {'admin'|'user'} [role='user']
*/
export function createUser(username, password, role = 'user') {
return http.post('/api/v1/auth/users', { username, password, role })
}
/**
* 更新用户角色 / 启用状态(admin 专用,至少传一项)
* @param {string} username
* @param {{ role?: 'admin'|'user', enabled?: boolean }} payload
*/
export function updateUser(username, payload) {
return http.patch(`/api/v1/auth/users/${encodeURIComponent(username)}`, payload)
}
/**
* 重置指定用户密码(admin 专用)
* @param {string} username
* @param {string} newPassword
*/
export function resetPassword(username, newPassword) {
return http.post(`/api/v1/auth/users/${encodeURIComponent(username)}/password`, {
new_password: newPassword
})
}
/**
* 删除用户(admin 专用)
* @param {string} username
*/
export function deleteUser(username) {
return http.delete(`/api/v1/auth/users/${encodeURIComponent(username)}`)
}
/**
* 修改自己的密码(当前登录用户)
* @param {string} oldPassword
* @param {string} newPassword
*/
export function changeMyPassword(oldPassword, newPassword) {
return http.post('/api/v1/auth/password', {
old_password: oldPassword,
new_password: newPassword
})
}
+49
View File
@@ -52,6 +52,55 @@ export function upload(formData) {
}) })
} }
/**
* multipart 批量文件上传入库(异步,逐文件创建任务)
* @param {File[]} files
* @returns {Promise<{tasks: Array<{filename:string, task_id:string}>, failed: Array<{filename:string, error:string}>}>}
*/
export function uploadBatch(files) {
const formData = new FormData()
files.forEach((f) => formData.append('files', f))
return http.post('/api/v1/documents/upload-batch', formData, {
headers: { 'Content-Type': 'multipart/form-data' }
})
}
/**
* 重新摘要入库(读原文件→删旧数据→新建任务)
* @param {string} docId
* @returns {Promise<{task_id:string, status:string}>}
*/
export function reingest(docId) {
return http.post(`/api/v1/documents/${encodeURIComponent(docId)}/reingest`)
}
/**
* 列出近期入库任务(按 updated_at 降序)
* @param {number} [limit=20]
* @returns {Promise<{items: Array, total: number}>}
*/
export function taskList(limit = 20) {
return http.get('/api/v1/documents/tasks', { params: { limit } })
}
/**
* 重试入库任务(用原 source 重新提交新任务)
* @param {string} taskId
* @returns {Promise<{task_id:string, status:string}>}
*/
export function retryTask(taskId) {
return http.post(`/api/v1/documents/tasks/${encodeURIComponent(taskId)}/retry`)
}
/**
* 删除入库任务
* @param {string} taskId
* @returns {Promise<{task_id:string, deleted:boolean}>}
*/
export function deleteTask(taskId) {
return http.delete(`/api/v1/documents/tasks/${encodeURIComponent(taskId)}`)
}
/** /**
* 查询入库任务状态 * 查询入库任务状态
* @param {string} taskId * @param {string} taskId
@@ -0,0 +1,21 @@
import { reactive } from 'vue'
/**
* 入库进度右侧 Drawer 的全局共享开关状态
*
* 该 Drawer 渲染在 MainLayout 中,但可由任意子页面(如 Library)触发打开,
* 因此通过单例 reactive 状态 + open()/close() 方法解耦,避免多层 prop 传递。
*/
const state = reactive({ open: false })
export function useTasksDrawer() {
return {
state,
open() {
state.open = true
},
close() {
state.open = false
}
}
}
+142 -14
View File
@@ -1,26 +1,31 @@
<script setup> <script setup>
import { computed, h, ref } from 'vue' import { computed, h, reactive, ref, watch } from 'vue'
import { useRoute, useRouter, RouterView } from 'vue-router' import { useRoute, useRouter, RouterView } from 'vue-router'
import { storeToRefs } from 'pinia' import { storeToRefs } from 'pinia'
import { Modal } from 'ant-design-vue' import { Modal, message } from 'ant-design-vue'
import { import {
DashboardOutlined, DashboardOutlined,
FileTextOutlined,
UploadOutlined,
SearchOutlined,
AppstoreOutlined, AppstoreOutlined,
SettingOutlined, SettingOutlined,
MenuFoldOutlined, MenuFoldOutlined,
MenuUnfoldOutlined, MenuUnfoldOutlined,
LogoutOutlined, LogoutOutlined,
DatabaseOutlined DatabaseOutlined,
ApiOutlined,
TeamOutlined,
ClockCircleOutlined
} from '@ant-design/icons-vue' } from '@ant-design/icons-vue'
import { useAuthStore } from '@/stores/useAuthStore' import { useAuthStore } from '@/stores/useAuthStore'
import { changeMyPassword } from '@/api/auth'
import { callApi } from '@/api/client'
import TasksDrawer from '@/views/TasksDrawer.vue'
import { useTasksDrawer } from '@/composables/useTasksDrawer'
const route = useRoute() const route = useRoute()
const router = useRouter() const router = useRouter()
const authStore = useAuthStore() const authStore = useAuthStore()
const { user, displayName } = storeToRefs(authStore) const { user, displayName } = storeToRefs(authStore)
const tasksDrawer = useTasksDrawer()
const collapsed = ref(false) const collapsed = ref(false)
@@ -30,16 +35,25 @@ const selectedKeys = computed(() => {
const openKeys = ref(['main']) const openKeys = ref(['main'])
const menuItems = [ const menuItems = computed(() => {
{ key: 'overview', icon: () => h(DashboardOutlined), label: '概览' }, const items = [
{ key: 'documents', icon: () => h(FileTextOutlined), label: '文档管理' }, { key: 'overview', icon: () => h(DashboardOutlined), label: '概览与检索' },
{ key: 'ingest', icon: () => h(UploadOutlined), label: '文档入库' }, { key: 'library', icon: () => h(AppstoreOutlined), label: '知识库' },
{ key: 'search', icon: () => h(SearchOutlined), label: '检索测试台' }, { key: 'tasks', icon: () => h(ClockCircleOutlined), label: '入库进度' },
{ key: 'categories', icon: () => h(AppstoreOutlined), label: '类目列表' }, { key: 'settings', icon: () => h(SettingOutlined), label: '设置' },
{ key: 'settings', icon: () => h(SettingOutlined), label: '设置' } { key: 'api-docs', icon: () => h(ApiOutlined), label: 'API 说明' }
] ]
if (authStore.isAdmin) {
items.push({ key: 'users', icon: () => h(TeamOutlined), label: '用户管理' })
}
return items
})
function handleMenuClick({ key }) { function handleMenuClick({ key }) {
if (key === 'tasks') {
tasksDrawer.open()
return
}
if (key && key !== route.name) { if (key && key !== route.name) {
router.push({ name: key }) router.push({ name: key })
} }
@@ -57,6 +71,68 @@ function handleLogout() {
} }
}) })
} }
/* ---------- 强制改密弹窗(must_change_password 用户) ---------- */
const passwordModalVisible = ref(false)
const passwordSubmitting = ref(false)
const passwordFormRef = ref(null)
const passwordForm = reactive({
old_password: '',
new_password: '',
confirm_password: ''
})
const passwordRules = {
old_password: [{ required: true, message: '请输入当前密码', trigger: 'blur' }],
new_password: [
{ required: true, message: '请输入新密码', trigger: 'blur' },
{ min: 8, message: '密码至少 8 位', trigger: 'blur' }
],
confirm_password: [
{ required: true, message: '请确认新密码', trigger: 'blur' },
{
validator: (_rule, value) =>
value === passwordForm.new_password
? Promise.resolve()
: Promise.reject(new Error('两次输入的密码不一致')),
trigger: 'blur'
}
]
}
// must_change_password 为 true 时自动弹出改密弹窗(不可关闭)
watch(
() => user.value?.must_change_password,
(val) => {
passwordModalVisible.value = val === true
},
{ immediate: true }
)
async function handleSubmitPassword() {
try {
await passwordFormRef.value.validate()
} catch {
return
}
passwordSubmitting.value = true
try {
await callApi(
() => changeMyPassword(passwordForm.old_password, passwordForm.new_password),
{ errorText: '修改密码失败' }
)
authStore.markPasswordChanged()
message.success('密码修改成功')
passwordModalVisible.value = false
passwordForm.old_password = ''
passwordForm.new_password = ''
passwordForm.confirm_password = ''
} catch {
// 错误已由 callApi 弹出
} finally {
passwordSubmitting.value = false
}
}
</script> </script>
<template> <template>
@@ -118,6 +194,58 @@ function handleLogout() {
QMDSearch Admin · 当前用户{{ user?.username || '-' }} QMDSearch Admin · 当前用户{{ user?.username || '-' }}
</a-layout-footer> </a-layout-footer>
</a-layout> </a-layout>
<!-- 入库进度右侧 Drawer -->
<TasksDrawer />
<!-- 强制改密弹窗must_change_password 用户不可关闭 -->
<a-modal
v-model:open="passwordModalVisible"
:mask-closable="false"
:closable="false"
:keyboard="false"
:destroy-on-close="true"
title="首次登录须修改密码"
:confirm-loading="passwordSubmitting"
ok-text="确认修改"
:cancel-button-props="{ style: { display: 'none' } }"
@ok="handleSubmitPassword"
>
<a-alert
type="warning"
show-icon
message="检测到首次登录或管理员重置密码,请立即修改密码后方可使用系统。"
style="margin-bottom: 16px"
/>
<a-form
ref="passwordFormRef"
:model="passwordForm"
:rules="passwordRules"
layout="vertical"
>
<a-form-item label="当前密码" name="old_password">
<a-input-password
v-model:value="passwordForm.old_password"
placeholder="请输入当前密码"
autocomplete="current-password"
/>
</a-form-item>
<a-form-item label="新密码" name="new_password">
<a-input-password
v-model:value="passwordForm.new_password"
placeholder="至少 8 位"
autocomplete="new-password"
/>
</a-form-item>
<a-form-item label="确认新密码" name="confirm_password">
<a-input-password
v-model:value="passwordForm.confirm_password"
placeholder="再次输入新密码"
autocomplete="new-password"
/>
</a-form-item>
</a-form>
</a-modal>
</a-layout> </a-layout>
</template> </template>
+21 -23
View File
@@ -18,37 +18,31 @@ const routes = [
path: 'overview', path: 'overview',
name: 'overview', name: 'overview',
component: () => import('@/views/Overview.vue'), component: () => import('@/views/Overview.vue'),
meta: { title: '概览', requiresAuth: true } meta: { title: '概览与检索', requiresAuth: true }
}, },
{ {
path: 'documents', path: 'library',
name: 'documents', name: 'library',
component: () => import('@/views/Documents.vue'), component: () => import('@/views/Library.vue'),
meta: { title: '文档管理', requiresAuth: true } meta: { title: '知识库', requiresAuth: true }
},
{
path: 'ingest',
name: 'ingest',
component: () => import('@/views/Ingest.vue'),
meta: { title: '文档入库', requiresAuth: true }
},
{
path: 'search',
name: 'search',
component: () => import('@/views/Search.vue'),
meta: { title: '检索测试台', requiresAuth: true }
},
{
path: 'categories',
name: 'categories',
component: () => import('@/views/Categories.vue'),
meta: { title: '类目列表', requiresAuth: true }
}, },
{ {
path: 'settings', path: 'settings',
name: 'settings', name: 'settings',
component: () => import('@/views/Settings.vue'), component: () => import('@/views/Settings.vue'),
meta: { title: '设置', requiresAuth: true } meta: { title: '设置', requiresAuth: true }
},
{
path: 'api-docs',
name: 'api-docs',
component: () => import('@/views/ApiDocs.vue'),
meta: { title: 'API 说明', requiresAuth: true }
},
{
path: 'users',
name: 'users',
component: () => import('@/views/Users.vue'),
meta: { title: '用户管理', requiresAuth: true, requiresAdmin: true }
} }
] ]
}, },
@@ -81,6 +75,10 @@ router.beforeEach((to) => {
return { name: 'login', query: { redirect: to.fullPath } } return { name: 'login', query: { redirect: to.fullPath } }
} }
if (to.meta?.requiresAdmin && !authStore.isAdmin) {
return { name: 'overview' }
}
if (to.name === 'login' && authStore.isAuthenticated) { if (to.name === 'login' && authStore.isAuthenticated) {
return { name: 'overview' } return { name: 'overview' }
} }
+26 -5
View File
@@ -39,13 +39,27 @@ export const useAuthStore = defineStore('auth', {
actions: { actions: {
/** /**
* 登录成功后保存 token + user * 登录成功后保存 token + user
* @param {{access_token:string, user:object}} data * 兼容后端两种返回:
* - 新格式 { token, username, role, must_change_password }
* - 旧格式 { access_token, user: { username, role, ... } }
* @param {object} data
*/ */
setAuth(data) { setAuth(data) {
this.token = data.access_token const token = data?.access_token ?? data?.token
this.user = data.user const user =
localStorage.setItem(TOKEN_STORAGE_KEY, data.access_token) data?.user ??
localStorage.setItem(USER_STORAGE_KEY, JSON.stringify(data.user)) (data
? {
username: data.username,
role: data.role,
must_change_password: data.must_change_password ?? false
}
: null)
if (!token || !user) return
this.token = token
this.user = user
localStorage.setItem(TOKEN_STORAGE_KEY, token)
localStorage.setItem(USER_STORAGE_KEY, JSON.stringify(user))
}, },
/** 清除登录态(登出 / 401) */ /** 清除登录态(登出 / 401) */
@@ -56,6 +70,13 @@ export const useAuthStore = defineStore('auth', {
localStorage.removeItem(USER_STORAGE_KEY) localStorage.removeItem(USER_STORAGE_KEY)
}, },
/** 改密成功后更新本地用户状态(清除 must_change_password 标记) */
markPasswordChanged() {
if (!this.user) return
this.user = { ...this.user, must_change_password: false }
localStorage.setItem(USER_STORAGE_KEY, JSON.stringify(this.user))
},
/** /**
* 登出 * 登出
*/ */
+14
View File
@@ -10,6 +10,20 @@ export function truncate(text, maxLen = 80) {
return s.length > maxLen ? `${s.slice(0, maxLen)}` : s return s.length > maxLen ? `${s.slice(0, maxLen)}` : s
} }
/**
* 格式化字节数为人类可读大小
* @param {number|null|undefined} bytes
* @returns {string}
*/
export function formatSize(bytes) {
if (bytes == null || bytes === '') return '-'
const n = Number(bytes)
if (Number.isNaN(n)) return '-'
if (n < 1024) return `${n} B`
if (n < 1024 * 1024) return `${(n / 1024).toFixed(1)} KB`
return `${(n / 1024 / 1024).toFixed(1)} MB`
}
/** /**
* 安全拼接类名 * 安全拼接类名
* @param {...(string | false | null | undefined)} args * @param {...(string | false | null | undefined)} args
+234
View File
@@ -0,0 +1,234 @@
<script setup>
import { computed } from 'vue'
import { ApiOutlined, SafetyCertificateOutlined, CodeOutlined, DownloadOutlined } from '@ant-design/icons-vue'
// 以下内容整理自项目 README.md 的「API 文档」章节
const apiList = [
{ method: 'GET', path: '/api/v1/health', desc: '健康检查', auth: false },
{ method: 'POST', path: '/api/v1/auth/register', desc: '用户注册(可关闭)', auth: false },
{ method: 'POST', path: '/api/v1/auth/login', desc: '用户登录,返回 JWT token', auth: false },
{ method: 'GET', path: '/api/v1/auth/me', desc: '获取当前用户信息', auth: true },
{ method: 'POST', path: '/api/v1/search', desc: '分层检索', auth: true },
{ method: 'POST', path: '/api/v1/documents', desc: '文档入库(JSON 文本,202 异步入库)', auth: true },
{ method: 'POST', path: '/api/v1/documents/upload', desc: '文件上传入库(multipart202 异步)', auth: true },
{ method: 'GET', path: '/api/v1/documents/tasks/{task_id}', desc: '入库任务状态查询', auth: true },
{ method: 'GET', path: '/api/v1/documents', desc: '文档列表(分页)', auth: true },
{ method: 'GET', path: '/api/v1/documents/{doc_id}', desc: '文档详情', auth: true },
{ method: 'DELETE', path: '/api/v1/documents/{doc_id}', desc: '删除文档(幂等)', auth: true },
{ method: 'GET', path: '/api/v1/knowledge/categories', desc: '知识分类类目集', auth: true },
{ method: 'GET', path: '/api/v1/knowledge/stats', desc: '统计(四层点数 + 类目分布)', auth: true },
{ method: 'GET', path: '/admin', desc: '管理后台(本页)', auth: false }
]
const methodColor = {
GET: 'green',
POST: 'blue',
DELETE: 'red',
PUT: 'orange'
}
const baseUrl = computed(() => `${window.location.origin}/admin/`.replace(/\/admin\/$/, ''))
const agentSkillUrl = computed(() => `${window.location.origin}/agent-skill`)
const loginExample = `curl -X POST ${baseUrl.value}/api/v1/auth/login \\
-H "Content-Type: application/json" \\
-d '{"username": "admin", "password": "your-password"}'`
const searchExample = `curl -X POST ${baseUrl.value}/api/v1/search \\
-H "Content-Type: application/json" \\
-H "Authorization: Bearer <token>" \\
-d '{"query": "如何配置 Redis 缓存", "top_k": 5}'`
const uploadExample = `curl -X POST ${baseUrl.value}/api/v1/documents/upload \\
-H "Authorization: Bearer <token>" \\
-F "file=@document.pdf"`
</script>
<template>
<div class="apidocs page-section">
<div class="apidocs__header">
<h2 class="page-title">
<ApiOutlined /> API 使用说明
</h2>
<span class="text-muted">接口前缀{{ baseUrl }}/api/v1</span>
</div>
<a-alert
class="apidocs__tip"
type="info"
show-icon
message="需要鉴权的接口请在请求头携带 Authorization: Bearer &lt;token&gt;token 通过 /auth/login 获取。"
/>
<!-- AI Agent Skill 下载 -->
<section class="apidocs__block apidocs__skill">
<h3 class="section-subtitle">
<DownloadOutlined /> AI Agent Skill 下载
</h3>
<p class="apidocs__p">
AI Agent 提供的接入包 <code>SKILL.md</code> Python 客户端示例覆盖文档上传文本 / 文件与分层检索
下载后解压到 Agent skills 目录即可启用
</p>
<a :href="agentSkillUrl" target="_blank" rel="noopener">
<a-button type="primary">
<template #icon><DownloadOutlined /></template>
下载 QMDSearch-Agent-Skill.zip
</a-button>
</a>
</section>
<!-- 统一响应格式 -->
<section class="apidocs__block">
<h3 class="section-subtitle">
<CodeOutlined /> 统一响应格式
</h3>
<ul class="apidocs__ul">
<li>所有接口返回统一 JSON 结构<code>code</code> / <code>data</code> / <code>message</code></li>
<li>错误码<code>0</code> 成功<code>1xxx</code> 客户端错误<code>2xxx</code> 服务端错误</li>
</ul>
<pre class="code-block">{
"code": 0,
"data": { ... },
"message": "ok"
}</pre>
</section>
<!-- API 列表 -->
<section class="apidocs__block">
<h3 class="section-subtitle">接口列表</h3>
<a-table
:columns="[
{ title: '方法', dataIndex: 'method', key: 'method', width: 90, align: 'center' },
{ title: '路径', dataIndex: 'path', key: 'path', width: 320, ellipsis: true },
{ title: '说明', dataIndex: 'desc', key: 'desc' },
{ title: '认证', dataIndex: 'auth', key: 'auth', width: 80, align: 'center' }
]"
:data-source="apiList"
:pagination="false"
row-key="path"
size="middle"
>
<template #bodyCell="{ column, text, record }">
<template v-if="column.key === 'method'">
<a-tag :color="methodColor[record.method]">{{ record.method }}</a-tag>
</template>
<template v-else-if="column.key === 'path'">
<code class="api-path">{{ text }}</code>
</template>
<template v-else-if="column.key === 'auth'">
<a-tag :color="record.auth ? 'volcano' : 'default'">
{{ record.auth ? '需鉴权' : '否' }}
</a-tag>
</template>
</template>
</a-table>
</section>
<!-- 认证说明 -->
<section class="apidocs__block">
<h3 class="section-subtitle">
<SafetyCertificateOutlined /> 获取 Token
</h3>
<p class="apidocs__p">先调用登录接口获取 JWT再在后续请求的 Header 中携带</p>
<pre class="code-block">{{ loginExample }}</pre>
</section>
<!-- 检索示例 -->
<section class="apidocs__block">
<h3 class="section-subtitle">
<CodeOutlined /> 检索示例
</h3>
<pre class="code-block">{{ searchExample }}</pre>
</section>
<!-- 文件上传示例 -->
<section class="apidocs__block">
<h3 class="section-subtitle">
<CodeOutlined /> 文件上传示例
</h3>
<pre class="code-block">{{ uploadExample }}</pre>
</section>
</div>
</template>
<style scoped>
.apidocs__header {
display: flex;
align-items: baseline;
justify-content: space-between;
flex-wrap: wrap;
gap: 8px;
margin-bottom: 16px;
}
.page-title :deep(.anticon) {
margin-right: 8px;
color: #1677ff;
}
.apidocs__tip {
margin-bottom: 20px;
}
.apidocs__skill {
padding: 16px 18px;
border: 1px solid rgba(22, 119, 255, 0.25);
border-radius: 10px;
background: rgba(22, 119, 255, 0.04);
}
.apidocs__block {
margin-bottom: 24px;
}
.section-subtitle :deep(.anticon) {
margin-right: 6px;
color: #1677ff;
}
.apidocs__ul {
margin: 0 0 12px;
padding-left: 20px;
color: #4b5563;
font-size: 13px;
line-height: 1.9;
}
.apidocs__ul code,
.api-path {
background: #f3f4f6;
border: 1px solid #e5e7eb;
border-radius: 4px;
padding: 1px 6px;
font-family: 'SFMono-Regular', Consolas, 'Liberation Mono', Menlo, monospace;
font-size: 12px;
color: #c0341d;
}
.api-path {
white-space: nowrap;
color: #1677ff;
background: rgba(22, 119, 255, 0.08);
border-color: rgba(22, 119, 255, 0.2);
}
.apidocs__p {
color: #4b5563;
font-size: 13px;
margin: 0 0 10px;
}
.code-block {
background: #0f172a;
color: #e2e8f0;
border-radius: 8px;
padding: 16px 18px;
overflow-x: auto;
font-family: 'SFMono-Regular', Consolas, 'Liberation Mono', Menlo, monospace;
font-size: 12.5px;
line-height: 1.7;
margin: 0;
white-space: pre;
}
</style>
-70
View File
@@ -1,70 +0,0 @@
<script setup>
import { onMounted, ref } from 'vue'
import { message } from 'ant-design-vue'
import { categories as fetchCategories } from '@/api/knowledge'
const isLoading = ref(false)
const categoriesData = ref([])
const columns = [
{ title: '名称', dataIndex: 'name', key: 'name' },
{ title: '描述', dataIndex: 'description', key: 'description' }
]
async function loadCategories() {
isLoading.value = true
try {
const data = await fetchCategories()
categoriesData.value = data.categories || []
} catch (err) {
message.error(err?.message || '加载类目列表失败')
} finally {
isLoading.value = false
}
}
onMounted(() => {
loadCategories()
})
</script>
<template>
<div class="categories page-section">
<div class="categories__header">
<h2 class="categories__title">类目列表</h2>
<a-button :loading="isLoading" @click="loadCategories">刷新</a-button>
</div>
<a-table
:columns="columns"
:data-source="categoriesData"
:pagination="false"
:loading="isLoading"
row-key="name"
size="small"
>
<template #bodyCell="{ column, record }">
<template v-if="column.key === 'name'">
<a-tag color="blue">{{ record.name }}</a-tag>
</template>
<template v-else-if="column.key === 'description'">
<span>{{ record.description || '-' }}</span>
</template>
</template>
</a-table>
</div>
</template>
<style scoped>
.categories__header {
display: flex;
align-items: center;
justify-content: space-between;
margin-bottom: 12px;
}
.categories__title {
font-size: 16px;
margin: 0;
}
</style>
-299
View File
@@ -1,299 +0,0 @@
<script setup>
import { onMounted, reactive, ref } from 'vue'
import { message, Modal } from 'ant-design-vue'
import {
list as fetchDocuments,
detail as fetchDocumentDetail,
remove as deleteDocument
} from '@/api/documents'
import { truncate } from '@/utils/format'
const isLoading = ref(false)
const isLoadingDetail = ref(false)
const isDeleting = ref(false)
const documents = ref([])
const nextOffset = ref(null)
const detailVisible = ref(false)
const detailData = ref(null)
const statusText = reactive({
text: ''
})
function updateStatusText() {
statusText.text =
nextOffset.value === null ? '已加载全部' : '还有更多,可继续加载'
}
async function loadDocuments(reset = false) {
isLoading.value = true
try {
const offset = reset ? null : nextOffset.value
const data = await fetchDocuments(20, offset)
if (reset) {
documents.value = data.items || []
} else {
documents.value = documents.value.concat(data.items || [])
}
nextOffset.value = data.next_offset ?? null
updateStatusText()
} catch (err) {
message.error(err?.message || '加载文档列表失败')
} finally {
isLoading.value = false
}
}
async function handleLoadMore() {
await loadDocuments(false)
}
function handleViewDetail(docId) {
detailVisible.value = true
detailData.value = null
loadDocDetail(docId)
}
async function loadDocDetail(docId) {
isLoadingDetail.value = true
try {
detailData.value = await fetchDocumentDetail(docId)
} catch (err) {
message.error(err?.message || '加载文档详情失败')
} finally {
isLoadingDetail.value = false
}
}
function handleDelete(doc) {
const title = doc.title || doc.doc_id
Modal.confirm({
title: '确认删除',
content: `确定删除文档「${title}」(${doc.doc_id}) 吗?该操作将删除四层集合中的全部数据,不可恢复。`,
okText: '删除',
okType: 'danger',
cancelText: '取消',
async onOk() {
isDeleting.value = true
try {
const data = await deleteDocument(doc.doc_id)
message.success(`已删除 ${data.deleted_total ?? 0} 条数据`)
detailVisible.value = false
await loadDocuments(true)
} catch (err) {
message.error(err?.message || '删除文档失败')
} finally {
isDeleting.value = false
}
}
})
}
function handleCloseDetail() {
detailVisible.value = false
detailData.value = null
}
const columns = [
{ title: '标题', dataIndex: 'title', key: 'title', ellipsis: true },
{ title: '类目', dataIndex: 'category', key: 'category', width: 140 },
{ title: '标签', dataIndex: 'tags', key: 'tags', width: 220 },
{ title: 'L1 摘要', dataIndex: 'summary', key: 'summary', ellipsis: true },
{ title: '操作', key: 'action', width: 160, fixed: 'right' }
]
function getDocTitle(record) {
return record?.title || '(无标题)'
}
onMounted(() => {
loadDocuments(true)
})
</script>
<template>
<div class="documents page-section">
<div class="documents__header">
<h2 class="documents__title">文档管理</h2>
</div>
<a-table
:columns="columns"
:data-source="documents"
:pagination="false"
:loading="isLoading"
row-key="doc_id"
size="small"
:scroll="{ x: 900 }"
>
<template #bodyCell="{ column, record }">
<template v-if="column.key === 'title'">
<span :title="record.title">{{ getDocTitle(record) }}</span>
</template>
<template v-else-if="column.key === 'category'">
<a-tag v-if="record.category" color="blue">{{ record.category }}</a-tag>
<span v-else class="text-muted">-</span>
</template>
<template v-else-if="column.key === 'tags'">
<template v-if="record.tags && record.tags.length">
<a-tag v-for="tag in record.tags" :key="tag">{{ tag }}</a-tag>
</template>
<span v-else class="text-muted">-</span>
</template>
<template v-else-if="column.key === 'summary'">
<span :title="record.summary">{{ truncate(record.summary, 80) }}</span>
</template>
<template v-else-if="column.key === 'action'">
<a-space>
<a-button type="link" size="small" @click="handleViewDetail(record.doc_id)">
详情
</a-button>
<a-button
type="link"
size="small"
danger
:loading="isDeleting"
@click="handleDelete(record)"
>
删除
</a-button>
</a-space>
</template>
</template>
</a-table>
<div class="toolbar documents__toolbar">
<a-button
:loading="isLoading"
:disabled="nextOffset === null"
@click="handleLoadMore"
>
加载更多
</a-button>
<span class="text-muted" style="margin-left: 12px">{{ statusText.text }}</span>
</div>
<a-drawer
:open="detailVisible"
title="文档详情"
placement="right"
width="640"
:destroy-on-close="true"
@close="handleCloseDetail"
>
<a-spin :spinning="isLoadingDetail">
<div v-if="detailData">
<a-descriptions :column="1" size="small" bordered>
<a-descriptions-item label="doc_id">
{{ detailData.l1?.doc_id || '-' }}
</a-descriptions-item>
<a-descriptions-item label="标题">
{{ detailData.l1?.title || '-' }}
</a-descriptions-item>
<a-descriptions-item label="类目">
{{ detailData.l1?.category || '-' }}
</a-descriptions-item>
<a-descriptions-item label="标签">
<template v-if="detailData.l1?.tags && detailData.l1.tags.length">
<a-tag v-for="tag in detailData.l1.tags" :key="tag">{{ tag }}</a-tag>
</template>
<span v-else class="text-muted">-</span>
</a-descriptions-item>
<a-descriptions-item label="chunks_count">
{{ detailData.chunks_count ?? 0 }}
</a-descriptions-item>
</a-descriptions>
<h3 class="documents__section-title">L1 全文</h3>
<pre class="documents__pre">{{ detailData.l1?.text || '' }}</pre>
<h3 class="documents__section-title">
L2 节点{{ (detailData.l2_nodes || []).length }}
</h3>
<div v-if="(detailData.l2_nodes || []).length === 0" class="text-muted"></div>
<div
v-for="(node, idx) in detailData.l2_nodes || []"
:key="`l2-${idx}`"
class="documents__node"
>
<div class="documents__node-path">{{ node.section_path || '' }}</div>
<div class="documents__node-text text-break">{{ node.text || '' }}</div>
</div>
<h3 class="documents__section-title">
L3 节点{{ (detailData.l3_nodes || []).length }}
</h3>
<div v-if="(detailData.l3_nodes || []).length === 0" class="text-muted"></div>
<div
v-for="(node, idx) in detailData.l3_nodes || []"
:key="`l3-${idx}`"
class="documents__node"
>
<div class="documents__node-path">{{ node.section_path || '' }}</div>
<div class="documents__node-text text-break">{{ node.text || '' }}</div>
</div>
</div>
<div v-else-if="!isLoadingDetail" class="text-muted">暂无数据</div>
</a-spin>
</a-drawer>
</div>
</template>
<style scoped>
.documents__header {
display: flex;
align-items: center;
justify-content: space-between;
margin-bottom: 12px;
}
.documents__title {
font-size: 16px;
margin: 0;
}
.documents__toolbar {
display: flex;
align-items: center;
}
.documents__section-title {
font-size: 14px;
margin: 16px 0 8px;
color: #374151;
}
.documents__pre {
background: #f9fafb;
border: 1px solid #e5e7eb;
border-radius: 4px;
padding: 8px;
white-space: pre-wrap;
word-break: break-word;
font-size: 12px;
max-height: 260px;
overflow: auto;
margin: 0;
}
.documents__node {
border-left: 3px solid #93c5fd;
padding: 6px 10px;
margin-bottom: 6px;
background: #f8fafc;
border-radius: 0 4px 4px 0;
}
.documents__node-path {
font-size: 12px;
color: #6b7280;
margin-bottom: 4px;
}
.documents__node-text {
font-size: 13px;
color: #1f2937;
}
</style>
-352
View File
@@ -1,352 +0,0 @@
<script setup>
import { computed, reactive, ref } from 'vue'
import { message } from 'ant-design-vue'
import { ingest as ingestDocument, upload as uploadDocument } from '@/api/documents'
import { useIngestPolling } from '@/composables/useIngestPolling'
import {
INGEST_STATUS_TEXT,
INGEST_STATUS_COLOR
} from '@/constants/ingest'
const { state: pollState, startPolling, stopPolling } = useIngestPolling()
const activeTab = ref('text')
const isSubmittingText = ref(false)
const isSubmittingFile = ref(false)
const textFormRef = ref(null)
const textForm = reactive({
title: '',
source: '',
text: ''
})
const textRules = {
title: [{ required: true, message: '请输入标题', trigger: 'blur' }],
text: [{ required: true, message: '请输入正文', trigger: 'blur' }]
}
const fileForm = reactive({
title: '',
source: ''
})
const fileList = ref([])
const rawFile = ref(null)
const ACCEPTED_EXTENSIONS = '.txt,.md,.html,.htm,.pdf,.docx'
const statusBadgeColor = computed(() => {
return INGEST_STATUS_COLOR[pollState.status] || 'default'
})
const statusBadgeText = computed(() => {
return INGEST_STATUS_TEXT[pollState.status] || pollState.status || '-'
})
const isTerminal = computed(() =>
['done', 'failed'].includes(pollState.status)
)
const resultData = computed(() => pollState.task?.result || null)
const errorData = computed(() => pollState.task?.error || pollState.error || null)
const summaryData = computed(() => resultData.value?.summary || {})
const partialSummary = computed(() => errorData.value?.partial_summary || null)
function handleFileChange(file) {
// a-upload before-upload 返回 false 表示不自动上传;保存原始 File 供 FormData 使用
rawFile.value = file
fileList.value = [file]
return false
}
function handleFileRemove() {
rawFile.value = null
fileList.value = []
}
async function handleSubmitText() {
try {
await textFormRef.value.validate()
} catch {
return
}
isSubmittingText.value = true
try {
const payload = {
title: textForm.title,
text: textForm.text
}
if (textForm.source) {
payload.source = textForm.source
}
const data = await ingestDocument(payload)
message.success('任务已提交')
startPolling(data.task_id)
} catch (err) {
message.error(err?.message || '提交入库失败')
} finally {
isSubmittingText.value = false
}
}
async function handleSubmitFile() {
if (!rawFile.value) {
message.warning('请选择文件')
return
}
isSubmittingFile.value = true
try {
const formData = new FormData()
formData.append('file', rawFile.value)
if (fileForm.title) {
formData.append('title', fileForm.title)
}
if (fileForm.source) {
formData.append('source', fileForm.source)
}
const data = await uploadDocument(formData)
message.success('任务已提交')
startPolling(data.task_id)
} catch (err) {
message.error(err?.message || '上传入库失败')
} finally {
isSubmittingFile.value = false
}
}
function handleCancelPolling() {
stopPolling()
message.info('已停止轮询')
}
</script>
<template>
<div class="ingest page-section">
<div class="ingest__header">
<h2 class="ingest__title">文档入库</h2>
</div>
<a-tabs v-model:activeKey="activeTab">
<a-tab-pane key="text" tab="文本入库">
<a-form
ref="textFormRef"
:model="textForm"
:rules="textRules"
layout="vertical"
>
<a-form-item label="标题" name="title">
<a-input v-model:value="textForm.title" placeholder="请输入标题" allow-clear />
</a-form-item>
<a-form-item label="来源" name="source">
<a-input
v-model:value="textForm.source"
placeholder="例如:manual / web / file"
allow-clear
/>
</a-form-item>
<a-form-item label="正文" name="text">
<a-textarea
v-model:value="textForm.text"
placeholder="请输入文档正文"
:auto-size="{ minRows: 8, maxRows: 18 }"
/>
</a-form-item>
<a-form-item>
<a-button
type="primary"
:loading="isSubmittingText || pollState.isPolling"
@click="handleSubmitText"
>
提交入库
</a-button>
</a-form-item>
</a-form>
</a-tab-pane>
<a-tab-pane key="file" tab="文件上传">
<a-form
:model="fileForm"
layout="vertical"
>
<a-form-item label="选择文件">
<a-upload
:file-list="fileList"
:accept="ACCEPTED_EXTENSIONS"
:max-count="1"
:before-upload="handleFileChange"
@remove="handleFileRemove"
>
<a-button :disabled="fileList.length >= 1">选择文件</a-button>
</a-upload>
<div class="text-muted" style="margin-top: 4px">
支持 .txt/.md/.html/.htm/.pdf/.docx
</div>
</a-form-item>
<a-form-item label="标题(可选,默认取文件名)" name="title">
<a-input
v-model:value="fileForm.title"
placeholder="留空则使用文件名去扩展"
allow-clear
/>
</a-form-item>
<a-form-item label="来源(可选,默认 file:原文件名)" name="source">
<a-input
v-model:value="fileForm.source"
placeholder="例如:manual / web"
allow-clear
/>
</a-form-item>
<a-form-item>
<a-button
type="primary"
:loading="isSubmittingFile || pollState.isPolling"
@click="handleSubmitFile"
>
上传入库
</a-button>
</a-form-item>
</a-form>
</a-tab-pane>
</a-tabs>
<div v-if="pollState.taskId" class="ingest__result">
<div class="ingest__result-header">
<span>任务已提交{{ pollState.taskId }}</span>
<a-tag :color="statusBadgeColor">{{ statusBadgeText }}</a-tag>
<a-button
v-if="pollState.isPolling"
type="link"
size="small"
@click="handleCancelPolling"
>
停止轮询
</a-button>
</div>
<a-alert
v-if="pollState.isTimeout"
class="ingest__alert"
type="warning"
show-icon
:message="`任务仍在进行,可稍后凭 task_id 查询:${pollState.taskId}`"
/>
<a-alert
v-if="pollState.error && !pollState.task"
class="ingest__alert"
type="error"
show-icon
:message="pollState.error?.message || '轮询失败'"
/>
<div v-if="isTerminal && pollState.status === 'done' && resultData" class="ingest__done">
<a-descriptions :column="1" size="small" bordered>
<a-descriptions-item label="document_id">
{{ resultData.document_id || '-' }}
</a-descriptions-item>
<a-descriptions-item label="类目">
{{ resultData.category || '-' }}置信度 {{ resultData.category_confidence ?? '-' }}
</a-descriptions-item>
<a-descriptions-item label="标签">
<template v-if="resultData.tags && resultData.tags.length">
<a-tag v-for="tag in resultData.tags" :key="tag">{{ tag }}</a-tag>
</template>
<span v-else class="text-muted">-</span>
</a-descriptions-item>
<a-descriptions-item label="总结层级">
{{ summaryData.level ?? '-' }}
</a-descriptions-item>
<a-descriptions-item label="写入集合">
{{ resultData.collection || '-' }}
</a-descriptions-item>
<a-descriptions-item label="chunks_count">
{{ resultData.chunks_count ?? 0 }}
</a-descriptions-item>
</a-descriptions>
<h3 class="ingest__section-title">L1 总结</h3>
<pre class="ingest__pre">{{ summaryData.l1_summary || '' }}</pre>
</div>
<div v-if="isTerminal && pollState.status === 'failed'" class="ingest__failed">
<a-descriptions :column="1" size="small" bordered>
<a-descriptions-item label="失败阶段">
{{ errorData?.stage || '-' }}
</a-descriptions-item>
<a-descriptions-item label="错误信息">
<span class="text-break">{{ errorData?.message || '-' }}</span>
</a-descriptions-item>
</a-descriptions>
<template v-if="partialSummary && partialSummary.l1_summary">
<a-alert
class="ingest__alert"
type="info"
show-icon
message="已产出总结保留:任务失败前已生成 L1 摘要"
/>
<h3 class="ingest__section-title">L1 总结</h3>
<pre class="ingest__pre">{{ partialSummary.l1_summary }}</pre>
</template>
</div>
</div>
</div>
</template>
<style scoped>
.ingest__header {
display: flex;
align-items: center;
justify-content: space-between;
margin-bottom: 12px;
}
.ingest__title {
font-size: 16px;
margin: 0;
}
.ingest__result {
margin-top: 16px;
border: 1px solid #e5e7eb;
border-radius: 6px;
background: #fafafa;
padding: 12px 16px;
}
.ingest__result-header {
display: flex;
align-items: center;
gap: 8px;
flex-wrap: wrap;
font-size: 13px;
margin-bottom: 8px;
}
.ingest__alert {
margin: 8px 0;
}
.ingest__done,
.ingest__failed {
margin-top: 8px;
}
.ingest__section-title {
font-size: 14px;
margin: 12px 0 8px;
color: #374151;
}
.ingest__pre {
background: #f9fafb;
border: 1px solid #e5e7eb;
border-radius: 4px;
padding: 8px;
white-space: pre-wrap;
word-break: break-word;
font-size: 12px;
max-height: 260px;
overflow: auto;
margin: 0;
}
</style>
+815
View File
@@ -0,0 +1,815 @@
<script setup>
import { computed, onMounted, reactive, ref } from 'vue'
import { message, Modal } from 'ant-design-vue'
import {
FolderOutlined,
FileTextOutlined,
UploadOutlined,
ReloadOutlined,
SearchOutlined,
ClockCircleOutlined,
TagsOutlined
} from '@ant-design/icons-vue'
import {
categories as fetchCategories,
stats as fetchStats
} from '@/api/knowledge'
import {
list as fetchDocuments,
detail as fetchDocumentDetail,
remove as deleteDocument,
reingest as reingestDocument,
ingest as ingestDocument,
upload as uploadDocument,
uploadBatch
} from '@/api/documents'
import { useIngestPolling } from '@/composables/useIngestPolling'
import { useTasksDrawer } from '@/composables/useTasksDrawer'
import { INGEST_STATUS_TEXT, INGEST_STATUS_COLOR } from '@/constants/ingest'
const tasksDrawer = useTasksDrawer()
/* ---------- 树形目录 ---------- */
const isLoading = ref(false)
const treeData = ref([])
const categoriesData = ref([])
const categoryDesc = ref(null)
const selectedCategory = ref(null)
const allDocs = ref([])
const searchKeyword = ref('')
const UNCATEGORIZED = 'uncategorized'
function formatSize(bytes) {
if (bytes == null) return ''
if (bytes < 1024) return `${bytes} B`
if (bytes < 1024 * 1024) return `${(bytes / 1024).toFixed(1)} KB`
return `${(bytes / 1024 / 1024).toFixed(1)} MB`
}
function buildTree() {
const docs = allDocs.value
const filtered = searchKeyword.value
? docs.filter((d) => {
const kw = searchKeyword.value.toLowerCase()
return (
(d.title || '').toLowerCase().includes(kw) ||
(d.summary || '').toLowerCase().includes(kw) ||
(d.category || '').toLowerCase().includes(kw)
)
})
: docs
// 类目 → 文档映射
const byCat = {}
filtered.forEach((d) => {
const cat = d.category || UNCATEGORIZED
if (!byCat[cat]) byCat[cat] = []
byCat[cat].push(d)
})
const nodes = []
// 定义类目顺序:taxonomy 中定义的类目在前,未定义但存在的类目追加
const defined = categoriesData.value.map((c) => c.name)
const known = new Set(defined)
Object.keys(byCat).forEach((cat) => {
if (!known.has(cat)) known.add(cat)
})
defined.forEach((cat) => {
if (byCat[cat]) {
nodes.push(makeCategoryNode(cat, byCat[cat]))
}
})
// 未在 taxonomy 中但实际存在的类目
Object.keys(byCat).forEach((cat) => {
if (!defined.includes(cat)) {
nodes.push(makeCategoryNode(cat, byCat[cat]))
}
})
return nodes
}
function makeCategoryNode(cat, docs) {
return {
key: `cat:${cat}`,
title: cat,
icon: () => h(FolderOutlined),
isLeaf: docs.length === 0,
category: cat,
count: docs.length,
children: docs.map((d) => ({
key: `doc:${d.doc_id}`,
title: d.title || '(无标题)',
icon: () => h(FileTextOutlined),
isLeaf: true,
doc: d
}))
}
}
function h(type) {
return { render: () => _h(type) }
}
function _h(type) {
return type
}
const treeDataComputed = computed(() => buildTree())
function handleTreeSelect(keys, info) {
const node = info.selectedNodes?.[0]
if (!node) return
const key = node.key
if (key.startsWith('cat:')) {
selectedCategory.value = node.category
categoryDesc.value = categoriesData.value.find((c) => c.name === node.category)?.description || ''
closeDocDetail()
} else if (key.startsWith('doc:')) {
selectedCategory.value = null
categoryDesc.value = null
openDocDetail(node.doc)
}
}
async function loadAll() {
isLoading.value = true
try {
const [cats, stats] = await Promise.all([fetchCategories(), fetchStats()])
categoriesData.value = cats.categories || []
// 加载全部文档(分页直到取完)
const docs = []
let offset = null
for (;;) {
const page = await fetchDocuments(100, offset)
docs.push(...(page.items || []))
offset = page.next_offset ?? null
if (!offset) break
}
allDocs.value = docs
// 更新类目计数(可选)
} catch (err) {
message.error(err?.message || '加载知识库失败')
} finally {
isLoading.value = false
}
}
function handleRefresh() {
loadAll()
}
onMounted(() => {
loadAll()
})
/* ---------- 文档详情 ---------- */
const docDetailVisible = ref(false)
const docDetailData = ref(null)
const isLoadingDetail = ref(false)
const isDeleting = ref(false)
const isReingesting = ref(false)
function openDocDetail(doc) {
docDetailVisible.value = true
docDetailData.value = null
loadDocDetail(doc.doc_id)
}
function closeDocDetail() {
docDetailVisible.value = false
docDetailData.value = null
}
async function loadDocDetail(docId) {
isLoadingDetail.value = true
try {
docDetailData.value = await fetchDocumentDetail(docId)
} catch (err) {
message.error(err?.message || '加载文档详情失败')
} finally {
isLoadingDetail.value = false
}
}
function handleDelete(doc) {
const docId = doc.doc_id
const title = doc.title || docId
Modal.confirm({
title: '确认删除',
content: `确定删除文档「${title}」(${docId}) 吗?该操作将删除四层集合中的全部数据,不可恢复。`,
okText: '删除',
okType: 'danger',
cancelText: '取消',
async onOk() {
isDeleting.value = true
try {
await deleteDocument(docId)
message.success('已删除')
closeDocDetail()
await loadAll()
} catch (err) {
message.error(err?.message || '删除文档失败')
} finally {
isDeleting.value = false
}
}
})
}
function handleReingest(doc) {
const docId = doc.doc_id
const title = doc.title || docId
Modal.confirm({
title: '重新摘要入库',
content: `将重新读取文档「${title}」(${docId}) 的原始文件,删除旧数据后重新生成三级总结并入库。生成新的 doc_id。是否继续?`,
okText: '重新入库',
cancelText: '取消',
async onOk() {
isReingesting.value = true
try {
const data = await reingestDocument(docId)
message.success(`已提交重新入库任务:${data.task_id}`)
} catch (err) {
message.error(err?.message || '重新入库失败')
} finally {
isReingesting.value = false
}
}
})
}
/* ---------- 入库抽屉 ---------- */
const ingestVisible = ref(false)
const activeTab = ref('text')
const { state: pollState, startPolling, stopPolling } = useIngestPolling()
const textFormRef = ref(null)
const textForm = reactive({ title: '', source: '', text: '' })
const textRules = {
title: [{ required: true, message: '请输入标题', trigger: 'blur' }],
text: [{ required: true, message: '请输入正文', trigger: 'blur' }]
}
const isSubmittingText = ref(false)
const isSubmittingFile = ref(false)
const isSubmittingBatch = ref(false)
const fileForm = reactive({ title: '', source: '' })
const fileList = ref([])
const rawFile = ref(null)
const batchResult = ref(null)
const ACCEPTED_EXTENSIONS = '.txt,.md,.html,.htm,.pdf,.docx'
const statusBadgeColor = computed(() => INGEST_STATUS_COLOR[pollState.status] || 'default')
const statusBadgeText = computed(() => INGEST_STATUS_TEXT[pollState.status] || pollState.status || '-')
const isTerminal = computed(() => ['done', 'failed'].includes(pollState.status))
const resultData = computed(() => pollState.task?.result || null)
const errorData = computed(() => pollState.task?.error || pollState.error || null)
const summaryData = computed(() => resultData.value?.summary || {})
const partialSummary = computed(() => errorData.value?.partial_summary || null)
function openIngest() {
ingestVisible.value = true
}
function handleFileChange(file) {
rawFile.value = file
fileList.value = [file]
return false
}
function handleFileRemove() {
rawFile.value = null
fileList.value = []
}
function handleBatchFileChange(_file, fileListArg) {
fileList.value = fileListArg.map((f) => f.originFileObj || f)
batchResult.value = null
return false
}
function handleBatchRemove(file) {
fileList.value = fileList.value.filter((f) => f !== file && f.name !== file.name)
batchResult.value = null
}
function handleBatchClear() {
fileList.value = []
batchResult.value = null
}
async function handleSubmitText() {
try {
await textFormRef.value.validate()
} catch {
return
}
isSubmittingText.value = true
try {
const payload = { title: textForm.title, text: textForm.text }
if (textForm.source) payload.source = textForm.source
const data = await ingestDocument(payload)
message.success('任务已提交')
startPolling(data.task_id)
} catch (err) {
message.error(err?.message || '提交入库失败')
} finally {
isSubmittingText.value = false
}
}
async function handleSubmitFile() {
if (!rawFile.value) {
message.warning('请选择文件')
return
}
isSubmittingFile.value = true
try {
const formData = new FormData()
formData.append('file', rawFile.value)
if (fileForm.title) formData.append('title', fileForm.title)
if (fileForm.source) formData.append('source', fileForm.source)
const data = await uploadDocument(formData)
message.success('任务已提交')
startPolling(data.task_id)
} catch (err) {
message.error(err?.message || '上传入库失败')
} finally {
isSubmittingFile.value = false
}
}
async function handleSubmitBatch() {
if (!fileList.value.length) {
message.warning('请选择文件')
return
}
isSubmittingBatch.value = true
try {
const data = await uploadBatch(fileList.value)
batchResult.value = data
const okCount = data.tasks?.length || 0
const failCount = data.failed?.length || 0
if (okCount > 0) {
message.success(`已提交 ${okCount} 个任务${failCount > 0 ? `${failCount} 个失败` : ''}`)
} else if (failCount > 0) {
message.error(`全部 ${failCount} 个文件上传失败`)
}
if (data.tasks?.length === 1 && failCount === 0) {
startPolling(data.tasks[0].task_id)
}
} catch (err) {
message.error(err?.message || '批量上传失败')
} finally {
isSubmittingBatch.value = false
}
}
function handleCancelPolling() {
stopPolling()
message.info('已停止轮询')
}
const batchColumns = [
{ title: '文件名', dataIndex: 'filename', key: 'filename', ellipsis: true },
{ title: 'task_id', dataIndex: 'task_id', key: 'task_id', width: 240 }
]
const failedColumns = [
{ title: '文件名', dataIndex: 'filename', key: 'filename', ellipsis: true },
{ title: '错误', dataIndex: 'error', key: 'error' }
]
</script>
<template>
<div class="library page-section">
<div class="library__header">
<h2 class="library__title">知识库</h2>
<div class="library__actions">
<a-input
v-model:value="searchKeyword"
placeholder="搜索标题 / 摘要 / 类目"
allow-clear
style="width: 260px"
>
<template #prefix><SearchOutlined /></template>
</a-input>
<a-button :loading="isLoading" @click="handleRefresh">
<template #icon><ReloadOutlined /></template>
刷新
</a-button>
<a-button @click="tasksDrawer.open()">
<template #icon><ClockCircleOutlined /></template>
入库进度
</a-button>
<a-button type="primary" @click="openIngest">
<template #icon><UploadOutlined /></template>
入库
</a-button>
</div>
</div>
<div class="library__body">
<div class="library__tree">
<a-spin :spinning="isLoading">
<a-tree
:tree-data="treeDataComputed"
:default-expand-all="true"
:show-icon="true"
block-node
@select="handleTreeSelect"
>
<template #title="{ title, count, category }">
<span v-if="category !== undefined" class="library__node-cat">
<span>{{ title }}</span>
<span class="text-muted library__node-count">{{ count }}</span>
</span>
<span v-else>{{ title }}</span>
</template>
</a-tree>
</a-spin>
</div>
<div class="library__detail">
<!-- 类目说明 -->
<div v-if="selectedCategory" class="library__panel">
<h3 class="library__panel-title">
<TagsOutlined /> {{ selectedCategory }}
</h3>
<div class="text-muted">{{ categoryDesc || '(无描述)' }}</div>
</div>
<!-- 未选中提示 -->
<div v-else-if="!docDetailVisible" class="library__placeholder text-muted">
从左侧选择类目或文档查看详情
</div>
</div>
</div>
<!-- 文档详情抽屉 -->
<a-drawer
:open="docDetailVisible"
title="文档详情"
placement="right"
width="620"
:destroy-on-close="true"
@close="closeDocDetail"
>
<a-spin :spinning="isLoadingDetail">
<div v-if="docDetailData">
<a-descriptions :column="1" size="small" bordered>
<a-descriptions-item label="doc_id">{{ docDetailData.l1?.doc_id || '-' }}</a-descriptions-item>
<a-descriptions-item label="标题">{{ docDetailData.l1?.title || '-' }}</a-descriptions-item>
<a-descriptions-item label="类目">{{ docDetailData.l1?.category || '-' }}</a-descriptions-item>
<a-descriptions-item label="标签">
<template v-if="docDetailData.l1?.tags && docDetailData.l1.tags.length">
<a-tag v-for="tag in docDetailData.l1.tags" :key="tag">{{ tag }}</a-tag>
</template>
<span v-else class="text-muted">-</span>
</a-descriptions-item>
<a-descriptions-item label="chunks_count">{{ docDetailData.chunks_count ?? 0 }}</a-descriptions-item>
<a-descriptions-item v-if="docDetailData.file" label="原文文件">
<a :href="docDetailData.file.url" target="_blank" rel="noopener">
{{ docDetailData.file.filename }}
<span v-if="docDetailData.file.size_bytes" class="text-muted">
{{ formatSize(docDetailData.file.size_bytes) }}
</span>
</a>
</a-descriptions-item>
</a-descriptions>
<div class="library__detail-actions">
<a-button size="small" :loading="isReingesting" @click="handleReingest(docDetailData.l1)">
重新摘要
</a-button>
<a-button size="small" danger :loading="isDeleting" @click="handleDelete(docDetailData.l1)">
删除
</a-button>
</div>
<h3 class="library__section-title">L1 全文</h3>
<pre class="library__pre">{{ docDetailData.l1?.text || '' }}</pre>
<h3 class="library__section-title">L2 节点{{ (docDetailData.l2_nodes || []).length }}</h3>
<div v-if="(docDetailData.l2_nodes || []).length === 0" class="text-muted"></div>
<div v-for="(node, idx) in docDetailData.l2_nodes || []" :key="`l2-${idx}`" class="library__node">
<div class="library__node-path">{{ node.section_path || '' }}</div>
<div class="text-break">{{ node.text || '' }}</div>
</div>
<h3 class="library__section-title">L3 节点{{ (docDetailData.l3_nodes || []).length }}</h3>
<div v-if="(docDetailData.l3_nodes || []).length === 0" class="text-muted"></div>
<div v-for="(node, idx) in docDetailData.l3_nodes || []" :key="`l3-${idx}`" class="library__node">
<div class="library__node-path">{{ node.section_path || '' }}</div>
<div class="text-break">{{ node.text || '' }}</div>
</div>
</div>
<div v-else-if="!isLoadingDetail" class="text-muted">暂无数据</div>
</a-spin>
</a-drawer>
<!-- 入库抽屉 -->
<a-drawer
:open="ingestVisible"
title="文档入库"
placement="right"
width="560"
:destroy-on-close="true"
@close="() => (ingestVisible = false)"
>
<a-tabs v-model:activeKey="activeTab">
<a-tab-pane key="text" tab="文本入库">
<a-form ref="textFormRef" :model="textForm" :rules="textRules" layout="vertical">
<a-form-item label="标题" name="title">
<a-input v-model:value="textForm.title" placeholder="请输入标题" allow-clear />
</a-form-item>
<a-form-item label="来源" name="source">
<a-input v-model:value="textForm.source" placeholder="例如:manual / web / file" allow-clear />
</a-form-item>
<a-form-item label="正文" name="text">
<a-textarea v-model:value="textForm.text" placeholder="请输入文档正文" :auto-size="{ minRows: 8, maxRows: 18 }" />
</a-form-item>
<a-form-item>
<a-button type="primary" :loading="isSubmittingText || pollState.isPolling" @click="handleSubmitText">
提交入库
</a-button>
</a-form-item>
</a-form>
</a-tab-pane>
<a-tab-pane key="file" tab="文件上传">
<a-form :model="fileForm" layout="vertical">
<a-form-item label="选择文件">
<a-upload :file-list="fileList" :accept="ACCEPTED_EXTENSIONS" :max-count="1" :before-upload="handleFileChange" @remove="handleFileRemove">
<a-button :disabled="fileList.length >= 1">选择文件</a-button>
</a-upload>
<div class="text-muted" style="margin-top: 4px">支持 .txt/.md/.html/.htm/.pdf/.docx</div>
</a-form-item>
<a-form-item label="标题(可选,默认取文件名)" name="title">
<a-input v-model:value="fileForm.title" placeholder="留空则使用文件名去扩展" allow-clear />
</a-form-item>
<a-form-item label="来源(可选,默认 file:原文件名)" name="source">
<a-input v-model:value="fileForm.source" placeholder="例如:manual / web" allow-clear />
</a-form-item>
<a-form-item>
<a-button type="primary" :loading="isSubmittingFile || pollState.isPolling" @click="handleSubmitFile">
上传入库
</a-button>
</a-form-item>
</a-form>
</a-tab-pane>
<a-tab-pane key="batch" tab="批量上传">
<a-form :model="fileForm" layout="vertical">
<a-form-item label="选择文件(可多选)">
<a-upload multiple :file-list="fileList" :accept="ACCEPTED_EXTENSIONS" :before-upload="handleBatchFileChange" @remove="handleBatchRemove">
<a-button>选择文件</a-button>
</a-upload>
<div class="text-muted" style="margin-top: 4px">支持 .txt/.md/.html/.htm/.pdf/.docx可多选逐文件异步入库</div>
</a-form-item>
<a-form-item v-if="fileList.length">
<a-button type="primary" :loading="isSubmittingBatch" @click="handleSubmitBatch">
批量上传入库{{ fileList.length }} 个文件
</a-button>
<a-button style="margin-left: 8px" :disabled="isSubmittingBatch" @click="handleBatchClear">清空</a-button>
</a-form-item>
</a-form>
<div v-if="batchResult" class="library__batch">
<a-alert
v-if="batchResult.failed && batchResult.failed.length"
class="library__alert"
type="error"
show-icon
:message="`${batchResult.failed.length} 个文件上传失败`"
/>
<a-table
v-if="batchResult.tasks && batchResult.tasks.length"
:data-source="batchResult.tasks"
:columns="batchColumns"
:pagination="false"
size="small"
row-key="task_id"
>
<template #bodyCell="{ column, record }">
<template v-if="column.key === 'task_id'">
<a-typography-text copyable :style="{ fontSize: '12px' }">{{ record.task_id }}</a-typography-text>
</template>
</template>
</a-table>
<a-table
v-if="batchResult.failed && batchResult.failed.length"
:data-source="batchResult.failed"
:columns="failedColumns"
:pagination="false"
size="small"
row-key="filename"
style="margin-top: 12px"
>
<template #bodyCell="{ column, record }">
<template v-if="column.key === 'error'">
<span class="text-break">{{ record.error }}</span>
</template>
</template>
</a-table>
</div>
</a-tab-pane>
</a-tabs>
<div v-if="pollState.taskId" class="library__result">
<div class="library__result-header">
<span>任务已提交{{ pollState.taskId }}</span>
<a-tag :color="statusBadgeColor">{{ statusBadgeText }}</a-tag>
<a-button v-if="pollState.isPolling" type="link" size="small" @click="handleCancelPolling">停止轮询</a-button>
</div>
<a-alert
v-if="pollState.isTimeout"
class="library__alert"
type="warning"
show-icon
:message="`任务仍在进行,可稍后凭 task_id 查询:${pollState.taskId}`"
/>
<div v-if="isTerminal && pollState.status === 'done' && resultData" class="library__done">
<a-descriptions :column="1" size="small" bordered>
<a-descriptions-item label="document_id">{{ resultData.document_id || '-' }}</a-descriptions-item>
<a-descriptions-item label="类目">{{ resultData.category || '-' }}置信度 {{ resultData.category_confidence ?? '-' }}</a-descriptions-item>
<a-descriptions-item label="标签">
<template v-if="resultData.tags && resultData.tags.length">
<a-tag v-for="tag in resultData.tags" :key="tag">{{ tag }}</a-tag>
</template>
<span v-else class="text-muted">-</span>
</a-descriptions-item>
<a-descriptions-item label="总结层级">{{ summaryData.level ?? '-' }}</a-descriptions-item>
<a-descriptions-item label="chunks_count">{{ resultData.chunks_count ?? 0 }}</a-descriptions-item>
</a-descriptions>
<h3 class="library__section-title">L1 总结</h3>
<pre class="library__pre">{{ summaryData.l1_summary || '' }}</pre>
</div>
<div v-if="isTerminal && pollState.status === 'failed'" class="library__failed">
<a-descriptions :column="1" size="small" bordered>
<a-descriptions-item label="失败阶段">{{ errorData?.stage || '-' }}</a-descriptions-item>
<a-descriptions-item label="错误信息"><span class="text-break">{{ errorData?.message || '-' }}</span></a-descriptions-item>
</a-descriptions>
<template v-if="partialSummary && partialSummary.l1_summary">
<h3 class="library__section-title">L1 总结</h3>
<pre class="library__pre">{{ partialSummary.l1_summary }}</pre>
</template>
</div>
</div>
</a-drawer>
</div>
</template>
<style scoped>
.library__header {
display: flex;
align-items: center;
justify-content: space-between;
margin-bottom: 12px;
flex-wrap: wrap;
gap: 8px;
}
.library__title {
font-size: 16px;
margin: 0;
}
.library__actions {
display: flex;
align-items: center;
gap: 8px;
flex-wrap: wrap;
}
.library__body {
display: flex;
gap: 16px;
min-height: 480px;
}
.library__tree {
width: 320px;
flex: 0 0 auto;
background: #fff;
border: 1px solid #f0f0f0;
border-radius: 8px;
padding: 12px;
max-height: 70vh;
overflow: auto;
}
.library__node-cat {
display: inline-flex;
align-items: center;
gap: 6px;
}
.library__node-count {
font-size: 12px;
}
.library__detail {
flex: 1;
min-width: 0;
}
.library__panel {
background: #fff;
border: 1px solid #f0f0f0;
border-radius: 8px;
padding: 16px;
}
.library__panel-title {
font-size: 15px;
margin: 0 0 8px;
display: flex;
align-items: center;
gap: 6px;
}
.library__placeholder {
display: flex;
align-items: center;
justify-content: center;
height: 100%;
min-height: 300px;
border: 1px dashed #d9d9d9;
border-radius: 8px;
}
.library__detail-actions {
display: flex;
gap: 8px;
margin: 12px 0;
}
.library__section-title {
font-size: 14px;
margin: 16px 0 8px;
color: #374151;
}
.library__pre {
background: #f9fafb;
border: 1px solid #e5e7eb;
border-radius: 4px;
padding: 8px;
white-space: pre-wrap;
word-break: break-word;
font-size: 12px;
max-height: 260px;
overflow: auto;
margin: 0;
}
.library__node {
border-left: 3px solid #93c5fd;
padding: 6px 10px;
margin-bottom: 6px;
background: #f8fafc;
border-radius: 0 4px 4px 0;
}
.library__node-path {
font-size: 12px;
color: #6b7280;
margin-bottom: 4px;
}
.library__result {
margin-top: 16px;
border: 1px solid #e5e7eb;
border-radius: 6px;
background: #fafafa;
padding: 12px 16px;
}
.library__result-header {
display: flex;
align-items: center;
gap: 8px;
flex-wrap: wrap;
font-size: 13px;
margin-bottom: 8px;
}
.library__alert {
margin: 8px 0;
}
.library__batch {
margin-top: 8px;
}
@media (max-width: 768px) {
.library__body {
flex-direction: column;
}
.library__tree {
width: 100%;
}
}
</style>
+229 -70
View File
@@ -1,5 +1,5 @@
<script setup> <script setup>
import { computed, onMounted, ref } from 'vue' import { computed, onMounted, reactive, ref } from 'vue'
import { message } from 'ant-design-vue' import { message } from 'ant-design-vue'
import { import {
DatabaseOutlined, DatabaseOutlined,
@@ -9,69 +9,82 @@ import {
FileTextOutlined, FileTextOutlined,
QuestionCircleOutlined, QuestionCircleOutlined,
ReloadOutlined, ReloadOutlined,
AppstoreOutlined AppstoreOutlined,
SearchOutlined
} from '@ant-design/icons-vue' } from '@ant-design/icons-vue'
import { stats as fetchStats } from '@/api/knowledge' import { stats as fetchStats } from '@/api/knowledge'
import { search as searchApi } from '@/api/search'
/* ---------- 检索 ---------- */
const isSearching = ref(false)
const resultData = ref(null)
const searchFormRef = ref(null)
const searchForm = reactive({
query: '',
top_k: 5,
summarize: false
})
const searchRules = {
query: [{ required: true, message: '请输入查询语句', trigger: 'blur' }],
top_k: [{ type: 'number', min: 1, max: 50, message: 'top_k 范围 1~50', trigger: 'change' }]
}
const routedCategoriesText = (cats) => {
if (!cats || !cats.length) return '(无)'
return cats.join(', ')
}
function eiValue(value) {
if (value === undefined || value === null || value === '') return '(无)'
if (Array.isArray(value)) {
return value.length ? value.join(', ') : '(无)'
}
return String(value)
}
const hits = (data) => data?.hits || []
async function handleSearch() {
try {
await searchFormRef.value.validate()
} catch {
return
}
isSearching.value = true
resultData.value = null
try {
resultData.value = await searchApi({
query: searchForm.query,
top_k: searchForm.top_k,
summarize: searchForm.summarize
})
} catch (err) {
message.error(err?.message || '检索失败')
} finally {
isSearching.value = false
}
}
/* ---------- 概览统计 ---------- */
const isLoading = ref(false) const isLoading = ref(false)
const statsData = ref(null) const statsData = ref(null)
const cardMeta = [ const cardMeta = [
{ { key: 'doc_l1', label: 'L1 文档总结', icon: DatabaseOutlined, color: '#1677ff', bg: 'rgba(22, 119, 255, 0.12)' },
key: 'doc_l1', { key: 'doc_l2', label: 'L2 大纲节点', icon: ApartmentOutlined, color: '#722ed1', bg: 'rgba(114, 46, 209, 0.12)' },
label: 'L1 文档总结', { key: 'doc_l3', label: 'L3 内容大纲', icon: FileSearchOutlined, color: '#13c2c2', bg: 'rgba(19, 194, 194, 0.12)' },
icon: DatabaseOutlined, { key: 'chunks', label: 'Chunks', icon: BlockOutlined, color: '#fa8c16', bg: 'rgba(250, 140, 22, 0.12)' },
color: '#1677ff', { key: '__documents_total', label: '文档总数', icon: FileTextOutlined, color: '#52c41a', bg: 'rgba(82, 196, 26, 0.12)' },
bg: 'rgba(22, 119, 255, 0.12)' { key: '__uncategorized', label: '未分类文档', icon: QuestionCircleOutlined, color: '#ff4d4f', bg: 'rgba(255, 77, 79, 0.12)' }
},
{
key: 'doc_l2',
label: 'L2 大纲节点',
icon: ApartmentOutlined,
color: '#722ed1',
bg: 'rgba(114, 46, 209, 0.12)'
},
{
key: 'doc_l3',
label: 'L3 内容大纲',
icon: FileSearchOutlined,
color: '#13c2c2',
bg: 'rgba(19, 194, 194, 0.12)'
},
{
key: 'chunks',
label: 'Chunks',
icon: BlockOutlined,
color: '#fa8c16',
bg: 'rgba(250, 140, 22, 0.12)'
},
{
key: '__documents_total',
label: '文档总数',
icon: FileTextOutlined,
color: '#52c41a',
bg: 'rgba(82, 196, 26, 0.12)'
},
{
key: '__uncategorized',
label: '未分类文档',
icon: QuestionCircleOutlined,
color: '#ff4d4f',
bg: 'rgba(255, 77, 79, 0.12)'
}
] ]
const cards = computed(() => { const cards = computed(() => {
const collections = statsData.value?.collections || {} const collections = statsData.value?.collections || {}
return cardMeta.map((m) => { return cardMeta.map((m) => {
let value = 0 let value = 0
if (m.key === '__documents_total') { if (m.key === '__documents_total') value = statsData.value?.documents_total ?? 0
value = statsData.value?.documents_total ?? 0 else if (m.key === '__uncategorized') value = statsData.value?.uncategorized_count ?? 0
} else if (m.key === '__uncategorized') { else value = collections[m.key] ?? 0
value = statsData.value?.uncategorized_count ?? 0
} else {
value = collections[m.key] ?? 0
}
return { ...m, value } return { ...m, value }
}) })
}) })
@@ -126,26 +139,108 @@ onMounted(() => {
<div class="overview page-section"> <div class="overview page-section">
<div class="overview__header"> <div class="overview__header">
<h2 class="page-title">概览</h2> <h2 class="page-title">概览</h2>
<a-space> <a-button :loading="isLoading" @click="loadStats">
<a-button :loading="isLoading" @click="loadStats"> <template #icon><ReloadOutlined /></template>
<template #icon><ReloadOutlined /></template> 刷新
刷新 </a-button>
</a-button>
</a-space>
</div> </div>
<!-- 检索测试 -->
<div class="overview__search">
<h3 class="overview__subtitle">
<SearchOutlined />
<span>检索</span>
</h3>
<a-form
ref="searchFormRef"
:model="searchForm"
:rules="searchRules"
layout="inline"
>
<a-form-item name="query" style="flex: 1; min-width: 260px">
<a-input
v-model:value="searchForm.query"
placeholder="请输入查询语句"
allow-clear
@pressEnter="handleSearch"
/>
</a-form-item>
<a-form-item name="top_k">
<a-input-number
v-model:value="searchForm.top_k"
:min="1"
:max="50"
style="width: 120px"
/>
</a-form-item>
<a-form-item name="summarize">
<a-checkbox v-model:checked="searchForm.summarize">AI 总结</a-checkbox>
</a-form-item>
<a-form-item>
<a-button type="primary" :loading="isSearching" @click="handleSearch">
检索
</a-button>
</a-form-item>
</a-form>
<div v-if="resultData" class="overview__result">
<a-descriptions :column="1" size="small" bordered>
<a-descriptions-item label="routed_categories">
{{ routedCategoriesText(resultData.routed_categories) }}
</a-descriptions-item>
</a-descriptions>
<a-alert
v-if="resultData.fallback"
class="overview__alert"
type="warning"
show-icon
message="fallback:本次检索触发了回退策略(路由类目无命中,已降级全局检索)"
/>
<div v-if="resultData.summary" class="overview__summary">
<div class="overview__summary-title">AI 总结</div>
<div class="text-break">{{ resultData.summary }}</div>
</div>
<div v-if="resultData.extracted_info" class="overview__extracted">
<div class="overview__extracted-title">AI 提取的关键信息</div>
<a-descriptions :column="1" size="small" bordered>
<a-descriptions-item label="改写">{{ eiValue(resultData.extracted_info.rewrite) }}</a-descriptions-item>
<a-descriptions-item label="关键词">{{ eiValue(resultData.extracted_info.keywords) }}</a-descriptions-item>
<a-descriptions-item label="实体">{{ eiValue(resultData.extracted_info.entities) }}</a-descriptions-item>
<a-descriptions-item label="意图">{{ eiValue(resultData.extracted_info.intent) }}</a-descriptions-item>
<a-descriptions-item label="时间范围">{{ eiValue(resultData.extracted_info.time_range) }}</a-descriptions-item>
<a-descriptions-item label="命中类目">{{ eiValue(resultData.extracted_info.categories) }}</a-descriptions-item>
</a-descriptions>
</div>
<div class="text-muted" style="margin: 12px 0 8px">
命中 {{ hits(resultData).length }}
</div>
<div
v-for="(hit, idx) in hits(resultData)"
:key="`hit-${idx}`"
class="overview__hit"
>
<div class="overview__hit-meta">
score={{ hit.score }} | doc_id={{ hit.doc_id }} | 标题={{ hit.title || '' }} | section={{ hit.section_path || '' }}
</div>
<div class="overview__hit-snippet text-break">{{ hit.text || '' }}</div>
<div v-if="hit.doc_summary" class="overview__hit-meta">
文档摘要{{ hit.doc_summary }}
</div>
</div>
</div>
</div>
<!-- 概览统计 -->
<a-spin :spinning="isLoading"> <a-spin :spinning="isLoading">
<div class="overview__cards"> <div class="overview__cards">
<div <div v-for="card in cards" :key="card.key" class="overview__card">
v-for="card in cards"
:key="card.key"
class="overview__card"
>
<div class="overview__card-body"> <div class="overview__card-body">
<div <div class="overview__card-icon" :style="{ background: card.bg, color: card.color }">
class="overview__card-icon"
:style="{ background: card.bg, color: card.color }"
>
<component :is="card.icon" /> <component :is="card.icon" />
</div> </div>
<div class="overview__card-info"> <div class="overview__card-info">
@@ -166,9 +261,7 @@ onMounted(() => {
{{ categoryBars.length }} 个类目 {{ categoryBars.length }} 个类目
</span> </span>
</div> </div>
<div v-if="categoryBars.length === 0" class="overview__empty text-muted"> <div v-if="categoryBars.length === 0" class="overview__empty text-muted">暂无数据</div>
暂无数据
</div>
<div v-else class="overview__bars"> <div v-else class="overview__bars">
<div <div
v-for="(bar, idx) in categoryBars" v-for="(bar, idx) in categoryBars"
@@ -200,6 +293,72 @@ onMounted(() => {
margin-bottom: 16px; margin-bottom: 16px;
} }
.overview__search {
background: #fff;
border: 1px solid #f0f0f0;
border-radius: 8px;
padding: 16px;
margin-bottom: 24px;
}
.overview__result {
margin-top: 16px;
}
.overview__alert {
margin: 12px 0;
}
.overview__summary {
background: #f0fdf4;
border: 1px solid #bbf7d0;
border-radius: 6px;
padding: 12px 14px;
margin: 12px 0;
font-size: 13px;
}
.overview__summary-title {
font-weight: 600;
margin-bottom: 6px;
color: #15803d;
}
.overview__extracted {
background: #f0f9ff;
border: 1px solid #bae6fd;
border-radius: 6px;
padding: 12px 14px;
margin: 12px 0;
font-size: 13px;
}
.overview__extracted-title {
font-weight: 600;
margin-bottom: 8px;
color: #0369a1;
}
.overview__hit {
border: 1px solid #e5e7eb;
border-radius: 6px;
padding: 10px 12px;
margin-bottom: 10px;
background: #fff;
}
.overview__hit-meta {
font-size: 12px;
color: #6b7280;
margin-bottom: 6px;
word-break: break-all;
}
.overview__hit-snippet {
font-size: 13px;
color: #1f2937;
}
.overview__cards { .overview__cards {
display: grid; display: grid;
grid-template-columns: repeat(auto-fill, minmax(220px, 1fr)); grid-template-columns: repeat(auto-fill, minmax(220px, 1fr));
@@ -272,7 +431,7 @@ onMounted(() => {
font-size: 14px; font-size: 14px;
font-weight: 600; font-weight: 600;
color: #374151; color: #374151;
margin: 0; margin: 0 0 12px;
display: flex; display: flex;
align-items: center; align-items: center;
gap: 6px; gap: 6px;
-234
View File
@@ -1,234 +0,0 @@
<script setup>
import { reactive, ref } from 'vue'
import { message } from 'ant-design-vue'
import { search as searchApi } from '@/api/search'
const isLoading = ref(false)
const resultData = ref(null)
const formRef = ref(null)
const formState = reactive({
query: '',
top_k: 5,
summarize: false
})
const rules = {
query: [{ required: true, message: '请输入查询语句', trigger: 'blur' }],
top_k: [{ type: 'number', min: 1, max: 50, message: 'top_k 范围 1~50', trigger: 'change' }]
}
const routedCategoriesText = (cats) => {
if (!cats || !cats.length) return '(无)'
return cats.join(', ')
}
async function handleSearch() {
try {
await formRef.value.validate()
} catch {
return
}
isLoading.value = true
resultData.value = null
try {
const payload = {
query: formState.query,
top_k: formState.top_k,
summarize: formState.summarize
}
resultData.value = await searchApi(payload)
} catch (err) {
message.error(err?.message || '检索失败')
} finally {
isLoading.value = false
}
}
function eiValue(value) {
if (value === undefined || value === null || value === '') return '(无)'
if (Array.isArray(value)) {
return value.length ? value.join(', ') : '(无)'
}
return String(value)
}
const hits = (data) => data?.hits || []
</script>
<template>
<div class="search page-section">
<div class="search__header">
<h2 class="search__title">检索测试台</h2>
</div>
<a-form
ref="formRef"
:model="formState"
:rules="rules"
layout="vertical"
>
<a-form-item label="查询语句" name="query">
<a-input
v-model:value="formState.query"
placeholder="请输入查询语句"
allow-clear
@pressEnter="handleSearch"
/>
</a-form-item>
<a-form-item label="top_k" name="top_k">
<a-input-number
v-model:value="formState.top_k"
:min="1"
:max="50"
style="width: 160px"
/>
</a-form-item>
<a-form-item name="summarize">
<a-checkbox v-model:checked="formState.summarize">
对结果生成 AI 总结
</a-checkbox>
</a-form-item>
<a-form-item>
<a-button type="primary" :loading="isLoading" @click="handleSearch">
检索
</a-button>
</a-form-item>
</a-form>
<div v-if="resultData" class="search__result">
<a-descriptions :column="1" size="small" bordered>
<a-descriptions-item label="routed_categories">
{{ routedCategoriesText(resultData.routed_categories) }}
</a-descriptions-item>
</a-descriptions>
<a-alert
v-if="resultData.fallback"
class="search__alert"
type="warning"
show-icon
message="fallback:本次检索触发了回退策略(路由类目无命中,已降级全局检索)"
/>
<div v-if="resultData.summary" class="search__summary">
<div class="search__summary-title">AI 总结</div>
<div class="text-break">{{ resultData.summary }}</div>
</div>
<div v-if="resultData.extracted_info" class="search__extracted">
<div class="search__extracted-title">AI 提取的关键信息</div>
<a-descriptions :column="1" size="small" bordered>
<a-descriptions-item label="改写">
{{ eiValue(resultData.extracted_info.rewrite) }}
</a-descriptions-item>
<a-descriptions-item label="关键词">
{{ eiValue(resultData.extracted_info.keywords) }}
</a-descriptions-item>
<a-descriptions-item label="实体">
{{ eiValue(resultData.extracted_info.entities) }}
</a-descriptions-item>
<a-descriptions-item label="意图">
{{ eiValue(resultData.extracted_info.intent) }}
</a-descriptions-item>
<a-descriptions-item label="时间范围">
{{ eiValue(resultData.extracted_info.time_range) }}
</a-descriptions-item>
<a-descriptions-item label="命中类目">
{{ eiValue(resultData.extracted_info.categories) }}
</a-descriptions-item>
</a-descriptions>
</div>
<div class="text-muted" style="margin: 12px 0 8px">
命中 {{ hits(resultData).length }}
</div>
<div
v-for="(hit, idx) in hits(resultData)"
:key="`hit-${idx}`"
class="search__hit"
>
<div class="search__hit-meta">
score={{ hit.score }} | doc_id={{ hit.doc_id }} | 标题={{ hit.title || '' }} | section={{ hit.section_path || '' }}
</div>
<div class="search__hit-snippet text-break">{{ hit.text || '' }}</div>
<div v-if="hit.doc_summary" class="search__hit-meta">
文档摘要{{ hit.doc_summary }}
</div>
</div>
</div>
</div>
</template>
<style scoped>
.search__header {
display: flex;
align-items: center;
justify-content: space-between;
margin-bottom: 12px;
}
.search__title {
font-size: 16px;
margin: 0;
}
.search__result {
margin-top: 16px;
}
.search__alert {
margin: 12px 0;
}
.search__summary {
background: #f0fdf4;
border: 1px solid #bbf7d0;
border-radius: 6px;
padding: 12px 14px;
margin: 12px 0;
font-size: 13px;
}
.search__summary-title {
font-weight: 600;
margin-bottom: 6px;
color: #15803d;
}
.search__extracted {
background: #f0f9ff;
border: 1px solid #bae6fd;
border-radius: 6px;
padding: 12px 14px;
margin: 12px 0;
font-size: 13px;
}
.search__extracted-title {
font-weight: 600;
margin-bottom: 8px;
color: #0369a1;
}
.search__hit {
border: 1px solid #e5e7eb;
border-radius: 6px;
padding: 10px 12px;
margin-bottom: 10px;
background: #fff;
}
.search__hit-meta {
font-size: 12px;
color: #6b7280;
margin-bottom: 6px;
word-break: break-all;
}
.search__hit-snippet {
font-size: 13px;
color: #1f2937;
}
</style>
+1 -1
View File
@@ -318,7 +318,7 @@ onMounted(() => {
<a-form-item label="model"> <a-form-item label="model">
<a-input <a-input
v-model:value="form.models[key].model" v-model:value="form.models[key].model"
placeholder="例如 qwen2.5:1.5b" placeholder="例如 qwen3:1.7b"
allow-clear allow-clear
/> />
</a-form-item> </a-form-item>
+394
View File
@@ -0,0 +1,394 @@
<script setup>
import { computed, onBeforeUnmount, ref, watch } from 'vue'
import { message, Modal } from 'ant-design-vue'
import {
ReloadOutlined,
ClockCircleOutlined,
EyeOutlined,
ArrowLeftOutlined,
UndoOutlined,
DeleteOutlined
} from '@ant-design/icons-vue'
import {
taskList,
taskStatus,
retryTask,
deleteTask
} from '@/api/documents'
import {
INGEST_STATUS_TEXT,
INGEST_STATUS_COLOR
} from '@/constants/ingest'
import { useTasksDrawer } from '@/composables/useTasksDrawer'
import { formatSize } from '@/utils/format'
const { state, close } = useTasksDrawer()
const tasks = ref([])
const total = ref(0)
const isLoading = ref(false)
const isPolling = ref(false)
/* 选中任务详情 */
const selectedTask = ref(null)
const isLoadingDetail = ref(false)
const isRetrying = ref(false)
const isDeleting = ref(false)
let timer = null
function formatTime(iso) {
if (!iso) return '-'
const d = new Date(iso)
if (Number.isNaN(d.getTime())) return iso
const pad = (n) => String(n).padStart(2, '0')
return `${d.getFullYear()}-${pad(d.getMonth() + 1)}-${pad(d.getDate())} ${pad(d.getHours())}:${pad(d.getMinutes())}:${pad(d.getSeconds())}`
}
function statusText(s) {
return INGEST_STATUS_TEXT[s] || s || '-'
}
function statusColor(s) {
return INGEST_STATUS_COLOR[s] || 'default'
}
const isTerminal = computed(() => ['done', 'failed'].includes(selectedTask.value?.status))
const detailResult = computed(() => selectedTask.value?.result || null)
const detailError = computed(() => selectedTask.value?.error || null)
const detailSummary = computed(() => detailResult.value?.summary || {})
const canRetry = computed(() => ['failed', 'done'].includes(selectedTask.value?.status))
const canDelete = computed(() => ['failed', 'done'].includes(selectedTask.value?.status))
/* 从任务记录提取详情展示字段(title/source/size */
const detailComputed = computed(() => {
const task = selectedTask.value
if (!task) return null
const source = task.source || {}
const metadata = source.metadata || {}
return {
title: task.title || source.title || '-',
source: task.source_text || source.source || '-',
size: task.size_bytes != null ? formatSize(task.size_bytes) : formatSize(metadata.original_size_bytes)
}
})
async function loadTasks(showSpinner = false) {
if (showSpinner) isLoading.value = true
try {
const data = await taskList(50)
tasks.value = data.items || []
total.value = data.total ?? tasks.value.length
} catch (err) {
message.error(err?.message || '加载入库任务失败')
} finally {
if (showSpinner) isLoading.value = false
}
}
function startPolling() {
if (timer) return
isPolling.value = true
timer = setInterval(async () => {
await loadTasks(false)
}, 2000)
}
function stopPolling() {
if (timer) {
clearInterval(timer)
timer = null
}
isPolling.value = false
}
function togglePolling() {
if (isPolling.value) stopPolling()
else startPolling()
}
/* 打开/关闭抽屉时启停轮询 */
watch(
() => state.open,
(open) => {
if (open) {
loadTasks(true)
startPolling()
} else {
stopPolling()
selectedTask.value = null
}
},
{ immediate: true }
)
async function openDetail(task) {
selectedTask.value = null
isLoadingDetail.value = true
try {
selectedTask.value = await taskStatus(task.task_id)
} catch (err) {
message.error(err?.message || '加载任务详情失败')
} finally {
isLoadingDetail.value = false
}
}
function backToList() {
selectedTask.value = null
}
async function handleRetry(task) {
const taskId = task?.task_id || selectedTask.value?.task_id
if (!taskId) return
isRetrying.value = true
try {
const data = await retryTask(taskId)
message.success(`已提交新任务:${data.task_id}`)
selectedTask.value = null
await loadTasks(false)
} catch (err) {
message.error(err?.message || '重试失败')
} finally {
isRetrying.value = false
}
}
function handleDelete(task) {
const taskId = task?.task_id || selectedTask.value?.task_id
if (!taskId) return
Modal.confirm({
title: '删除入库任务',
content: `确定删除任务「${taskId}」吗?删除后不可恢复。`,
okText: '删除',
okType: 'danger',
cancelText: '取消',
async onOk() {
isDeleting.value = true
try {
await deleteTask(taskId)
message.success('任务已删除')
selectedTask.value = null
await loadTasks(false)
} catch (err) {
message.error(err?.message || '删除任务失败')
} finally {
isDeleting.value = false
}
}
})
}
onBeforeUnmount(stopPolling)
const columns = [
{ title: '状态', dataIndex: 'status', key: 'status', width: 90 },
{ title: '文件名', dataIndex: 'filename', key: 'filename', ellipsis: true },
{ title: 'task_id', dataIndex: 'task_id', key: 'task_id', width: 200 },
{ title: 'doc_id', dataIndex: 'doc_id', key: 'doc_id', width: 200 },
{ title: '创建时间', dataIndex: 'created_at', key: 'created_at', width: 160 },
{ title: '操作', dataIndex: 'action', key: 'action', width: 130 }
]
</script>
<template>
<a-drawer
:open="state.open"
:title="selectedTask ? '任务详情' : '入库进度'"
placement="right"
width="640"
:destroy-on-close="true"
@close="close"
>
<!-- 列表视图 -->
<template v-if="!selectedTask">
<div class="tasks-drawer__header">
<span class="text-muted"> {{ total }} </span>
<div class="tasks-drawer__actions">
<a-button size="small" :loading="isLoading" @click="loadTasks(true)">
<template #icon><ReloadOutlined /></template>
刷新
</a-button>
<a-button size="small" :type="isPolling ? 'default' : 'primary'" @click="togglePolling">
{{ isPolling ? '停止轮询' : '开始轮询' }}
</a-button>
</div>
</div>
<a-alert
v-if="isPolling"
type="info"
show-icon
message="每 2 秒自动刷新近期入库任务。"
style="margin-bottom: 12px"
/>
<a-table
:columns="columns"
:data-source="tasks"
:pagination="false"
:loading="isLoading"
row-key="task_id"
size="small"
:scroll="{ x: 720 }"
>
<template #bodyCell="{ column, record }">
<template v-if="column.key === 'status'">
<a-tag :color="statusColor(record.status)">{{ statusText(record.status) }}</a-tag>
</template>
<template v-else-if="column.key === 'filename'">
<span :title="record.filename">{{ record.filename || '-' }}</span>
</template>
<template v-else-if="column.key === 'task_id'">
<a-typography-text copyable :style="{ fontSize: '12px' }">{{ record.task_id }}</a-typography-text>
</template>
<template v-else-if="column.key === 'doc_id'">
<a-typography-text
v-if="record.doc_id"
copyable
:style="{ fontSize: '12px' }"
>
{{ record.doc_id }}
</a-typography-text>
<span v-else class="text-muted">-</span>
</template>
<template v-else-if="column.key === 'created_at'">
{{ formatTime(record.created_at) }}
</template>
<template v-else-if="column.key === 'action'">
<a-button type="link" size="small" @click="openDetail(record)">
<template #icon><EyeOutlined /></template>
详情
</a-button>
<a-button
v-if="['failed', 'done'].includes(record.status)"
type="link"
size="small"
@click="handleRetry(record)"
>
<template #icon><UndoOutlined /></template>
重试
</a-button>
<a-button
v-if="['done', 'failed'].includes(record.status)"
type="link"
size="small"
danger
@click="handleDelete(record)"
>
<template #icon><DeleteOutlined /></template>
删除
</a-button>
</template>
</template>
</a-table>
</template>
<!-- 详情视图 -->
<template v-else>
<a-button size="small" type="link" style="margin-bottom: 12px; padding-left: 0" @click="backToList">
<template #icon><ArrowLeftOutlined /></template>
返回列表
</a-button>
<a-spin :spinning="isLoadingDetail">
<template v-if="selectedTask">
<a-descriptions :column="1" size="small" bordered>
<a-descriptions-item label="状态">
<a-tag :color="statusColor(selectedTask.status)">{{ statusText(selectedTask.status) }}</a-tag>
</a-descriptions-item>
<a-descriptions-item label="task_id">{{ selectedTask.task_id || '-' }}</a-descriptions-item>
<a-descriptions-item label="文件名">{{ selectedTask.filename || '-' }}</a-descriptions-item>
<a-descriptions-item label="标题">{{ detailComputed?.title || '-' }}</a-descriptions-item>
<a-descriptions-item label="来源">{{ detailComputed?.source || '-' }}</a-descriptions-item>
<a-descriptions-item label="文件大小">{{ detailComputed?.size || '-' }}</a-descriptions-item>
<a-descriptions-item label="doc_id">{{ selectedTask.doc_id || '-' }}</a-descriptions-item>
<a-descriptions-item label="创建时间">{{ formatTime(selectedTask.created_at) }}</a-descriptions-item>
<a-descriptions-item label="更新时间">{{ formatTime(selectedTask.updated_at) }}</a-descriptions-item>
</a-descriptions>
<!-- 完成详情 -->
<div v-if="isTerminal && selectedTask.status === 'done' && detailResult" class="tasks-drawer__block">
<h4 class="tasks-drawer__block-title">入库结果</h4>
<a-descriptions :column="1" size="small" bordered>
<a-descriptions-item label="document_id">{{ detailResult.document_id || '-' }}</a-descriptions-item>
<a-descriptions-item label="类目">{{ detailResult.category || '-' }}置信度 {{ detailResult.category_confidence ?? '-' }}</a-descriptions-item>
<a-descriptions-item label="标签">
<template v-if="detailResult.tags && detailResult.tags.length">
<a-tag v-for="tag in detailResult.tags" :key="tag">{{ tag }}</a-tag>
</template>
<span v-else class="text-muted">-</span>
</a-descriptions-item>
<a-descriptions-item label="总结层级">{{ detailSummary.level ?? '-' }}</a-descriptions-item>
<a-descriptions-item label="chunks_count">{{ detailResult.chunks_count ?? 0 }}</a-descriptions-item>
<a-descriptions-item v-if="detailResult.deduplicated" label="去重">复用既有文档</a-descriptions-item>
</a-descriptions>
<h4 class="tasks-drawer__block-title">L1 总结</h4>
<pre class="tasks-drawer__pre">{{ detailSummary.l1_summary || '' }}</pre>
</div>
<!-- 失败详情 -->
<div v-if="isTerminal && selectedTask.status === 'failed' && detailError" class="tasks-drawer__block">
<a-alert
type="error"
show-icon
:message="`失败阶段:${detailError.stage || '-'}`"
:description="detailError.message || '-'"
/>
<template v-if="detailError.partial_summary && detailError.partial_summary.l1_summary">
<h4 class="tasks-drawer__block-title">L1 总结部分</h4>
<pre class="tasks-drawer__pre">{{ detailError.partial_summary.l1_summary }}</pre>
</template>
</div>
<div class="tasks-drawer__actions">
<a-button v-if="canRetry" :loading="isRetrying" @click="handleRetry(selectedTask)">
<template #icon><UndoOutlined /></template>
重试
</a-button>
<a-button v-if="canDelete" danger :loading="isDeleting" @click="handleDelete(selectedTask)">
<template #icon><DeleteOutlined /></template>
删除
</a-button>
</div>
</template>
</a-spin>
</template>
</a-drawer>
</template>
<style scoped>
.tasks-drawer__header {
display: flex;
align-items: center;
justify-content: space-between;
margin-bottom: 8px;
}
.tasks-drawer__actions {
display: flex;
align-items: center;
gap: 8px;
}
.tasks-drawer__block {
margin-top: 16px;
}
.tasks-drawer__block-title {
font-size: 13px;
margin: 12px 0 6px;
color: #374151;
}
.tasks-drawer__pre {
background: #f9fafb;
border: 1px solid #e5e7eb;
border-radius: 4px;
padding: 8px;
white-space: pre-wrap;
word-break: break-word;
font-size: 12px;
max-height: 220px;
overflow: auto;
margin: 0;
}
</style>
+289
View File
@@ -0,0 +1,289 @@
<script setup>
import { computed, onMounted, reactive, ref } from 'vue'
import { message, Modal } from 'ant-design-vue'
import {
TeamOutlined,
UserAddOutlined,
ReloadOutlined,
KeyOutlined,
DeleteOutlined,
StopOutlined,
CheckCircleOutlined
} from '@ant-design/icons-vue'
import {
listUsers,
createUser,
updateUser,
resetPassword,
deleteUser
} from '@/api/auth'
import { useAuthStore } from '@/stores/useAuthStore'
const authStore = useAuthStore()
const currentUsername = computed(() => authStore.user?.username || '')
const isLoading = ref(false)
const users = ref([])
const createVisible = ref(false)
const creating = ref(false)
const createForm = reactive({ username: '', password: '', role: 'user' })
const resetVisible = ref(false)
const resetting = ref(false)
const resetTarget = ref('')
const resetForm = reactive({ new_password: '' })
const columns = [
{ title: '用户名', dataIndex: 'username', key: 'username' },
{ title: '角色', dataIndex: 'role', key: 'role', width: 100, align: 'center' },
{ title: '状态', key: 'enabled', width: 90, align: 'center' },
{ title: '强制改密', key: 'must_change_password', width: 90, align: 'center' },
{ title: '创建时间', dataIndex: 'created_at', key: 'created_at', width: 180 },
{ title: '操作', key: 'action', width: 230, fixed: 'right' }
]
async function loadUsers() {
isLoading.value = true
try {
users.value = await listUsers()
} catch (err) {
message.error(err?.message || '加载用户列表失败')
} finally {
isLoading.value = false
}
}
function formatTime(ts) {
if (!ts) return '-'
const d = new Date(ts)
if (Number.isNaN(d.getTime())) return String(ts)
return d.toLocaleString('zh-CN', { hour12: false })
}
// 不能对当前登录账号做删除/禁用(后端亦会拦截)
function isSelf(username) {
return username === currentUsername.value
}
function openCreate() {
createForm.username = ''
createForm.password = ''
createForm.role = 'user'
createVisible.value = true
}
async function handleCreate() {
if (!createForm.username.trim()) {
message.warning('请输入用户名')
return
}
if (createForm.password.length < 8) {
message.warning('密码至少 8 位')
return
}
creating.value = true
try {
await createUser(createForm.username.trim(), createForm.password, createForm.role)
message.success(`已创建用户 ${createForm.username.trim()}`)
createVisible.value = false
await loadUsers()
} catch (err) {
message.error(err?.message || '创建用户失败')
} finally {
creating.value = false
}
}
function openReset(username) {
resetTarget.value = username
resetForm.new_password = ''
resetVisible.value = true
}
async function handleReset() {
if (resetForm.new_password.length < 8) {
message.warning('新密码至少 8 位')
return
}
resetting.value = true
try {
await resetPassword(resetTarget.value, resetForm.new_password)
message.success(`已重置 ${resetTarget.value} 的密码`)
resetVisible.value = false
} catch (err) {
message.error(err?.message || '重置密码失败')
} finally {
resetting.value = false
}
}
async function toggleEnabled(record) {
const enabled = !record.enabled
try {
await updateUser(record.username, { enabled })
message.success(`${record.username}${enabled ? '启用' : '禁用'}`)
await loadUsers()
} catch (err) {
message.error(err?.message || '更新状态失败')
}
}
function handleDelete(record) {
Modal.confirm({
title: '确认删除用户',
content: `确定删除用户「${record.username}」吗?该操作不可恢复。`,
okText: '删除',
okType: 'danger',
cancelText: '取消',
async onOk() {
try {
await deleteUser(record.username)
message.success(`已删除 ${record.username}`)
await loadUsers()
} catch (err) {
message.error(err?.message || '删除用户失败')
}
}
})
}
onMounted(() => {
loadUsers()
})
</script>
<template>
<div class="users page-section">
<div class="users__header">
<h2 class="page-title">
<TeamOutlined /> 用户管理
</h2>
<a-space>
<a-button :loading="isLoading" @click="loadUsers">
<template #icon><ReloadOutlined /></template>
刷新
</a-button>
<a-button type="primary" @click="openCreate">
<template #icon><UserAddOutlined /></template>
新建用户
</a-button>
</a-space>
</div>
<a-table
:columns="columns"
:data-source="users"
:pagination="false"
:loading="isLoading"
row-key="username"
size="middle"
>
<template #bodyCell="{ column, record }">
<template v-if="column.key === 'username'">
<span>{{ record.username }}</span>
<a-tag v-if="isSelf(record.username)" color="gold" style="margin-left: 6px">当前</a-tag>
</template>
<template v-else-if="column.key === 'role'">
<a-tag :color="record.role === 'admin' ? 'green' : 'blue'">{{ record.role }}</a-tag>
</template>
<template v-else-if="column.key === 'enabled'">
<a-tag :color="record.enabled ? 'success' : 'default'">
{{ record.enabled ? '启用' : '禁用' }}
</a-tag>
</template>
<template v-else-if="column.key === 'must_change_password'">
<a-tag v-if="record.must_change_password" color="orange">需改密</a-tag>
<span v-else class="text-muted">-</span>
</template>
<template v-else-if="column.key === 'created_at'">
<span class="text-muted">{{ formatTime(record.created_at) }}</span>
</template>
<template v-else-if="column.key === 'action'">
<a-space>
<a-button type="link" size="small" @click="openReset(record.username)">
<template #icon><KeyOutlined /></template>
重置密码
</a-button>
<a-button
type="link"
size="small"
:disabled="isSelf(record.username)"
@click="toggleEnabled(record)"
>
<template #icon>
<StopOutlined v-if="record.enabled" />
<CheckCircleOutlined v-else />
</template>
{{ record.enabled ? '禁用' : '启用' }}
</a-button>
<a-button
type="link"
size="small"
danger
:disabled="isSelf(record.username)"
@click="handleDelete(record)"
>
<template #icon><DeleteOutlined /></template>
删除
</a-button>
</a-space>
</template>
</template>
</a-table>
<!-- 新建用户 -->
<a-modal
v-model:open="createVisible"
title="新建用户"
ok-text="创建"
cancel-text="取消"
:confirm-loading="creating"
@ok="handleCreate"
>
<a-form layout="vertical">
<a-form-item label="用户名" required>
<a-input v-model:value="createForm.username" placeholder="登录用户名" />
</a-form-item>
<a-form-item label="密码" required>
<a-input-password v-model:value="createForm.password" placeholder="至少 8 位" />
</a-form-item>
<a-form-item label="角色">
<a-radio-group v-model:value="createForm.role">
<a-radio value="user">普通用户 (user)</a-radio>
<a-radio value="admin">管理员 (admin)</a-radio>
</a-radio-group>
</a-form-item>
</a-form>
</a-modal>
<!-- 重置密码 -->
<a-modal
v-model:open="resetVisible"
title="重置密码"
ok-text="重置"
cancel-text="取消"
:confirm-loading="resetting"
@ok="handleReset"
>
<p class="text-muted">目标用户{{ resetTarget }}</p>
<a-form-item label="新密码" required>
<a-input-password v-model:value="resetForm.new_password" placeholder="至少 8 位" />
</a-form-item>
</a-modal>
</div>
</template>
<style scoped>
.users__header {
display: flex;
align-items: center;
justify-content: space-between;
margin-bottom: 16px;
}
.page-title :deep(.anticon) {
margin-right: 8px;
color: #1677ff;
}
</style>
+144
View File
@@ -0,0 +1,144 @@
#!/usr/bin/env bash
# =============================================================================
# QMDSearch 自动部署脚本(本地 CI/CD 入口)
#
# 设计原则(贴合 NAS 自托管 + 挂载式部署):
# 1. 检测变化 —— 对比 origin/main,区分 backend / frontend / deps 范围
# 2. 校验 —— 本地跑 pytest(CI 门禁),坏代码绝不上线
# 3. 按需构建 —— 仅前端变化时本地预构建(dist 被 gitignoreNAS 端会再构建)
# 4. 提交推送 —— git commit + push origin main,推送后 SSH 到 NAS 执行真正部署
#
# 用法:
# ./scripts/deploy.sh 正常流程:检测 → 校验 → 按需构建 → 提交推送 → SSH 到 NAS 部署
# ./scripts/deploy.sh --dry 零副作用自测:只 检测/校验/构建,不提交不推送
# ./scripts/deploy.sh --no-push 提交但不推送(准备好后手动 push 或再跑一次)
# ./scripts/deploy.sh --no-ssh 推送后不自动 SSH 部署(仅同步代码到 NAS git)
# ./scripts/deploy.sh --skip-tests 跳过 pytest(谨慎)
# ./scripts/deploy.sh --skip-fe 跳过前端构建(即便前端变化)
# ./scripts/deploy.sh --build-fe 强制重新构建前端
# ./scripts/deploy.sh --help
# =============================================================================
set -euo pipefail
REMOTE="origin"
BRANCH="main"
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cd "$ROOT"
NO_PUSH=0
DRY=0
NO_SSH=0
SKIP_TESTS=0
SKIP_FE=0
FORCE_FE=0
while [[ $# -gt 0 ]]; do
case "$1" in
--no-push) NO_PUSH=1 ;;
--dry) DRY=1 ;;
--no-ssh) NO_SSH=1 ;;
--skip-tests) SKIP_TESTS=1 ;;
--skip-fe) SKIP_FE=1 ;;
--build-fe) FORCE_FE=1 ;;
--help|-h) sed -n '1,22p' "$0"; exit 0 ;;
*) echo "未知参数: $1"; exit 2 ;;
esac
shift
done
log(){ printf '\033[36m[deploy]\033[0m %s\n' "$*"; }
ok(){ printf '\033[32m[ ok ]\033[0m %s\n' "$*"; }
err(){ printf '\033[31m[err ]\033[0m %s\n' "$*" >&2; }
# ---- 1. 变化检测 --------------------------------------------------------------
log "同步远端引用..."
git fetch --quiet "$REMOTE" "$BRANCH" 2>/dev/null || log "fetch 失败(离线?继续本地流程)"
LOCAL_CHANGES="$(git status --porcelain)"
AHEAD="$(git rev-list --count "$REMOTE/$BRANCH..HEAD" 2>/dev/null || echo 0)"
# 汇总「已提交未推送」+「未提交」的变更文件,去重
CHANGED_FILES="$(
{ git diff --name-only "$REMOTE/$BRANCH" HEAD 2>/dev/null; echo "$LOCAL_CHANGES" | awk '{print $2}'; } \
| grep -v '^$' | sort -u
)"
has(){ echo "$CHANGED_FILES" | grep -qE "$1"; }
BACKEND_CHANGED=0; FRONTEND_CHANGED=0; DEPS_CHANGED=0
if has '^(app/|scripts/|tests/|.*\.py$|Dockerfile|docker-compose\.yml|\.env\.example)'; then BACKEND_CHANGED=1; fi
if has '^frontend/(src/|public/|index\.html|package\.json|vite\.config|package-lock\.json)'; then FRONTEND_CHANGED=1; fi
if has '^(pyproject\.toml|uv\.lock|Dockerfile)'; then DEPS_CHANGED=1; fi
if [[ -z "$CHANGED_FILES" && "$AHEAD" -eq 0 ]]; then
ok "无代码变化,无需部署。"
exit 0
fi
log "变更范围 → backend=$BACKEND_CHANGED frontend=$FRONTEND_CHANGED deps=$DEPS_CHANGED (ahead=$AHEAD)"
# ---- 2. 校验(本地 CI gate---------------------------------------------------
if [[ "$SKIP_TESTS" -eq 0 && "$BACKEND_CHANGED" -eq 1 ]]; then
log "运行后端测试 (uv run pytest)..."
if ! uv run pytest -q --no-header 2>&1 | tail -n 30; then
err "测试失败,已中止部署。请修复后再运行 deploy.sh"
exit 1
fi
ok "后端测试通过"
# 顺带做一次非阻断的静态检查
if command -v uv >/dev/null 2>&1; then
log "运行 ruff 静态检查(不阻断)..."
uv run ruff check app tests 2>&1 | tail -n 15 || true
fi
fi
# ---- 3. 按需构建前端 ----------------------------------------------------------
if [[ ("$FRONTEND_CHANGED" -eq 1 || "$FORCE_FE" -eq 1) && "$SKIP_FE" -eq 0 ]]; then
log "构建前端 (npm ci && npm run build)..."
if ! (cd frontend && npm ci --no-audit --no-fund && npm run build); then
err "前端构建失败,已中止。"
exit 1
fi
ok "前端构建完成 -> frontend/dist"
log "提示: frontend/dist 已被 gitignoreNAS 端 post-receive 会重新构建以生效。"
fi
# ---- 4. 提交 + 推送 -----------------------------------------------------------
if [[ "$DRY" -eq 1 ]]; then
log "(--dry) 零副作用模式:跳过提交与推送。以上为检测结果,确认无误后可去掉 --dry 正式部署。"
exit 0
fi
if [[ -n "$LOCAL_CHANGES" ]]; then
log "暂存并提交本地变化..."
git add -A # .gitignore 已排除 .env / data / node_modules / .workbuddy / .trae 等
SCOPE=""
[[ "$BACKEND_CHANGED" -eq 1 ]] && SCOPE="$SCOPE backend"
[[ "$FRONTEND_CHANGED" -eq 1 ]] && SCOPE="$SCOPE frontend"
[[ "$DEPS_CHANGED" -eq 1 ]] && SCOPE="$SCOPE deps"
MSG="chore(deploy): 自动部署 $(date '+%Y-%m-%d %H:%M')${SCOPE:+ |$SCOPE}"
git commit -q -m "$MSG"
ok "已提交: $MSG"
fi
if [[ "$NO_PUSH" -eq 1 ]]; then
log "(--no-push) 未推送。当前领先远端 $AHEAD 个提交,确认无误后去掉 --no-push 再跑。"
exit 0
fi
export QMDS_SKIP_PREPUSH=1 # 避免与 pre-push hook 重复跑测试
log "推送到 $REMOTE/$BRANCH(触发 NAS 自动部署)..."
if ! git push "$REMOTE" "$BRANCH"; then
err "推送失败:可能远端有更新,请先 git pull 解决冲突后重试。"
exit 1
fi
ok "已推送。代码已同步至 NAS git。"
if [[ "$NO_SSH" -eq 0 ]]; then
log "通过 SSH 在 NAS 上执行部署..."
if bash "$ROOT/scripts/nas-deploy-ssh.sh"; then
ok "NAS 部署完成。"
else
err "NAS 部署脚本返回非 0(详见上方 NAS 输出)。"
exit 1
fi
else
log "(--no-ssh) 已跳过 NAS 端部署。如需部署请运行: ./scripts/nas-deploy-ssh.sh"
fi
+48
View File
@@ -0,0 +1,48 @@
#!/usr/bin/env bash
# =============================================================================
# 安装本地 git pre-push 门禁
#
# 作用:任何「推送到 origin(NAS)」的操作前,先跑后端测试;若前端有变化,
# 先在本地预构建一次。任一环节失败则阻断推送,确保坏代码绝不上线。
#
# 说明:
# - deploy.sh 推送前会设 QMDS_SKIP_PREPUSH=1,避免与自身校验重复跑。
# - 仅对 origin 远端生效,推其他远端不受影响。
#
# 卸载:rm .git/hooks/pre-push
# =============================================================================
set -e
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
HOOK="$ROOT/.git/hooks/pre-push"
mkdir -p "$(dirname "$HOOK")"
cat > "$HOOK" <<'HOOK_EOF'
#!/usr/bin/env bash
# QMDSearch pre-push 校验门禁(由 scripts/install-hooks.sh 安装)
REMOTE="$1"
# 仅对 NAS 远端 (origin) 做校验;且 deploy.sh 已设 QMDS_SKIP_PREPUSH 时跳过
if [[ "$REMOTE" != "origin" ]]; then exit 0; fi
if [[ -n "${QMDS_SKIP_PREPUSH:-}" ]]; then exit 0; fi
echo "[pre-push] 推送前校验后端测试..."
if ! uv run pytest -q --no-header 2>&1 | tail -n 20; then
echo "[pre-push] 测试失败,已阻断推送。请先修复后再 push。" >&2
exit 1
fi
# 暂存区若含前端变化,本地先预构建一次,避免推到 NAS 才翻车
if git diff --cached --name-only | grep -qE '^frontend/'; then
echo "[pre-push] 检测到前端变化,本地预构建..."
if ! (cd frontend && npm ci --no-audit --no-fund && npm run build); then
echo "[pre-push] 前端构建失败,已阻断推送。" >&2
exit 1
fi
fi
echo "[pre-push] 校验通过,允许推送。"
HOOK_EOF
chmod +x "$HOOK"
echo "已安装 pre-push 门禁 -> $HOOK"
echo "如需卸载: rm $HOOK"
+120
View File
@@ -0,0 +1,120 @@
#!/usr/bin/env bash
# =============================================================================
# QMDSearch NAS 端部署(通过 SSH 直连执行,密钥免密)
#
# 环境约束(已实测):
# - NAS 上有 docker compose v2,但【没有 node/npm】→ 前端必须在本地构建后 rsync 过去
# - compose 的 app 服务只有 image: 无 build: → 依赖变化用 `docker build -t qmdsearch-app:latest .`
# - kp 已在 docker 组,可免 sudo 跑 docker
# - 部署目录 /vol1/qmdsearch 是 git 工作树,以 origin/main 为唯一真相源
#
# 流程:
# 1. SSH 到 NASgit fetch + git reset --hard origin/main(同步受管代码),并算出变更范围
# 2. 本地:若前端变化 → 本地 npm 构建 → rsync frontend/dist 到 NASNAS 无 node
# 3. SSH 到 NAS:依赖变化则 docker build + force-recreate;否则 restart app
# 4. 健康检查 /health;失败回滚到上一稳定提交
#
# 用法:
# ./scripts/nas-deploy-ssh.sh 正式部署
# ./scripts/nas-deploy-ssh.sh --dry-run 只预览将要部署的变更,不重启
# 环境变量:NAS_HOST(默认 nas DEPLOY_DIR(默认 /vol1/qmdsearch APP_PORT(默认 8000
# =============================================================================
set -uo pipefail
DEPLOY_HOST="${NAS_HOST:-nas}"
DEPLOY_DIR="${DEPLOY_DIR:-/vol1/qmdsearch}"
APP_PORT="${APP_PORT:-8000}"
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
DRY_RUN=0
[[ "${1:-}" == "--dry-run" ]] && DRY_RUN=1
# 前置检查
if ! ssh -o BatchMode=yes -o ConnectTimeout=5 "$DEPLOY_HOST" true 2>/dev/null; then
echo "[local] 无法免密 SSH 到 $DEPLOY_HOST,请先配置 SSH 密钥登录。" >&2
exit 1
fi
command -v rsync >/dev/null 2>&1 || { echo "[local] 本机缺少 rsync,请先安装。" >&2; exit 1; }
# ---- Phase 1: NAS 同步受管代码 + 计算变更范围 ----
SCOPE=$(ssh "$DEPLOY_HOST" "bash -s -- '${DEPLOY_DIR}'" <<'REMOTE'
DEPLOY_DIR="$1"
set -uo pipefail
cd "$DEPLOY_DIR" || { echo "NOCD"; exit 1; }
git fetch origin main 2>&1 | tail -n 2
NEW=$(git rev-parse origin/main)
PREV=$(cat .last_deployed_commit 2>/dev/null || echo "NONE")
if [[ "$PREV" == "$NEW" ]]; then echo "UP_TO_DATE"; exit 0; fi
OLD_HEAD=$(git rev-parse HEAD)
git reset --hard origin/main 2>&1 | tail -n 2
BASE="$OLD_HEAD"
[[ "$PREV" != "NONE" ]] && BASE="$PREV"
CHANGED=$(git diff --name-only "$BASE" "$NEW")
FE=0; DEPS=0
echo "$CHANGED" | grep -qE '^frontend/(src/|public/|index\.html|package\.json|vite\.config|package-lock\.json)' && FE=1
echo "$CHANGED" | grep -qE '^(pyproject\.toml|uv\.lock|Dockerfile)' && DEPS=1
echo "SCOPE FE=$FE DEPS=$DEPS PREV=$PREV NEW=$NEW"
REMOTE
)
if echo "$SCOPE" | grep -q "NOCD"; then echo "[nas] 无法进入 $DEPLOY_DIR"; exit 1; fi
if echo "$SCOPE" | grep -q "UP_TO_DATE"; then echo "[nas] 已是最新,无需更新。"; exit 0; fi
FE=$(echo "$SCOPE" | grep -oE 'FE=[01]' | cut -d= -f2)
DEPS=$(echo "$SCOPE" | grep -oE 'DEPS=[01]' | cut -d= -f2)
PREV=$(echo "$SCOPE" | grep -oE 'PREV=[0-9a-f]+' | cut -d= -f2)
NEW=$(echo "$SCOPE" | grep -oE 'NEW=[0-9a-f]+' | cut -d= -f2)
echo "[deploy] 将部署 $PREV -> $NEW (FE=$FE DEPS=$DEPS)"
if [[ "$DRY_RUN" == "1" ]]; then
echo "[deploy][dry-run] 变更文件:"
if [[ "$PREV" != "NONE" ]]; then
ssh "$DEPLOY_HOST" "cd $DEPLOY_DIR && git diff --name-only $PREV $NEW" 2>/dev/null | sed 's/^/ /'
else
echo " (首次部署,将全量同步受管代码)"
fi
exit 0
fi
# ---- Phase 2: 前端本地构建 + rsync 到 NASNAS 无 node----
if [[ "$FE" == "1" ]]; then
echo "[local] 本地构建前端..."
if ! (cd "$ROOT/frontend" && npm ci --no-audit --no-fund && npm run build); then
echo "[local][WARN] 前端构建失败(当前环境可能受限或缺少 node 依赖),跳过前端同步;后端仍会部署,但 frontend/dist 可能为旧版本。待在可正常构建的环境中重跑即补齐。"
else
echo "[local] rsync frontend/dist -> $DEPLOY_HOST:$DEPLOY_DIR/frontend/dist"
if ! rsync -az --delete "$ROOT/frontend/dist/" "$DEPLOY_HOST:$DEPLOY_DIR/frontend/dist/"; then
echo "[local][WARN] rsync 失败,跳过前端同步。"
fi
fi
fi
# ---- Phase 3: NAS 容器部署 + 健康检查 ----
ssh "$DEPLOY_HOST" "bash -s -- '$DEPLOY_DIR' '$APP_PORT' '$DEPS' '$NEW' '$PREV'" <<'REMOTE'
DEPLOY_DIR="$1"; APP_PORT="$2"; DEPS="$3"; NEW="$4"; PREV="$5"
set -uo pipefail
cd "$DEPLOY_DIR"
if [[ "$DEPS" == "1" ]]; then
echo "[nas] 依赖变化,重建镜像: docker build -t qmdsearch-app:latest ."
if ! docker build -t qmdsearch-app:latest . 2>&1 | tail -n 20; then
echo "[nas] 镜像构建失败,回滚到 $PREV"; git checkout -f "$PREV"; exit 1
fi
echo "[nas] 用新镜像重建容器: docker compose up -d --force-recreate app"
docker compose up -d --force-recreate app 2>&1 | tail -n 10
else
echo "[nas] 仅代码/前端变化,重启 app: docker compose restart app"
docker compose restart app 2>&1 | tail -n 5
fi
echo "[nas] 健康检查 http://localhost:$APP_PORT/api/v1/health ..."
for i in $(seq 1 20); do
if curl -fsS "http://localhost:$APP_PORT/api/v1/health" >/dev/null 2>&1; then
echo "$NEW" > .last_deployed_commit
echo "[nas] 健康检查通过 ✓ 部署完成"
exit 0
fi
sleep 3
done
echo "[nas] 健康检查失败,回滚到 $PREV"
git checkout -f "$PREV"
docker compose restart app
exit 1
REMOTE
+107
View File
@@ -0,0 +1,107 @@
#!/usr/bin/env bash
# =============================================================================
# QMDSearch NAS 端自动部署钩子 (post-receive)
#
# 这是真正的「持续部署(CD)」:当本地 deploy.sh 把代码 push 到 NAS 后,
# 此钩子自动完成 checkout → 按需构建 → 重启/重建容器 → 健康检查 → 失败回滚。
#
# 安装方式(二选一):
# A. 作为 bare 仓库的 hook(推荐,全自动):
# # 假设 NAS 上 bare 仓库在 /volume1/git/QMDSearch.git,工作树在 /volume1/docker/QMDSearch
# cp scripts/nas-post-receive /volume1/git/QMDSearch.git/hooks/post-receive
# chmod +x /volume1/git/QMDSearch.git/hooks/post-receive
# # 在该 bare 仓库 config 中设置:
# # [core] bare = true
# # [hooks] 并确保工作树目录已 clone 过一次
# # 首次需手动:git --work-tree=/volume1/docker/QMDSearch --git-dir=/volume1/git/QMDSearch.git checkout -f
#
# B. 作为普通脚本手动触发(无 bare 仓库时):
# cd /volume1/docker/QMDSearch && git pull && sudo bash scripts/nas-post-receive
#
# 前置依赖(NAS 上需具备):git、docker、docker compose、node/npm
# 可调环境变量:DEPLOY_DIR(工作树)、APP_PORT、LOG_FILE
# =============================================================================
set -uo pipefail
DEPLOY_DIR="${DEPLOY_DIR:-/volume1/docker/QMDSearch}" # ← 改成你的 NAS 部署目录
GIT_DIR="$DEPLOY_DIR/.git"
LOG_FILE="${LOG_FILE:-$DEPLOY_DIR/deploy.log}"
APP_PORT="${APP_PORT:-8000}"
PREV_COMMIT_FILE="$DEPLOY_DIR/.last_deployed_commit"
log(){ printf '%s [nas-deploy] %s\n' "$(date '+%Y-%m-%d %H:%M:%S')" "$*" | tee -a "$LOG_FILE"; }
rollback(){
log "回滚到上一稳定提交: ${PREV:-<无>}"
if [[ -n "${PREV:-}" ]]; then
git --work-tree="$DEPLOY_DIR" --git-dir="$GIT_DIR" checkout -f "$PREV"
(cd "$DEPLOY_DIR" && docker compose restart app) | tee -a "$LOG_FILE"
echo "$PREV" > "$PREV_COMMIT_FILE"
fi
}
# 解析 post-receive 通过 stdin 传入的 <old> <new> <ref>
newrev=""; refname=""
while read -r oldrev newrev refname; do :; done
# 仅处理 main 分支的推送;手动运行时 newrev 为空,回退到 HEAD
if [[ -z "$newrev" ]]; then
newrev="$(git --git-dir="$GIT_DIR" rev-parse HEAD 2>/dev/null || echo "")"
fi
if [[ -n "$refname" && "$refname" != "refs/heads/main" ]]; then
log "忽略非 main 分支推送 ($refname)"
exit 0
fi
log "=== 开始部署 (newrev=${newrev:0:8}) ==="
# 1) 检出到工作树
git --work-tree="$DEPLOY_DIR" --git-dir="$GIT_DIR" checkout -f
cd "$DEPLOY_DIR" || { log "无法进入 $DEPLOY_DIR"; exit 1; }
# 2) 计算变化范围
PREV="$(cat "$PREV_COMMIT_FILE" 2>/dev/null || echo "")"
if [[ -n "$PREV" && -n "$newrev" ]]; then
CHANGED="$(git --git-dir="$GIT_DIR" diff --name-only "$PREV" "$newrev")"
else
CHANGED="$(git --git-dir="$GIT_DIR" show --name-only --format= "$newrev")"
fi
has(){ echo "$CHANGED" | grep -qE "$1"; }
FE_CHANGED=0; DEPS_CHANGED=0
has '^frontend/(src/|public/|index\.html|package\.json|vite\.config|package-lock\.json)' && FE_CHANGED=1
has '^(pyproject\.toml|uv\.lock|Dockerfile)' && DEPS_CHANGED=1
log "变更范围 → frontend=$FE_CHANGED deps=$DEPS_CHANGED"
# 3) 按需构建前端(dist 不入库,必须在此构建)
if [[ "$FE_CHANGED" -eq 1 ]]; then
log "检测前端变化,重新构建..."
if ! (cd frontend && npm ci --no-audit --no-fund && npm run build) 2>&1 | tee -a "$LOG_FILE"; then
log "前端构建失败,中止部署(保留上一版本)"; exit 1
fi
ok(){ :; }; log "前端构建完成"
fi
# 4) 部署容器
if [[ "$DEPS_CHANGED" -eq 1 ]]; then
log "依赖变化,重建镜像并重启: docker compose up -d --build app"
docker compose up -d --build app 2>&1 | tee -a "$LOG_FILE"
else
log "仅代码变化,重启 app 服务: docker compose restart app"
docker compose restart app 2>&1 | tee -a "$LOG_FILE"
fi
# 5) 健康检查(失败自动回滚)
log "健康检查 http://localhost:$APP_PORT/health ..."
for i in $(seq 1 15); do
if curl -fsS "http://localhost:$APP_PORT/health" >/dev/null 2>&1; then
log "健康检查通过 ✓"
[[ -n "$newrev" ]] && echo "$newrev" > "$PREV_COMMIT_FILE"
log "=== 部署完成 ==="
exit 0
fi
sleep 2
done
log "健康检查失败,触发回滚"
rollback
exit 1
+58
View File
@@ -0,0 +1,58 @@
#!/usr/bin/env bash
# =============================================================================
# QMDSearch 文件变化监视器(纯轮询,无需 fswatch / inotifywait
#
# 作用:持续监听仓库代码变化,实现「每次更新代码后自动检测」。
# - 默认模式:检测到变化 → 本地跑测试 + 前端预构建 → 提示你运行 deploy.sh
# - --auto 模式:检测到变化 → 直接调用 ./scripts/deploy.sh 全自动部署
# (注意:--auto 会在保存即提交推送,慎用,建议确认改动完整后再用)
#
# 用法:
# ./scripts/watch.sh 监听并本地校验,提示部署
# ./scripts/watch.sh --auto 监听变化后全自动部署
# ./scripts/watch.sh --interval 5 轮询间隔秒数(默认 3)
# =============================================================================
set -uo pipefail
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cd "$ROOT"
AUTO=0
INTERVAL=3
while [[ $# -gt 0 ]]; do
case "$1" in
--auto) AUTO=1 ;;
--interval) INTERVAL="${2:-3}"; shift ;;
*) echo "未知参数: $1"; exit 2 ;;
esac
shift
done
SNAPSHOT="$(mktemp)"
record(){ git status --porcelain > "$SNAPSHOT"; }
record
if [[ "$AUTO" -eq 1 ]]; then
echo "[watch] 监听 $ROOTAUTO 模式:变化即部署,间隔 ${INTERVAL}sCtrl-C 退出)"
else
echo "[watch] 监听 $ROOT(校验模式:变化即本地校验并提示,间隔 ${INTERVAL}sCtrl-C 退出)"
fi
while true; do
sleep "$INTERVAL"
if ! git status --porcelain | diff -q - "$SNAPSHOT" >/dev/null 2>&1; then
echo "[watch] $(date '+%H:%M:%S') 检测到代码变化"
if [[ "$AUTO" -eq 1 ]]; then
bash "$ROOT/scripts/deploy.sh"
else
echo "[watch] 运行本地校验..."
uv run pytest -q --no-header 2>&1 | tail -n 10 || true
if git status --porcelain | awk '{print $2}' | grep -qE '^frontend/'; then
echo "[watch] 前端变化,本地预构建..."
(cd frontend && npm ci --no-audit --no-fund && npm run build) 2>&1 | tail -n 5 || true
fi
echo "[watch] 校验完成。确认无误后运行: ./scripts/deploy.sh"
fi
record
fi
done
+145
View File
@@ -0,0 +1,145 @@
---
name: qmdsearch-agent
description: 通过 QMDSearch REST API 让 AI Agent 上传文档(文本/文件)并执行分层检索查询。配置服务地址与 Bearer Token 后,即可在对话中入库知识库并检索答案。
metadata: {"clawdbot":{"emoji":"📚"}}
---
# QMDSearch Agent 接入 Skill
本 skill 指导 AI Agent 调用 **QMDSearch**(面向 AI Agent 的分层信息检索服务)的 HTTP API,完成两类核心操作:
1. **上传文档**:把文本或文件送入知识库(异步入库,自动三级总结 + 向量化)。
2. **查询 / 检索**:对知识库做分层语义检索,返回最相关原文 chunk。
## 何时使用
- 用户希望把一份文档 / 笔记 / 网页内容加入知识库 → 调用上传接口。
- 用户就知识库内容提问 → 调用检索接口。
- 用户想确认文档是否入库成功 → 轮询任务状态。
- 需要列出 / 删除知识库文档、查看统计 → 调用对应只读 / 管理接口。
## 前置配置(必须)
启用前先确定:
- `QMDSEARCH_BASE_URL`:服务基地址,例如 `http://localhost:8000` 或 NAS 地址 `http://<nas-ip>:8000`
- `QMDSEARCH_TOKEN`Bearer Token。通过 `POST {BASE}/api/v1/auth/login`(用户名 + 密码)获取;session TTL 12h,失效后重新登录。
> 所有变更类请求(上传 / 删除)与检索请求都必须在 Header 携带 `Authorization: Bearer <token>`。
> 当前代码实现中,查询类只读接口(文档列表 / 详情 / 类目 / 统计 / 健康)同样要求 Bearer(与登录态绑定),请始终携带 token 以避免 `1005` 未认证。
## 统一约定
- 路径前缀:`{BASE}/api/v1`
- 响应体:`{ "code": 0, "data": {...}, "message": "ok" }``code=0` 成功;`1xxx` 客户端错误;`2xxx` 服务端错误。
- 入库是**异步**的:上传返回 `202 + task_id`,需轮询 `GET /documents/tasks/{task_id}` 直到 `status=done`(或 `failed`)。
## 1. 获取 Token
```bash
curl -X POST {BASE}/api/v1/auth/login \
-H "Content-Type: application/json" \
-d '{"username":"admin","password":"<password>"}'
# → {"code":0,"data":{"token":"<JWT>","username":"admin","role":"admin","must_change_password":false},"message":"ok"}
```
`must_change_password=true` 时,登录成功但调用其他接口会返回 `1006`,需先 `POST /api/v1/auth/password` 改密。
## 2. 上传文档(文本)
`POST {BASE}/api/v1/documents`JSON body `{text, title?, source?, metadata?}`
```bash
curl -X POST {BASE}/api/v1/documents \
-H "Authorization: Bearer <token>" \
-H "Content-Type: application/json" \
-d '{"text":"QMDSearch 支持 L1/L2/L3 分层检索……", "title":"QMDSearch 介绍", "source":"notes"}'
# → 202 {"code":0,"data":{"task_id":"<uuid>","status":"pending"},"message":"ok"}
```
- `text` 必填且非空;`title` / `source` 可选;`metadata``dict[str,str]`
- 相同 `sha256(text)` 会命中**去重**,直接复用旧文档(`data``deduplicated=true`)。
## 3. 上传文件
`POST {BASE}/api/v1/documents/upload``multipart/form-data`
- `file`:必填,支持 `.txt .md .html .htm .pdf .docx`
- `title` / `source`:可选(默认取原文件名)
- `metadata`:可选 JSON 字符串
```bash
curl -X POST {BASE}/api/v1/documents/upload \
-H "Authorization: Bearer <token>" \
-F "file=@document.pdf" \
-F "source=manual"
# → 202 {"code":0,"data":{"task_id":"<uuid>","status":"pending","saved_path":"..."},"message":"ok"}
```
服务端按扩展名提取纯文本(PDF 扫描件自动 OCR 降级),再复用同一入库流水线。
## 4. 轮询入库任务
`GET {BASE}/api/v1/documents/tasks/{task_id}`(无需鉴权):
```bash
curl {BASE}/api/v1/documents/tasks/<task_id>
# done → {"code":0,"data":{"task_id":...,"status":"done","result":{...文档ID/总结/分类...},"created_at":...,"updated_at":...}}
# failed → {"code":0,"data":{"task_id":...,"status":"failed","error":"..."}}
```
建议:提交后每 12s 轮询,直到 `status∈{done,failed}``done``result.document_id` 即入库文档 ID。
## 5. 检索(查询)
`POST {BASE}/api/v1/search`JSON `{query, top_k?, summarize?}`
```bash
curl -X POST {BASE}/api/v1/search \
-H "Authorization: Bearer <token>" \
-H "Content-Type: application/json" \
-d '{"query":"如何配置 Redis 缓存","top_k":5,"summarize":false}'
```
返回 `data``{query, hits[], routed_categories[], fallback, extracted_info, summary?}`
每个 `hit``{text, doc_id, title, section_path, score, doc_summary}`
- `top_k`:返回条数(默认 `RETRIEVAL_FINAL_K`,通常 5)。
- `summarize=true`:额外返回 `summary`(对检索结果做 AI 总结)。
- `fallback=true`:走了全库兜底(未走类目路由),召回较广。
- `extracted_info`:query 解析出的关键词 / 实体 / 意图 / 命中类目。
## 6. 其他管理接口(按需)
| 方法 | 路径 | 说明 |
|------|------|------|
| POST | `/documents` | 文本入库(202 + task_id |
| POST | `/documents/upload` | 文件入库(202 + task_id |
| GET | `/documents/tasks/{task_id}` | 任务状态(done / failed |
| GET | `/documents?limit=&offset=` | 文档列表(分页游标) |
| GET | `/documents/{doc_id}` | 文档详情(L1 + L2/L3 + chunks 数) |
| GET | `/documents/{doc_id}/file` | 下载原始文件 |
| DELETE | `/documents/{doc_id}` | 删除文档(幂等) |
| GET | `/knowledge/categories` | 知识分类类目集 |
| GET | `/knowledge/stats` | 统计(四层点数 + 类目分布) |
| GET | `/health` | 健康检查 |
## 错误码速查
- `1001` 参数 / 格式错误(空文本、不支持的类型、弱密码、超大小)
- `1004` 资源不存在(task / doc 不存在)
- `1005` 未认证或凭证无效 / 账号已禁用
- `1006` 权限不足 / 首次登录须先改密
- `2000` 服务端内部错误(检索 / 入库 / 列表失败)
- `2001` 认证服务暂不可用
## Agent 最佳实践
1. **先登录拿 token**,缓存 12h,失效再用 `/auth/login` 刷新。
2. **上传走异步**:拿到 `task_id` 后轮询,不要在对话里阻塞等待,可告诉用户“正在入库,稍后查询”。
3. **去重友好**:重复提交同文本会自动复用,不必担心重复。
4. **大文件 / 批量**:逐文件上传并各自轮询;注意 `UPLOAD_MAX_SIZE_MB`(默认 20MB)。
5. **检索调参**:召回不足调大 `top_k`;需要摘要设 `summarize=true`;关注 `fallback` 判断是否需要改写 query。
6. **凭证安全**:token 等同会话,不要写入日志或外发。
完整请求 / 响应示例(含 Python 客户端封装)见 `references/api-examples.md`
@@ -0,0 +1,170 @@
# QMDSearch API 示例与 Python 客户端封装
> 基址占位符 `{BASE}` 替换为实际服务地址,例如 `http://localhost:8000` 或 `http://<nas-ip>:8000`。
## 一、curl 示例
### 登录
```bash
curl -s -X POST {BASE}/api/v1/auth/login \
-H "Content-Type: application/json" \
-d '{"username":"admin","password":"<password>"}'
```
### 文本入库(202 + task_id
```bash
curl -s -X POST {BASE}/api/v1/documents \
-H "Authorization: Bearer <token>" \
-H "Content-Type: application/json" \
-d '{"text":"QMDSearch 是面向 AI Agent 的分层信息检索服务……","title":"QMDSearch 介绍","source":"notes"}'
```
### 文件上传(202 + task_id
```bash
curl -s -X POST {BASE}/api/v1/documents/upload \
-H "Authorization: Bearer <token>" \
-F "file=@document.pdf" \
-F "source=manual"
```
### 轮询任务状态
```bash
curl -s {BASE}/api/v1/documents/tasks/<task_id>
```
### 检索
```bash
curl -s -X POST {BASE}/api/v1/search \
-H "Authorization: Bearer <token>" \
-H "Content-Type: application/json" \
-d '{"query":"如何配置 Redis 缓存","top_k":5,"summarize":false}'
```
## 二、Python 客户端封装
可直接在 Agent 工具代码里复用:
```python
import time
import requests
class QMDSearchClient:
"""QMDSearch 最小客户端:登录、入库(文本/文件)、轮询、检索。"""
def __init__(self, base_url: str, username: str, password: str,
poll_interval: float = 1.5, poll_timeout: float = 120.0):
self.base = base_url.rstrip("/")
self.username = username
self.password = password
self.poll_interval = poll_interval
self.poll_timeout = poll_timeout
self.token: str | None = None
# ---- 鉴权 ----
def login(self) -> str:
r = requests.post(
f"{self.base}/api/v1/auth/login",
json={"username": self.username, "password": self.password},
timeout=30,
)
r.raise_for_status()
body = r.json()
if body.get("code") != 0:
raise RuntimeError(f"登录失败: {body}")
self.token = body["data"]["token"]
return self.token
def _headers(self) -> dict:
if not self.token:
self.login()
return {"Authorization": f"Bearer {self.token}"}
def _ok(self, resp: requests.Response):
resp.raise_for_status()
body = resp.json()
if body.get("code") != 0:
raise RuntimeError(f"API 错误 code={body.get('code')} msg={body.get('message')}")
return body["data"]
# ---- 入库 ----
def ingest_text(self, text: str, title: str = "", source: str = "",
metadata: dict | None = None) -> str:
data = {"text": text}
if title:
data["title"] = title
if source:
data["source"] = source
if metadata:
data["metadata"] = {str(k): str(v) for k, v in metadata.items()}
resp = requests.post(f"{self.base}/api/v1/documents",
headers=self._headers(), json=data, timeout=30)
return self._ok(resp)["task_id"]
def upload_file(self, path: str, title: str = "", source: str = "",
metadata: dict | None = None) -> str:
files = {"file": open(path, "rb")}
data = {}
if title:
data["title"] = title
if source:
data["source"] = source
if metadata:
data["metadata"] = str({str(k): str(v) for k, v in metadata.items()})
resp = requests.post(f"{self.base}/api/v1/documents/upload",
headers=self._headers(), files=files, data=data, timeout=60)
return self._ok(resp)["task_id"]
# ---- 轮询 ----
def wait_task(self, task_id: str) -> dict:
deadline = time.time() + self.poll_timeout
while time.time() < deadline:
resp = requests.get(f"{self.base}/api/v1/documents/tasks/{task_id}", timeout=30)
data = self._ok(resp)
if data["status"] == "done":
return data["result"]
if data["status"] == "failed":
raise RuntimeError(f"入库失败: {data.get('error')}")
time.sleep(self.poll_interval)
raise TimeoutError(f"任务 {task_id} 轮询超时")
def ingest_text_wait(self, *args, **kwargs) -> dict:
return self.wait_task(self.ingest_text(*args, **kwargs))
def upload_file_wait(self, *args, **kwargs) -> dict:
return self.wait_task(self.upload_file(*args, **kwargs))
# ---- 检索 ----
def search(self, query: str, top_k: int | None = None,
summarize: bool = False) -> dict:
payload = {"query": query, "summarize": summarize}
if top_k is not None:
payload["top_k"] = top_k
resp = requests.post(f"{self.base}/api/v1/search",
headers=self._headers(), json=payload, timeout=60)
return self._ok(resp)
# 用法
if __name__ == "__main__":
client = QMDSearchClient("http://localhost:8000", "admin", "<password>")
client.login()
# 文本入库并等待完成
result = client.ingest_text_wait("这是一篇关于分层检索的笔记……", title="笔记")
print("document_id =", result["document_id"])
# 检索
hits = client.search("分层检索是什么", top_k=5)["hits"]
for h in hits:
print(f"[{h['score']:.3f}] {h['title']}: {h['text'][:80]}")
```
## 三、响应结构速记
- 文本 / 文件入库 `202``data = {task_id, status:"pending"[, saved_path]}`
- 任务 `done``data.result = {document_id, summary:{l1_summary,l2_outline,l3_content_outline,level}, category, chunks_count, tags, category_confidence, deduplicated}`
- 检索 → `data = {query, hits:[{text,doc_id,title,section_path,score,doc_summary}], routed_categories, fallback, extracted_info, summary?}`
+8 -24
View File
@@ -1,39 +1,23 @@
"""pytest 全局夹具:覆盖 JWT 认证依赖,让现有 API 测试默认以 admin 身份运行 """pytest 全局夹具
业务接口(search/knowledge/settings)仍使用 app.core.auth 的 JWT 依赖, 鉴权采用 app.api.deps 的会话体系(UserStore / SessionStore):
这里通过 autouse 夹具把两个依赖统一替换为返回固定 admin AuthUser 的 lambda - 文档变更类端点(POST /documents、/documents/upload、DELETE /documents/{id})、
使现有 API 测试无需改动即可通过认证。单个测试需要走真实认证逻辑时, /auth/* 与 Settings 变更端点需要登录;
可在测试函数内 pop 掉对应 overrideautouse fixture yield 后会统一 clear。 - 查询类端点(POST /search、GET /knowledge/*、GET /documents*、
GET /documents/{id}/file)免登录。
文档变更类端点(POST /documents、/documents/upload、DELETE /documents/{id} auth_stores / admin_headers 夹具注入内存存储并签发真实 session token
使用 app.api.deps 的会话认证(UserStore/SessionStore),由 auth_stores / 供需要登录的接口测试使用;查询类端点测试直接无 token 调用即可,无需覆盖依赖。
admin_headers 夹具注入内存存储并签发真实 session token。
另外清理 LLM 客户端 / 解析插件 / 去重策略三处进程级缓存,避免跨测试串扰 另外清理 LLM 客户端 / 解析插件 / 去重策略三处进程级缓存,避免跨测试串扰
(不同测试可能注入不同的 redis/ollama 实例,缓存按配置签名而非实例区分)。 (不同测试可能注入不同的 redis/ollama 实例,缓存按配置签名而非实例区分)。
""" """
import asyncio import asyncio
from datetime import UTC, datetime
from typing import Any from typing import Any
import pytest import pytest
from app.core.auth import get_current_user, require_admin
from app.main import app
from app.models.auth import AuthUser
TEST_USER = AuthUser(username="testuser", role="admin", created_at=datetime.now(UTC))
@pytest.fixture(autouse=True)
def override_auth():
"""所有测试默认以 admin 身份运行(旧 JWT 依赖);测试结束清理 dependency_overrides"""
app.dependency_overrides[get_current_user] = lambda: TEST_USER
app.dependency_overrides[require_admin] = lambda: TEST_USER
yield
app.dependency_overrides.clear()
@pytest.fixture @pytest.fixture
def auth_stores(monkeypatch: pytest.MonkeyPatch): def auth_stores(monkeypatch: pytest.MonkeyPatch):
+4 -2
View File
@@ -204,11 +204,13 @@ class TestAdminClosedLoop:
assert item_b["category"] == DOC_B_CATEGORY assert item_b["category"] == DOC_B_CATEGORY
assert item_b["tags"] == [] assert item_b["tags"] == []
# 2. 详情:l1/l2_nodes/l3_nodes/chunks_count 与写入一致 # 2. 详情:l1/l2_nodes/l3_nodes/chunks_count/file 与写入一致
resp = client.get(f"/api/v1/documents/{DOC_A}") resp = client.get(f"/api/v1/documents/{DOC_A}")
assert resp.status_code == 200 assert resp.status_code == 200
data = resp.json()["data"] data = resp.json()["data"]
assert set(data.keys()) == {"l1", "l2_nodes", "l3_nodes", "chunks_count"} assert set(data.keys()) == {"l1", "l2_nodes", "l3_nodes", "chunks_count", "file"}
# 文本入库无 metadata → file=None
assert data["file"] is None
l1 = data["l1"] l1 = data["l1"]
assert l1["doc_id"] == DOC_A assert l1["doc_id"] == DOC_A
assert l1["title"] == DOC_A_TITLE assert l1["title"] == DOC_A_TITLE
+299 -10
View File
@@ -5,14 +5,16 @@
2. 五区块可识别标记(文案与 section id) 2. 五区块可识别标记(文案与 section id)
3. 零外部依赖:无 http(s) 外链资源、无 CDN 引用 3. 零外部依赖:无 http(s) 外链资源、无 CDN 引用
4. fetch 调用路径与后端 API 契约一致(含 /api/v1/auth/* 4. fetch 调用路径与后端 API 契约一致(含 /api/v1/auth/*
5. 删除操作的 confirm() 二次确认逻辑 5. 删除/重置操作的通用 overlay 弹窗组件(替换原生 prompt/confirm
6. 登录门禁:登录卡片、localStorage key、/auth/me 验证、/auth/login 路径 6. 登录门禁:登录卡片、localStorage key、/auth/me 验证、/auth/login 路径
7. 顶栏用户区:用户名、角色徽章、修改密码、退出登录 7. 顶栏用户区:用户名、角色徽章、修改密码、退出登录
8. 修改密码:旧/新/确认表单、must_change_password 强制改密 8. 修改密码:旧/新/确认表单、must_change_password 强制改密
9. 用户管理区块:列表/创建表单/角色下拉/重置/删除 confirm、admin 角色门禁 9. 用户管理区块:列表/创建表单/角色下拉/重置密码弹窗/删除确认弹窗、admin 角色门禁
10. 请求拦截:Authorization Bearer 注入、1005 回登录、1006 错误条 10. 请求拦截:Authorization Bearer 注入、1005 回登录、1006 错误条
11. API 指南区块:导航/section、API_GUIDE 清单与真实路由一致性、试一下面板、 11. API 指南区块:导航/section、API_GUIDE 清单与真实路由一致性、试一下面板、
curl 复制、auth 标注、upload 文件选择、禁止自定义 URL curl 复制、auth 标注、upload 文件选择、禁止自定义 URL
12. 入库进度区块(Task 2):section/nav、表格结构、2s 轮询启停、状态徽章复用、
done/failed 操作按钮、reingest 端点、批量上传 multiple + upload-batch 路径
""" """
import re import re
@@ -39,9 +41,20 @@ def client(monkeypatch: pytest.MonkeyPatch) -> Iterator[TestClient]:
@pytest.fixture @pytest.fixture
def admin_html(client: TestClient) -> str: def admin_html(client: TestClient) -> str:
"""请求 /admin 并返回 HTML 文本(前置断言 200""" """请求 /admin 并返回旧版 admin.html 文本(前置断言 200
当 app/static/admin 符号链接存在时(Vue SPA 部署),/admin 端点会返回 SPA 入口
而非旧版 admin.html。此时直接读取 admin.html 文件内容进行测试。
"""
resp = client.get("/admin") resp = client.get("/admin")
assert resp.status_code == 200 assert resp.status_code == 200
# 检测是否返回了 Vue SPA(无 section-overview 标记)
if "section-overview" not in resp.text:
# SPA 模式:直接读取旧版 admin.html 文件
from pathlib import Path
admin_html_path = Path(__file__).resolve().parent.parent / "app" / "static" / "admin.html"
return admin_html_path.read_text(encoding="utf-8")
return resp.text return resp.text
@@ -86,6 +99,8 @@ def test_admin_page_fetch_paths(admin_html: str) -> None:
"/api/v1/auth/logout", "/api/v1/auth/logout",
"/api/v1/auth/password", "/api/v1/auth/password",
"/api/v1/auth/users", "/api/v1/auth/users",
"/api/v1/documents/upload-batch",
"/api/v1/documents/tasks",
): ):
assert path in admin_html assert path in admin_html
@@ -107,6 +122,13 @@ def test_admin_page_fetch_paths_in_real_routes(admin_html: str) -> None:
"""页面 api() 调用路径均在真实后端路由集合内(含 tasks 与 /api/v1/auth/* 路径)""" """页面 api() 调用路径均在真实后端路由集合内(含 tasks 与 /api/v1/auth/* 路径)"""
route_paths = _collect_route_paths(app.routes) route_paths = _collect_route_paths(app.routes)
assert "/api/v1/documents/tasks/{task_id}" in route_paths assert "/api/v1/documents/tasks/{task_id}" in route_paths
# Task 2 新增端点在真实路由集合内
for new_path in (
"/api/v1/documents/tasks",
"/api/v1/documents/upload-batch",
"/api/v1/documents/{doc_id}/reingest",
):
assert new_path in route_paths, f"后端缺少 Task 2 新增端点: {new_path}"
for auth_path in ( for auth_path in (
"/api/v1/auth/login", "/api/v1/auth/login",
"/api/v1/auth/me", "/api/v1/auth/me",
@@ -140,9 +162,103 @@ def test_admin_page_ingest_polling(admin_html: str) -> None:
assert "disabled" in admin_html assert "disabled" in admin_html
def test_admin_page_has_confirm(admin_html: str) -> None: def test_admin_page_progress_section(admin_html: str) -> None:
"""删除操作包含 confirm() 二次确认逻辑""" """入库进度区块:导航按钮、section、表格结构、刷新按钮、空列表占位"""
assert "confirm(" in admin_html # section 与导航按钮(所有登录用户可见)
assert 'id="section-progress"' in admin_html
assert 'id="nav-progress"' in admin_html
assert 'data-target="section-progress"' in admin_html
assert "入库进度" in admin_html
# 刷新按钮与状态统计
assert 'id="btn-refresh-progress"' in admin_html
assert 'id="progress-stats"' in admin_html
# 表格 tbody 与表头列:文件名 / 状态 / 创建时间 / 更新时间 / 操作
assert 'id="progress-tbody"' in admin_html
for col in ("文件名", "状态", "创建时间", "更新时间", "操作"):
assert col in admin_html
# 空列表占位文案
assert "暂无入库任务" in admin_html
def test_admin_page_progress_polling(admin_html: str) -> None:
"""入库进度区块:2s 轮询常量、startProgressPolling/stopProgressPolling 函数、GET /documents/tasks 路径"""
# 进度区块专用轮询常量(2s
assert "PROGRESS_POLL_INTERVAL_MS" in admin_html
assert "2000" in admin_html
# 轮询启停函数
assert "function startProgressPolling" in admin_html
assert "function stopProgressPolling" in admin_html
assert "progressPollTimer" in admin_html
# 数据来源:GET /documents/tasks?limit=50
assert "/api/v1/documents/tasks?limit=50" in admin_html
# loadProgress / renderProgress / buildProgressRow 函数
assert "function loadProgress" in admin_html
assert "function renderProgress" in admin_html
assert "function buildProgressRow" in admin_html
def test_admin_page_progress_polling_lifecycle(admin_html: str) -> None:
"""进度区块轮询生命周期:activateSection 切走暂停、切回恢复;showLogin 停止"""
# activateSection 开头调用 stopProgressPolling(切走即暂停)
assert "stopProgressPolling();" in admin_html
# 切到 section-progress 时恢复轮询
assert 'targetId === "section-progress"' in admin_html
assert 'startProgressPolling()' in admin_html
# showLogin 退出时停止轮询
assert "stopProgressPolling" in admin_html
def test_admin_page_progress_status_badge_reuse(admin_html: str) -> None:
"""进度表格状态徽章复用 INGEST_STATUS_TEXT 中文映射与 status-* 配色"""
# statusBadgeEl 共享构造函数(无 id 可重复使用)
assert "function statusBadgeEl" in admin_html
# makeStatusBadge 复用 statusBadgeEl + 加 id
assert "statusBadgeEl(status)" in admin_html
# 状态徽章 CSS 类复用
for cls in ("status-running", "status-done", "status-failed"):
assert cls in admin_html
def test_admin_page_progress_actions(admin_html: str) -> None:
"""进度表格操作列:done 查看文档+重新入库、failed 重试、进行中无操作;reingest 端点"""
# done 状态操作按钮
assert "查看文档" in admin_html
assert "重新入库" in admin_html
# failed 状态重试按钮
assert "重试" in admin_html
# reingestDocument 函数与 POST /documents/{doc_id}/reingest 端点
assert "function reingestDocument" in admin_html
assert "/reingest" in admin_html
# 查看文档:切到 section-docs 并调用 loadDocDetail
assert 'activateSection("section-docs")' in admin_html
assert "loadDocDetail" in admin_html
# 状态统计文案
assert "进行中" in admin_html
def test_admin_page_batch_upload(admin_html: str) -> None:
"""批量上传:input multiple 属性、upload-batch 端点、批量结果展示、自动切进度区块"""
# file input multiple 属性
assert 'id="upload-file"' in admin_html
assert "multiple" in admin_html
# 批量上传分支:files > 1 时走 upload-batch 端点
assert "/api/v1/documents/upload-batch" in admin_html
assert 'fileInput.files.length > 1' in admin_html
# FormData 用 files 字段逐文件 append
assert 'batchForm.append("files"' in admin_html
# 批量结果展示函数
assert "function renderBatchUploadResult" in admin_html
assert "批量上传完成" in admin_html
# 成功后自动切到入库进度区块
assert 'activateSection("section-progress")' in admin_html
# 单文件上传仍走原 upload 端点(保持既有逻辑)
assert "/api/v1/documents/upload\"" in admin_html or "/api/v1/documents/upload'," in admin_html
def test_admin_page_no_prompt_confirm_calls(admin_html: str) -> None:
"""清理:弹窗组件已替换原生 prompt/confirm,全页面无 prompt( 与 confirm( 调用残留"""
assert "prompt(" not in admin_html
assert "confirm(" not in admin_html
def test_admin_page_login_card(admin_html: str) -> None: def test_admin_page_login_card(admin_html: str) -> None:
@@ -193,8 +309,52 @@ def test_admin_page_password_form(admin_html: str) -> None:
assert "passwordForced" in admin_html assert "passwordForced" in admin_html
def test_admin_page_profile_section(admin_html: str) -> None:
"""个人中心区块:导航按钮、section、账号信息卡片、改密表单、退出按钮、
user 角色默认进入、GET /auth/me 与 POST /auth/password 路径"""
# section 与导航按钮(不限 admin,所有登录用户可见)
assert 'id="section-profile"' in admin_html
assert 'id="nav-profile"' in admin_html
assert 'data-target="section-profile"' in admin_html
assert "个人中心" in admin_html
# 账号信息卡片字段:用户名 / 角色 / 创建时间 / 须改密 / 启用状态
assert 'id="profile-cards"' in admin_html
for label in ("用户名", "角色", "创建时间", "须改密", "启用状态"):
assert label in admin_html
# 改密表单:旧密码 / 新密码 / 确认 / 强度提示 / 提交按钮
assert 'id="profile-password-form"' in admin_html
assert 'id="profile-old-password"' in admin_html
assert 'id="profile-new-password"' in admin_html
assert 'id="profile-confirm-password"' in admin_html
assert 'id="profile-strength-hint"' in admin_html
assert 'id="btn-profile-password-submit"' in admin_html
# 强度提示复用 Task 3 文案(<8 弱 / ≥8 中 / ≥12 强)
assert "密码强度:" in admin_html
for level in ("", "", ""):
assert level in admin_html
# 两次不一致本地拦截
assert "两次输入的新密码不一致" in admin_html
# 提交按钮 loading 态
assert "提交中…" in admin_html
# 退出登录按钮(复用现有 logout 逻辑)
assert 'id="btn-profile-logout"' in admin_html
assert "退出登录" in admin_html
assert "doLogout" in admin_html
# 数据来源:GET /auth/me 加载账号信息;POST /auth/password 改密
assert "/api/v1/auth/me" in admin_html
assert "/api/v1/auth/password" in admin_html
# 个人中心加载与渲染函数
assert "loadProfile" in admin_html
assert "renderProfile" in admin_html
# user 角色默认进个人中心(admin 仍默认进概览)
assert 'activateSection("section-profile")' in admin_html
assert 'activateSection("section-overview")' in admin_html
# activateSection 触发 loadProfile
assert 'targetId === "section-profile"' in admin_html
def test_admin_page_users_section(admin_html: str) -> None: def test_admin_page_users_section(admin_html: str) -> None:
"""用户管理区块:用户列表/创建表单/角色下拉/重置密码/删除 confirm/角色门禁""" """用户管理区块:用户列表/创建表单/角色下拉/重置密码弹窗/删除确认弹窗/角色门禁"""
assert 'id="section-users"' in admin_html assert 'id="section-users"' in admin_html
assert 'id="users-tbody"' in admin_html assert 'id="users-tbody"' in admin_html
# 表头列:用户名/角色/须改密/创建时间/操作 # 表头列:用户名/角色/须改密/创建时间/操作
@@ -208,10 +368,10 @@ def test_admin_page_users_section(admin_html: str) -> None:
assert '<option value="admin"' in admin_html assert '<option value="admin"' in admin_html
assert '<option value="user"' in admin_html assert '<option value="user"' in admin_html
assert "创建用户" in admin_html assert "创建用户" in admin_html
# 操作列:重置密码(弹输入)与删除(confirm 二次确认 # 操作列:重置密码 + 删除均走通用弹窗组件(无 prompt/confirm 调用
assert "重置密码" in admin_html assert "重置密码" in admin_html
assert "prompt(" in admin_html assert "resetUserPassword" in admin_html
assert "确定删除用户" in admin_html assert "deleteUser" in admin_html
# 用户管理端点 # 用户管理端点
assert "/api/v1/auth/users" in admin_html assert "/api/v1/auth/users" in admin_html
# 角色门禁:仅 admin 挂载该区块进 DOM,非 admin 完全不渲染 # 角色门禁:仅 admin 挂载该区块进 DOM,非 admin 完全不渲染
@@ -220,6 +380,115 @@ def test_admin_page_users_section(admin_html: str) -> None:
assert "unmountUsersSection" in admin_html assert "unmountUsersSection" in admin_html
def test_admin_page_users_filter_and_inline_edit(admin_html: str) -> None:
"""用户管理:列表搜索筛选 + 行内角色编辑 + 启用开关 + 当前 admin 自身行防降级"""
# 列表搜索筛选:用户名搜索框 + 角色筛选下拉
assert 'id="user-search"' in admin_html
assert 'id="user-role-filter"' in admin_html
assert '<option value="all"' in admin_html
# 输入/变更事件触发本地过滤(不重新请求后端)
assert 'addEventListener("input", applyUsersFilter)' in admin_html
assert 'addEventListener("change", applyUsersFilter)' in admin_html
assert "applyUsersFilter" in admin_html
# 表头加「启用」列
assert "启用" in admin_html
# 行内角色编辑:select + 保存按钮(仅当值改变时启用)
assert "user-role-select" in admin_html
assert "user-role-save" in admin_html
assert "updateUserRole" in admin_html
# 行内启用/禁用开关:按钮文字 + 样式区分
assert "user-enabled-toggle" in admin_html
assert "toggleUserEnabled" in admin_html
# PATCH /auth/users/{username} 端点调用(role 与 enabled 两条分支)
assert 'method: "PATCH"' in admin_html
assert "encodeURIComponent(username)" in admin_html
# 当前登录 admin 自身行防自锁降级:role select disabled + 启用开关 disabled
assert "isSelf" in admin_html
assert "不能禁用当前登录账号" in admin_html
def test_admin_page_modal_component(admin_html: str) -> None:
"""通用 overlay 弹窗组件:modal-overlay/modal-card 结构 + openModal/closeModal API"""
# CSS 类
assert "modal-overlay" in admin_html
assert "modal-card" in admin_html
assert "modal-title" in admin_html
assert "modal-content" in admin_html
assert "modal-actions" in admin_html
# 挂载点容器
assert 'id="modal-root"' in admin_html
# 通用 JS API
assert "function openModal(" in admin_html
assert "function closeModal(" in admin_html
assert "function isModalSubmitting(" in admin_html
assert "function setModalSubmitting(" in admin_html
assert "function showModelError(" in admin_html
assert "function hideModelError(" in admin_html
# 提交中标记属性
assert "data-submitting" in admin_html
def test_admin_page_modal_cancel_and_loading(admin_html: str) -> None:
"""弹窗可取消(点遮罩/取消按钮/ESC)+ 提交中不可取消 + loading 态(提交中禁用)"""
# 点遮罩关闭:e.target === overlay
assert "e.target === overlay" in admin_html
# 取消按钮存在
assert "取消" in admin_html
# ESC 键关闭最顶层弹窗
assert "Escape" in admin_html
# 提交中不可关闭:遮罩点击与 ESC 均检查 data-submitting !== "true"
assert 'data-submitting") !== "true"' in admin_html
assert 'data-submitting") === "true"' in admin_html
# 提交 loading 文案 + 按钮禁用
assert "提交中…" in admin_html
assert 'submitBtn.disabled = true' in admin_html
# isModalSubmitting 守卫:取消按钮在提交中不响应
assert "isModalSubmitting(" in admin_html
def test_admin_page_reset_password_modal(admin_html: str) -> None:
"""重置密码弹窗:新密码/确认/强度提示/两次不一致本地拦截/loading 态"""
# 弹窗标题含用户名占位
assert "重置用户" in admin_html
assert "密码" in admin_html
# 新密码 + 确认密码输入框(动态创建,id 经 JS 属性赋值)
assert 'newInput.id = "modal-reset-new"' in admin_html
assert 'confirmInput.id = "modal-reset-confirm"' in admin_html
# 密码强度提示节点
assert 'hint.id = "modal-reset-strength"' in admin_html
assert "modal-strength-hint" in admin_html
# 强度三档纯文案:<8 弱 / ≥8 中 / ≥12 强
assert "密码强度:" in admin_html
for level in ("", "", ""):
assert level in admin_html
# 两次不一致本地拦截
assert "两次输入的新密码不一致" in admin_html
# 重置密码端点
assert "/api/v1/auth/users/" in admin_html
assert "/password" in admin_html
# 提交按钮 loading 态
assert "setModalSubmitting" in admin_html
def test_admin_page_delete_user_modal(admin_html: str) -> None:
"""删除确认弹窗:警告文案 + 输入用户名匹配才可提交 + loading 态"""
# 弹窗标题
assert "删除用户" in admin_html
# 警告文案
assert "此操作不可恢复,将清除该用户及其全部会话" in admin_html
# 「请输入用户名 {username} 以确认」+ 文本输入框(动态创建,id 经 JS 属性赋值)
assert "请输入用户名" in admin_html
assert "以确认" in admin_html
assert 'input.id = "modal-delete-input"' in admin_html
# 输入 !== username 时删除按钮禁用;匹配后启用
assert "submitBtn.disabled = true" in admin_html
assert "input.value !== username" in admin_html
# 删除端点
assert 'method: "DELETE"' in admin_html
# 提交按钮 loading 态
assert "setModalSubmitting" in admin_html
def test_admin_page_auth_interceptor(admin_html: str) -> None: def test_admin_page_auth_interceptor(admin_html: str) -> None:
"""请求拦截:统一注入 Authorization Bearer1005 回登录;1006 错误条提示""" """请求拦截:统一注入 Authorization Bearer1005 回登录;1006 错误条提示"""
assert 'options.headers["Authorization"] = "Bearer " + token' in admin_html assert 'options.headers["Authorization"] = "Bearer " + token' in admin_html
@@ -231,6 +500,22 @@ def test_admin_page_auth_interceptor(admin_html: str) -> None:
assert "1006" in admin_html assert "1006" in admin_html
def test_admin_page_doc_detail_file_link(admin_html: str) -> None:
"""文档详情区:有关联文件时展示原始文件行与下载链接(textContent 防 XSS"""
# 条件判断与字段访问
assert "data.file" in admin_html
assert "file.filename" in admin_html
assert "file.size_bytes" in admin_html
# 大小格式化辅助
assert "formatFileSize" in admin_html
# 下载链接用 el("a") 创建,设 href 与 download 属性,不 innerHTML
assert 'el("a"' in admin_html
assert "download" in admin_html
# 行标签文案
assert "原始文件" in admin_html
assert "下载" in admin_html
def test_admin_page_api_guide_section(admin_html: str) -> None: def test_admin_page_api_guide_section(admin_html: str) -> None:
"""API 指南区块:导航按钮、section、静态清单与鉴权标注(user 角色也可见)""" """API 指南区块:导航按钮、section、静态清单与鉴权标注(user 角色也可见)"""
assert 'data-target="section-api-guide"' in admin_html assert 'data-target="section-api-guide"' in admin_html
@@ -320,15 +605,19 @@ def test_admin_page_api_guide_paths_in_real_routes(admin_html: str) -> None:
assert (method, path) in route_methods, f"API_GUIDE 端点 {method} {path} 不在后端路由集合内" assert (method, path) in route_methods, f"API_GUIDE 端点 {method} {path} 不在后端路由集合内"
# 全部真实端点覆盖:health/search/documents*/knowledge*/auth* # 全部真实端点覆盖:health/search/documents*/knowledge*/auth*
# Task 2 新增端点:upload-batch / tasks 列表 / reingest
expected = { expected = {
("GET", "/api/v1/health"), ("GET", "/api/v1/health"),
("POST", "/api/v1/search"), ("POST", "/api/v1/search"),
("POST", "/api/v1/documents"), ("POST", "/api/v1/documents"),
("POST", "/api/v1/documents/upload"), ("POST", "/api/v1/documents/upload"),
("POST", "/api/v1/documents/upload-batch"),
("GET", "/api/v1/documents/tasks"),
("GET", "/api/v1/documents/tasks/{task_id}"), ("GET", "/api/v1/documents/tasks/{task_id}"),
("GET", "/api/v1/documents"), ("GET", "/api/v1/documents"),
("GET", "/api/v1/documents/{doc_id}"), ("GET", "/api/v1/documents/{doc_id}"),
("DELETE", "/api/v1/documents/{doc_id}"), ("DELETE", "/api/v1/documents/{doc_id}"),
("POST", "/api/v1/documents/{doc_id}/reingest"),
("GET", "/api/v1/knowledge/categories"), ("GET", "/api/v1/knowledge/categories"),
("GET", "/api/v1/knowledge/stats"), ("GET", "/api/v1/knowledge/stats"),
("POST", "/api/v1/auth/login"), ("POST", "/api/v1/auth/login"),
+168 -2
View File
@@ -245,8 +245,9 @@ class TestUsersCrud:
users = body["data"] users = body["data"]
assert len(users) == 1 assert len(users) == 1
admin = users[0] admin = users[0]
assert set(admin.keys()) == {"username", "role", "must_change_password", "created_at"} assert set(admin.keys()) == {"username", "role", "must_change_password", "enabled", "created_at"}
assert admin["username"] == "admin" assert admin["username"] == "admin"
assert admin["enabled"] is True
assert "password_hash" not in admin assert "password_hash" not in admin
assert "salt" not in admin assert "salt" not in admin
@@ -271,10 +272,11 @@ class TestUsersCrud:
assert body["code"] == 0 assert body["code"] == 0
data = body["data"] data = body["data"]
assert set(data.keys()) == {"username", "role", "must_change_password", "created_at"} assert set(data.keys()) == {"username", "role", "must_change_password", "enabled", "created_at"}
assert data["username"] == "carol" assert data["username"] == "carol"
assert data["role"] == "user" assert data["role"] == "user"
assert data["must_change_password"] is False assert data["must_change_password"] is False
assert data["enabled"] is True
# 新用户可登录 # 新用户可登录
assert _login(client, "carol", "carol-pass-123")["code"] == 0 assert _login(client, "carol", "carol-pass-123")["code"] == 0
@@ -386,6 +388,170 @@ class TestUsersCrud:
assert body["code"] == 1001 assert body["code"] == 1001
class TestUpdateUser:
"""PATCH /api/v1/auth/users/{username}:更新角色/启用状态"""
def test_update_role_success(self, client: TestClient, admin_headers: dict[str, str]) -> None:
client.post(
"/api/v1/auth/users",
json={"username": "carol", "password": "carol-pass-123"},
headers=admin_headers,
)
body = client.patch(
"/api/v1/auth/users/carol", json={"role": "admin"}, headers=admin_headers
).json()
assert body["code"] == 0
assert body["data"]["role"] == "admin"
assert body["data"]["enabled"] is True
def test_update_enabled_success(
self, client: TestClient, auth_stores, admin_headers: dict[str, str]
) -> None:
user_store, _ = auth_stores
_create_user(user_store, "dave", "dave-pass-123", role="user")
body = client.patch(
"/api/v1/auth/users/dave", json={"enabled": False}, headers=admin_headers
).json()
assert body["code"] == 0
assert body["data"]["enabled"] is False
assert body["data"]["role"] == "user"
def test_update_both_role_and_enabled(
self, client: TestClient, auth_stores, admin_headers: dict[str, str]
) -> None:
user_store, _ = auth_stores
_create_user(user_store, "erin", "erin-pass-123", role="user")
body = client.patch(
"/api/v1/auth/users/erin",
json={"role": "admin", "enabled": False},
headers=admin_headers,
).json()
assert body["code"] == 0
assert body["data"]["role"] == "admin"
assert body["data"]["enabled"] is False
def test_at_least_one_field_required(self, client: TestClient, admin_headers: dict[str, str]) -> None:
body = client.patch(
"/api/v1/auth/users/admin", json={}, headers=admin_headers
).json()
assert body["code"] == 1001
assert body["data"] is None
def test_user_not_found(self, client: TestClient, admin_headers: dict[str, str]) -> None:
body = client.patch(
"/api/v1/auth/users/ghost", json={"role": "user"}, headers=admin_headers
).json()
assert body["code"] == 1004
assert body["data"] is None
def test_invalid_role(self, client: TestClient, admin_headers: dict[str, str]) -> None:
body = client.patch(
"/api/v1/auth/users/admin", json={"role": "superuser"}, headers=admin_headers
).json()
assert body["code"] == 1001
def test_last_admin_demote_forbidden(self, client: TestClient, admin_headers: dict[str, str]) -> None:
# 唯一 admin 降级为 user → 1001
body = client.patch(
"/api/v1/auth/users/admin", json={"role": "user"}, headers=admin_headers
).json()
assert body["code"] == 1001
# 未生效
assert client.get("/api/v1/auth/users", headers=admin_headers).json()["code"] == 0
def test_last_admin_disable_forbidden(self, client: TestClient, admin_headers: dict[str, str]) -> None:
# 唯一 admin 禁用 → 1001
body = client.patch(
"/api/v1/auth/users/admin", json={"enabled": False}, headers=admin_headers
).json()
assert body["code"] == 1001
def test_non_admin_forbidden(self, client: TestClient, auth_stores) -> None:
user_store, session_store = auth_stores
_create_user(user_store, "bob", "bob-pass-123", role="user")
headers = _headers(session_store, "bob", "user")
body = client.patch(
"/api/v1/auth/users/bob", json={"role": "admin"}, headers=headers
).json()
assert body["code"] == 1006
def test_demote_when_multiple_admins_ok(
self, client: TestClient, auth_stores, admin_headers: dict[str, str]
) -> None:
# 存在第二个 admin 时,可降级其中一个
user_store, _ = auth_stores
_create_user(user_store, "admin2", "admin2-pass-123", role="admin")
body = client.patch(
"/api/v1/auth/users/admin2", json={"role": "user"}, headers=admin_headers
).json()
assert body["code"] == 0
assert body["data"]["role"] == "user"
class TestDisabledUser:
"""禁用用户:登录拦截 + 旧 token 失效 + 业务端点拦截"""
def test_disabled_user_login_blocked(
self, client: TestClient, auth_stores, admin_headers: dict[str, str]
) -> None:
user_store, _ = auth_stores
_create_user(user_store, "bob", "bob-pass-123", role="user")
# 禁用 bob
client.patch(
"/api/v1/auth/users/bob", json={"enabled": False}, headers=admin_headers
)
# 登录 → 1005 账号已禁用
body = _login(client, "bob", "bob-pass-123")
assert body["code"] == 1005
assert body["data"] is None
def test_disabled_user_old_token_invalidated(
self, client: TestClient, auth_stores, admin_headers: dict[str, str]
) -> None:
user_store, _ = auth_stores
_create_user(user_store, "bob", "bob-pass-123", role="user")
# bob 登录拿到 token
login_body = _login(client, "bob", "bob-pass-123")
assert login_body["code"] == 0
bob_headers = {"Authorization": f"Bearer {login_body['data']['token']}"}
# 禁用 bob → session 清除
client.patch(
"/api/v1/auth/users/bob", json={"enabled": False}, headers=admin_headers
)
# 旧 token 调业务端点 → 1005session 已清)
body = client.get("/api/v1/auth/me", headers=bob_headers).json()
assert body["code"] == 1005
def test_disabled_user_blocked_even_with_valid_session(
self, client: TestClient, auth_stores, admin_headers: dict[str, str]
) -> None:
"""禁用用户即使持有效 session(直接签发绕过登录),业务端点仍拦截 1005"""
user_store, session_store = auth_stores
_create_user(user_store, "bob", "bob-pass-123", role="user")
# 禁用 bob → session 清除
client.patch(
"/api/v1/auth/users/bob", json={"enabled": False}, headers=admin_headers
)
# 直接为 bob 签发新 session 绕过登录与清理,验证 get_current_user 的 enabled 拦截
bob_headers = _headers(session_store, "bob", "user")
body = client.get("/api/v1/auth/me", headers=bob_headers).json()
assert body["code"] == 1005
class TestDocumentAuth: class TestDocumentAuth:
"""文档端点鉴权:变更类需登录,GET 系列免登录""" """文档端点鉴权:变更类需登录,GET 系列免登录"""
+335
View File
@@ -0,0 +1,335 @@
"""POST /documents/upload-batch、POST /documents/{id}/reingest、GET /documents/tasks 端点测试
TestClient + FakeManager + FakeQdrant,不真实联网。
"""
from collections.abc import Iterator
from pathlib import Path
from typing import Any
import pytest
from fastapi.testclient import TestClient
from app.api.v1 import document as document_module
from app.main import app
from app.models.document import DocumentInput
from app.services.qdrant import QdrantService
class FakeManager:
"""假入库任务管理器:支持 submit/get/list_tasks"""
def __init__(self, task_id: str = "task-1") -> None:
self.task_id = task_id
self._counter = 0
self.submitted: list[DocumentInput] = []
self._tasks: list[dict[str, Any]] = []
async def submit(self, doc: DocumentInput) -> str:
self.submitted.append(doc)
self._counter += 1
tid = f"{self.task_id}-{self._counter}"
self._tasks.append(
{
"task_id": tid,
"status": "pending",
"filename": doc.metadata.get("original_filename") or doc.title,
"created_at": f"2026-01-0{self._counter}T00:00:00+00:00",
"updated_at": f"2026-01-0{self._counter}T00:00:00+00:00",
"doc_id": None,
}
)
return tid
async def get(self, task_id: str) -> dict[str, Any] | None:
return None
async def list_tasks(self, limit: int = 20) -> list[dict[str, Any]]:
return list(self._tasks[:limit])
class FakeQdrant:
"""假 Qdrant 服务:可控的 get_l1_metadata/delete_by_doc_id"""
def __init__(self, meta: dict[str, str] | None = None) -> None:
self.meta = meta
self.deleted: list[str] = []
async def get_l1_metadata(self, doc_id: str) -> dict[str, str] | None:
return self.meta
async def delete_by_doc_id(self, doc_id: str) -> dict[str, int]:
self.deleted.append(doc_id)
return {"doc_l1": 1}
@pytest.fixture
def client(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path, admin_headers: dict[str, str]
) -> Iterator[TestClient]:
async def _noop_ensure_collections(self: QdrantService) -> None:
return None
monkeypatch.setattr(QdrantService, "ensure_collections", _noop_ensure_collections)
monkeypatch.setattr(document_module.settings, "upload_dir", str(tmp_path / "uploads"))
with TestClient(app) as test_client:
test_client.headers.update(admin_headers)
yield test_client
def _inject_manager(monkeypatch: pytest.MonkeyPatch, manager: FakeManager) -> None:
monkeypatch.setattr(document_module, "_get_task_manager", lambda: manager)
def _inject_qdrant(monkeypatch: pytest.MonkeyPatch, qdrant: FakeQdrant) -> None:
monkeypatch.setattr(document_module, "_get_qdrant", lambda: qdrant)
# ----------------------- upload-batch -----------------------
def test_batch_upload_multiple_files_success(
client: TestClient, monkeypatch: pytest.MonkeyPatch
) -> None:
"""多文件全部成功:3 文件 → 3 tasksfailed 为空"""
manager = FakeManager()
_inject_manager(monkeypatch, manager)
resp = client.post(
"/api/v1/documents/upload-batch",
files=[
("files", ("a.txt", b"content a", "text/plain")),
("files", ("b.md", b"# B", "text/markdown")),
("files", ("c.txt", b"content c", "text/plain")),
],
)
assert resp.status_code == 202
data = resp.json()["data"]
assert len(data["tasks"]) == 3
assert data["failed"] == []
assert len(manager.submitted) == 3
filenames = [t["filename"] for t in data["tasks"]]
assert filenames == ["a.txt", "b.md", "c.txt"]
def test_batch_upload_partial_failure(
client: TestClient, monkeypatch: pytest.MonkeyPatch
) -> None:
"""部分失败:1 个不支持扩展名进 failed,其他成功"""
manager = FakeManager()
_inject_manager(monkeypatch, manager)
resp = client.post(
"/api/v1/documents/upload-batch",
files=[
("files", ("ok.txt", b"good", "text/plain")),
("files", ("bad.xlsx", b"binary", "application/octet-stream")),
],
)
assert resp.status_code == 202
data = resp.json()["data"]
assert len(data["tasks"]) == 1
assert data["tasks"][0]["filename"] == "ok.txt"
assert len(data["failed"]) == 1
assert data["failed"][0]["filename"] == "bad.xlsx"
assert "不支持" in data["failed"][0]["error"]
assert len(manager.submitted) == 1
def test_batch_upload_all_fail(
client: TestClient, monkeypatch: pytest.MonkeyPatch
) -> None:
"""全部失败:tasks 空,failed 2 条,未提交任何任务"""
manager = FakeManager()
_inject_manager(monkeypatch, manager)
resp = client.post(
"/api/v1/documents/upload-batch",
files=[
("files", ("a.exe", b"x", "application/octet-stream")),
("files", ("b.bin", b"y", "application/octet-stream")),
],
)
assert resp.status_code == 202
data = resp.json()["data"]
assert data["tasks"] == []
assert len(data["failed"]) == 2
assert len(manager.submitted) == 0
def test_batch_upload_empty_list_returns_1001(
client: TestClient, monkeypatch: pytest.MonkeyPatch
) -> None:
"""空文件列表:1001"""
manager = FakeManager()
_inject_manager(monkeypatch, manager)
resp = client.post("/api/v1/documents/upload-batch", files=[])
body = resp.json()
assert body["code"] == 1001
assert manager.submitted == []
# ----------------------- reingest -----------------------
def test_reingest_success(
client: TestClient, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""有原始文件:202 + task_id,删旧数据后提交新任务"""
upload_dir = tmp_path / "uploads"
upload_dir.mkdir(parents=True, exist_ok=True)
raw = upload_dir / "raw_report.txt"
raw.write_bytes(b"original content")
meta = {
"raw_file_path": str(raw),
"original_filename": "raw_report.txt",
}
qdrant = FakeQdrant(meta=meta)
_inject_qdrant(monkeypatch, qdrant)
manager = FakeManager(task_id="reingest-task")
_inject_manager(monkeypatch, manager)
resp = client.post("/api/v1/documents/some-doc/reingest")
assert resp.status_code == 202
data = resp.json()["data"]
assert data["task_id"] == "reingest-task-1"
assert data["status"] == "pending"
# 旧数据已删
assert qdrant.deleted == ["some-doc"]
# 新任务已提交,文本来自原文件
assert len(manager.submitted) == 1
assert manager.submitted[0].text == "original content"
assert manager.submitted[0].title == "raw_report" # 文件名 stem
def test_reingest_no_raw_file_returns_1001(
client: TestClient, monkeypatch: pytest.MonkeyPatch
) -> None:
"""无原始文件(meta 无 raw_file_path):1001"""
qdrant = FakeQdrant(meta={"some": "metadata"})
_inject_qdrant(monkeypatch, qdrant)
manager = FakeManager()
_inject_manager(monkeypatch, manager)
resp = client.post("/api/v1/documents/doc1/reingest")
body = resp.json()
assert body["code"] == 1001
assert "无原始文件" in body["message"]
assert len(manager.submitted) == 0
def test_reingest_missing_file_returns_1004(
client: TestClient, monkeypatch: pytest.MonkeyPatch
) -> None:
"""文件不存在:1004"""
qdrant = FakeQdrant(
meta={"raw_file_path": "/nonexistent/path.txt", "original_filename": "path.txt"}
)
_inject_qdrant(monkeypatch, qdrant)
manager = FakeManager()
_inject_manager(monkeypatch, manager)
resp = client.post("/api/v1/documents/doc1/reingest")
body = resp.json()
assert body["code"] == 1004
assert len(manager.submitted) == 0
def test_reingest_doc_not_found_returns_1004(
client: TestClient, monkeypatch: pytest.MonkeyPatch
) -> None:
"""文档不存在(meta 为 None):1004"""
qdrant = FakeQdrant(meta=None)
_inject_qdrant(monkeypatch, qdrant)
manager = FakeManager()
_inject_manager(monkeypatch, manager)
resp = client.post("/api/v1/documents/nope/reingest")
body = resp.json()
assert body["code"] == 1004
assert len(manager.submitted) == 0
# ----------------------- tasks 列表 -----------------------
def test_list_ingest_tasks_returns_items(
client: TestClient, monkeypatch: pytest.MonkeyPatch
) -> None:
"""任务列表:返回近期任务,含 items 与 total"""
manager = FakeManager()
_inject_manager(monkeypatch, manager)
manager._tasks = [
{
"task_id": "t1",
"status": "done",
"filename": "a.txt",
"created_at": "2026-01-01T00:00:00+00:00",
"updated_at": "2026-01-01T00:00:00+00:00",
"doc_id": "d1",
},
{
"task_id": "t2",
"status": "failed",
"filename": "b.txt",
"created_at": "2026-01-02T00:00:00+00:00",
"updated_at": "2026-01-02T00:00:00+00:00",
"doc_id": None,
},
]
resp = client.get("/api/v1/documents/tasks")
assert resp.status_code == 200
data = resp.json()["data"]
assert data["total"] == 2
assert len(data["items"]) == 2
assert data["items"][0]["task_id"] == "t1"
assert data["items"][0]["doc_id"] == "d1"
def test_list_ingest_tasks_limit(
client: TestClient, monkeypatch: pytest.MonkeyPatch
) -> None:
"""limit 截断生效"""
manager = FakeManager()
_inject_manager(monkeypatch, manager)
manager._tasks = [
{
"task_id": f"t{i}",
"status": "done",
"filename": "f",
"created_at": "",
"updated_at": "",
"doc_id": None,
}
for i in range(10)
]
resp = client.get("/api/v1/documents/tasks?limit=5")
data = resp.json()["data"]
assert data["total"] == 5
assert len(data["items"]) == 5
def test_list_ingest_tasks_requires_auth(
client: TestClient, monkeypatch: pytest.MonkeyPatch
) -> None:
"""无 token1005 未认证"""
manager = FakeManager()
_inject_manager(monkeypatch, manager)
resp = client.get("/api/v1/documents/tasks", headers={"Authorization": ""})
body = resp.json()
assert body["code"] == 1005
+339
View File
@@ -0,0 +1,339 @@
"""批量上传与入库进度集成验证
在真实内存 Qdrant + FakeOllama + 确定性向量环境下,全链路验证:
- POST /documents/upload-batch 批量上传闭环(全部成功 / 部分失败)
- GET /documents/tasks 任务列表(按 updated_at 降序、每项含 filename/doc_id
- POST /documents/{doc_id}/reingest 重新入库(成功 / 边界 1001/1004
复用 tests/test_e2e_integration.py 的 FakeOllama / DeterministicEmbedding / FakeCache
upload_dir 用 tmp_path 真实落盘,monkeypatch 模块级 _task_manager / _qdrant 单例。
"""
from pathlib import Path
import pytest
from fastapi.testclient import TestClient
from qdrant_client import AsyncQdrantClient
from app.api.v1 import document as document_module
from app.config import Settings
from app.core.chunker import Chunker
from app.core.classifier import Classifier
from app.core.ingest_tasks import IngestTaskManager, IngestTaskStatus
from app.core.ingestion import Ingester
from app.core.query_parser import QueryParser
from app.core.retriever import Retriever
from app.core.sparse import SparseEncoder
from app.core.summarizer import Summarizer
from app.main import app
from app.models.knowledge import load_taxonomy
from app.services.qdrant import COLLECTION_CHUNKS, QdrantService
from tests.test_e2e_integration import (
FAKE_L1_SUMMARY,
DeterministicEmbedding,
FakeCache,
FakeOllama,
)
async def _make_env() -> tuple[QdrantService, Ingester, Retriever]:
"""构建集成环境:真实组件 + 内存 Qdrant + FakeOllama + 确定性向量"""
qdrant = QdrantService(client=AsyncQdrantClient(location=":memory:"))
await qdrant.ensure_collections()
ollama = FakeOllama()
taxonomy = load_taxonomy()
embedding = DeterministicEmbedding()
ingester = Ingester(
summarizer=Summarizer(ollama=ollama), # type: ignore[arg-type]
classifier=Classifier(ollama=ollama, taxonomy=taxonomy), # type: ignore[arg-type]
chunker=Chunker(),
embedding=embedding, # type: ignore[arg-type]
sparse=SparseEncoder(),
qdrant=qdrant,
)
retriever = Retriever(
qdrant=qdrant,
query_parser=QueryParser(ollama=ollama, taxonomy=taxonomy, cache=FakeCache()), # type: ignore[arg-type]
embedding=embedding, # type: ignore[arg-type]
sparse_encoder=SparseEncoder(),
)
return qdrant, ingester, retriever
def _patch_app(
monkeypatch: pytest.MonkeyPatch,
manager: IngestTaskManager,
qdrant: QdrantService,
retriever: Retriever,
upload_dir: Path,
) -> None:
"""替换模块级单例:任务管理器 / Qdrant / Retriever / upload_dirlifespan 建集合改空操作"""
async def _noop_ensure_collections(self: QdrantService) -> None:
return None
monkeypatch.setattr(QdrantService, "ensure_collections", _noop_ensure_collections)
monkeypatch.setattr(document_module, "_task_manager", manager)
monkeypatch.setattr(document_module, "_qdrant", qdrant)
monkeypatch.setattr(document_module.settings, "upload_dir", str(upload_dir))
def _md_content(name: str) -> bytes:
"""构造可触发完整三级总结的 .md 文件内容(不同 name 保证文本不同,避免去重)"""
paragraph = f"这是 {name} 章节的正文内容,包含足够信息量用于测试切分与向量化流程。" * 20
text = f"# {name} 安装指南\n{paragraph}\n\n## 环境准备\n{paragraph}\n\n## 安装步骤\n{paragraph}"
return text.encode("utf-8")
class TestBatchUploadIntegration:
"""批量上传闭环:3 文件全成功 → 任务列表 → 文档详情含原始文件信息"""
async def test_batch_upload_full_loop(
self,
monkeypatch: pytest.MonkeyPatch,
tmp_path: Path,
admin_headers: dict[str, str],
) -> None:
qdrant, ingester, retriever = await _make_env()
manager = IngestTaskManager(ingester, None, Settings())
_patch_app(monkeypatch, manager, qdrant, retriever, tmp_path / "uploads")
with TestClient(app) as client:
# 1. 批量上传 3 个 .md202 + tasks 含 3 个 + failed 空
resp = client.post(
"/api/v1/documents/upload-batch",
files=[
("files", ("a.md", _md_content("Alpha"), "text/markdown")),
("files", ("b.md", _md_content("Beta"), "text/markdown")),
("files", ("c.md", _md_content("Gamma"), "text/markdown")),
],
headers=admin_headers,
)
assert resp.status_code == 202
data = resp.json()["data"]
assert len(data["tasks"]) == 3
assert data["failed"] == []
task_ids = [t["task_id"] for t in data["tasks"]]
filenames = [t["filename"] for t in data["tasks"]]
assert filenames == ["a.md", "b.md", "c.md"]
# 2. 逐个等待终态:全部 done
doc_ids: list[str] = []
for tid in task_ids:
final = await manager.wait_done(tid)
assert final["status"] == IngestTaskStatus.DONE
doc_ids.append(final["result"]["document_id"])
# 3. GET /documents/tasksitems 含 3 项,按 updated_at 降序,每项含 filename/doc_id
resp_list = client.get(
"/api/v1/documents/tasks?limit=10", headers=admin_headers
)
body_list = resp_list.json()
assert body_list["code"] == 0
items = body_list["data"]["items"]
assert body_list["data"]["total"] == 3
assert len(items) == 3
# 按 updated_at 降序
updated = [it["updated_at"] for it in items]
assert updated == sorted(updated, reverse=True)
# 每项含 filename 与 doc_id
for it in items:
assert it["filename"] in {"a.md", "b.md", "c.md"}
assert it["doc_id"] in doc_ids
# 4. GET /documents/{doc_id}file 字段非 null(有原始文件信息)
resp_detail = client.get(f"/api/v1/documents/{doc_ids[0]}")
body_detail = resp_detail.json()
assert body_detail["code"] == 0
file_info = body_detail["data"]["file"]
assert file_info is not None
assert file_info["filename"] == filenames[0]
assert file_info["size_bytes"] > 0
assert file_info["url"] == f"/api/v1/documents/{doc_ids[0]}/file"
class TestBatchUploadPartialFailure:
"""部分失败:1 个 .unsupported 扩展名进 failed,其余 2 个成功"""
async def test_batch_upload_partial_failure(
self,
monkeypatch: pytest.MonkeyPatch,
tmp_path: Path,
admin_headers: dict[str, str],
) -> None:
qdrant, ingester, retriever = await _make_env()
manager = IngestTaskManager(ingester, None, Settings())
_patch_app(monkeypatch, manager, qdrant, retriever, tmp_path / "uploads")
with TestClient(app) as client:
resp = client.post(
"/api/v1/documents/upload-batch",
files=[
("files", ("ok1.md", _md_content("Ok1"), "text/markdown")),
("files", ("ok2.md", _md_content("Ok2"), "text/markdown")),
("files", ("bad.unsupported", b"whatever", "application/octet-stream")),
],
headers=admin_headers,
)
assert resp.status_code == 202
data = resp.json()["data"]
assert len(data["tasks"]) == 2
assert len(data["failed"]) == 1
assert data["failed"][0]["filename"] == "bad.unsupported"
assert "不支持" in data["failed"][0]["error"]
# 2 个成功任务均可进入终态 done
for t in data["tasks"]:
final = await manager.wait_done(t["task_id"])
assert final["status"] == IngestTaskStatus.DONE
class TestReingestIntegration:
"""重新入库:成功路径 + 边界(无原始文件 / 文档不存在 / 文件已删)"""
async def test_reingest_success(
self,
monkeypatch: pytest.MonkeyPatch,
tmp_path: Path,
admin_headers: dict[str, str],
) -> None:
qdrant, ingester, retriever = await _make_env()
manager = IngestTaskManager(ingester, None, Settings())
_patch_app(monkeypatch, manager, qdrant, retriever, tmp_path / "uploads")
with TestClient(app) as client:
# 先上传 1 个文件,拿到 doc_id
resp_up = client.post(
"/api/v1/documents/upload-batch",
files=[("files", ("re.md", _md_content("Reingest"), "text/markdown"))],
headers=admin_headers,
)
assert resp_up.status_code == 202
up_data = resp_up.json()["data"]
assert len(up_data["tasks"]) == 1
old_task_id = up_data["tasks"][0]["task_id"]
old_final = await manager.wait_done(old_task_id)
old_doc_id = old_final["result"]["document_id"]
# 重新入库:202 + task_id
resp_re = client.post(
f"/api/v1/documents/{old_doc_id}/reingest", headers=admin_headers
)
assert resp_re.status_code == 202
re_body = resp_re.json()
assert re_body["code"] == 0
new_task_id = re_body["data"]["task_id"]
assert new_task_id
# 等待终态:done
new_final = await manager.wait_done(new_task_id)
assert new_final["status"] == IngestTaskStatus.DONE
new_doc_id = new_final["result"]["document_id"]
# 旧 doc_id 数据已删:GET /documents/{old_doc_id} → 1004
resp_old = client.get(f"/api/v1/documents/{old_doc_id}")
assert resp_old.json()["code"] == 1004
# 新 doc_id 详情:L1 summary 存在,file 非 null(保留原文件信息)
resp_new = client.get(f"/api/v1/documents/{new_doc_id}")
body_new = resp_new.json()
assert body_new["code"] == 0
assert body_new["data"]["l1"]["text"] == FAKE_L1_SUMMARY
assert body_new["data"]["file"] is not None
assert body_new["data"]["file"]["filename"] == "re.md"
# 旧 chunks 已删:chunks 集合中 old_doc_id 点数为 0
old_count = await qdrant.client.count(
collection_name=COLLECTION_CHUNKS,
count_filter=qdrant.build_filter(doc_ids=[old_doc_id]),
exact=True,
)
assert old_count.count == 0
async def test_reingest_text_only_doc_returns_1001(
self,
monkeypatch: pytest.MonkeyPatch,
tmp_path: Path,
admin_headers: dict[str, str],
) -> None:
"""纯文本入库的 doc_id → reingest → 1001(无原始文件)"""
qdrant, ingester, retriever = await _make_env()
manager = IngestTaskManager(ingester, None, Settings())
_patch_app(monkeypatch, manager, qdrant, retriever, tmp_path / "uploads")
with TestClient(app) as client:
# 纯文本入库(无原始文件落盘记录)
resp_doc = client.post(
"/api/v1/documents",
json={
"text": "# 纯文本\n这是一段纯文本入库的内容,用于测试 reingest 边界。",
"title": "纯文本",
},
headers=admin_headers,
)
assert resp_doc.status_code == 202
task_id = resp_doc.json()["data"]["task_id"]
final = await manager.wait_done(task_id)
doc_id = final["result"]["document_id"]
# reingest → 1001
resp_re = client.post(
f"/api/v1/documents/{doc_id}/reingest", headers=admin_headers
)
body = resp_re.json()
assert body["code"] == 1001
assert "无原始文件" in body["message"]
async def test_reingest_doc_not_found_returns_1004(
self,
monkeypatch: pytest.MonkeyPatch,
tmp_path: Path,
admin_headers: dict[str, str],
) -> None:
"""不存在的 doc_id → reingest → 1004"""
qdrant, ingester, retriever = await _make_env()
manager = IngestTaskManager(ingester, None, Settings())
_patch_app(monkeypatch, manager, qdrant, retriever, tmp_path / "uploads")
with TestClient(app) as client:
resp = client.post(
"/api/v1/documents/nonexistent-doc/reingest", headers=admin_headers
)
body = resp.json()
assert body["code"] == 1004
async def test_reingest_file_deleted_returns_1004(
self,
monkeypatch: pytest.MonkeyPatch,
tmp_path: Path,
admin_headers: dict[str, str],
) -> None:
"""文件被删(删 upload_dir 下落盘文件后)→ reingest → 1004"""
qdrant, ingester, retriever = await _make_env()
manager = IngestTaskManager(ingester, None, Settings())
_patch_app(monkeypatch, manager, qdrant, retriever, tmp_path / "uploads")
with TestClient(app) as client:
# 上传 1 个文件
resp_up = client.post(
"/api/v1/documents/upload-batch",
files=[("files", ("del.md", _md_content("Deleted"), "text/markdown"))],
headers=admin_headers,
)
assert resp_up.status_code == 202
tid = resp_up.json()["data"]["tasks"][0]["task_id"]
final = await manager.wait_done(tid)
doc_id = final["result"]["document_id"]
# 删除 upload_dir 下落盘的原始文件
meta = await qdrant.get_l1_metadata(doc_id)
assert meta is not None
raw_path = Path(meta["raw_file_path"])
assert raw_path.is_file()
raw_path.unlink()
# reingest → 1004(文件不存在)
resp_re = client.post(
f"/api/v1/documents/{doc_id}/reingest", headers=admin_headers
)
body = resp_re.json()
assert body["code"] == 1004
+169
View File
@@ -0,0 +1,169 @@
"""GET /api/v1/documents/{doc_id}/file 文件下载端点测试(TestClient + FakeQdrant,不真实联网)
覆盖分支:成功下载、文档不存在、无关联文件、文件已从磁盘删除、路径越界、免登录访问。
"""
from collections.abc import Iterator
from pathlib import Path
import pytest
from fastapi.testclient import TestClient
from app.api.v1 import document as document_module
from app.main import app
from app.services.qdrant import QdrantService
class FakeQdrant:
"""假 QdrantService:仅实现 get_l1_metadata,按 doc_id 返回固定 metadata 或 None"""
def __init__(self, meta: dict[str, str] | None = None) -> None:
self.meta = meta
async def get_l1_metadata(self, doc_id: str) -> dict[str, str] | None:
return self.meta
@pytest.fixture
def client(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> Iterator[TestClient]:
"""TestClientQdrant 集合初始化空操作;upload_dir 指向临时目录
不挂 admin_headers:下载端点免登录,验证无 token 可访问。
"""
async def _noop_ensure_collections(self: QdrantService) -> None:
return None
monkeypatch.setattr(QdrantService, "ensure_collections", _noop_ensure_collections)
monkeypatch.setattr(document_module.settings, "upload_dir", str(tmp_path / "uploads"))
with TestClient(app) as test_client:
yield test_client
def _install_fake(monkeypatch: pytest.MonkeyPatch, fake: FakeQdrant) -> None:
"""将 _get_qdrant 单例替换为假服务"""
monkeypatch.setattr(document_module, "_get_qdrant", lambda: fake)
def test_download_file_success(
client: TestClient, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""文档存在且有关联文件 → 200 + 文件内容正确 + content-disposition 含文件名"""
upload_dir = tmp_path / "uploads"
upload_dir.mkdir(parents=True, exist_ok=True)
target = upload_dir / "doc-1_notes.txt"
content = b"hello world file content"
target.write_bytes(content)
fake = FakeQdrant(
meta={
"raw_file_path": str(target),
"original_filename": "notes.txt",
"original_size_bytes": str(len(content)),
}
)
_install_fake(monkeypatch, fake)
resp = client.get("/api/v1/documents/doc-1/file")
assert resp.status_code == 200
assert resp.content == content
disposition = resp.headers.get("content-disposition", "")
assert "attachment" in disposition
assert "notes.txt" in disposition
def test_download_file_doc_not_found(
client: TestClient, monkeypatch: pytest.MonkeyPatch
) -> None:
"""文档不存在(metadata 为 None)→ 1004"""
fake = FakeQdrant(meta=None)
_install_fake(monkeypatch, fake)
resp = client.get("/api/v1/documents/missing/file")
body = resp.json()
assert body["code"] == 1004
assert "文件" in body["message"]
def test_download_file_no_associated_file(
client: TestClient, monkeypatch: pytest.MonkeyPatch
) -> None:
"""文档存在但 raw_file_path 为空 → 1004"""
fake = FakeQdrant(meta={"raw_file_path": "", "original_filename": "x.txt"})
_install_fake(monkeypatch, fake)
resp = client.get("/api/v1/documents/doc-1/file")
body = resp.json()
assert body["code"] == 1004
assert "未关联文件" in body["message"]
def test_download_file_missing_on_disk(
client: TestClient, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""文件已从磁盘删除(path.is_file() False)→ 1004"""
missing = tmp_path / "uploads" / "gone.txt"
fake = FakeQdrant(meta={"raw_file_path": str(missing), "original_filename": "gone.txt"})
_install_fake(monkeypatch, fake)
resp = client.get("/api/v1/documents/doc-1/file")
body = resp.json()
assert body["code"] == 1004
assert "文件不存在" in body["message"]
def test_download_file_path_traversal_rejected(
client: TestClient, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""raw_file_path 指向 upload_dir 之外 → 1004(路径越界校验)"""
outside = tmp_path / "secret.txt"
outside.write_bytes(b"secret")
fake = FakeQdrant(meta={"raw_file_path": str(outside), "original_filename": "secret.txt"})
_install_fake(monkeypatch, fake)
resp = client.get("/api/v1/documents/doc-1/file")
body = resp.json()
assert body["code"] == 1004
assert "文件不存在" in body["message"]
def test_download_file_no_auth_required(
client: TestClient, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""免登录:不携带 Authorization 头可正常下载(端点不挂鉴权依赖)"""
upload_dir = tmp_path / "uploads"
upload_dir.mkdir(parents=True, exist_ok=True)
target = upload_dir / "doc-1_open.txt"
target.write_bytes(b"open content")
fake = FakeQdrant(meta={"raw_file_path": str(target), "original_filename": "open.txt"})
_install_fake(monkeypatch, fake)
resp = client.get("/api/v1/documents/doc-1/file")
assert resp.status_code == 200
assert resp.content == b"open content"
def test_download_file_uses_original_filename_when_missing(
client: TestClient, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""metadata 缺少 original_filename 时回退到 path.name 作为下载文件名"""
upload_dir = tmp_path / "uploads"
upload_dir.mkdir(parents=True, exist_ok=True)
target = upload_dir / "fallback_name.txt"
target.write_bytes(b"fb")
fake = FakeQdrant(meta={"raw_file_path": str(target)})
_install_fake(monkeypatch, fake)
resp = client.get("/api/v1/documents/doc-1/file")
assert resp.status_code == 200
assert "fallback_name.txt" in resp.headers.get("content-disposition", "")
+161
View File
@@ -0,0 +1,161 @@
"""文件链接集成验证
在真实内存 Qdrant + FakeOllama + 真实临时 upload_dir 环境下,验证文件上传到下载的全链路闭环:
POST /documents/upload → 入库 wait_done → GET /documents/{id}file 字段)→ GET /documents/{id}/file
并覆盖纯文本入库无文件关联的对照路径(详情 file=None、下载端点 1004)。
"""
from pathlib import Path
from urllib.parse import unquote
import pytest
from fastapi.testclient import TestClient
from qdrant_client import AsyncQdrantClient
from app.api.v1 import document as document_module
from app.config import Settings
from app.core.chunker import Chunker
from app.core.classifier import Classifier
from app.core.ingest_tasks import IngestTaskManager, IngestTaskStatus
from app.core.ingestion import Ingester
from app.core.sparse import SparseEncoder
from app.core.summarizer import Summarizer
from app.main import app
from app.models.knowledge import load_taxonomy
from app.services.qdrant import QdrantService
from tests.test_e2e_integration import (
FAKE_CATEGORY,
DeterministicEmbedding,
FakeOllama,
)
async def _make_env() -> tuple[QdrantService, Ingester]:
"""构建集成环境:真实组件 + 内存 Qdrant + FakeOllama + 确定性向量"""
qdrant = QdrantService(client=AsyncQdrantClient(location=":memory:"))
await qdrant.ensure_collections()
ollama = FakeOllama()
taxonomy = load_taxonomy()
embedding = DeterministicEmbedding()
ingester = Ingester(
summarizer=Summarizer(ollama=ollama), # type: ignore[arg-type]
classifier=Classifier(ollama=ollama, taxonomy=taxonomy), # type: ignore[arg-type]
chunker=Chunker(),
embedding=embedding,
sparse=SparseEncoder(),
qdrant=qdrant,
)
return qdrant, ingester
def _patch_app(
monkeypatch: pytest.MonkeyPatch,
manager: IngestTaskManager,
qdrant: QdrantService,
upload_dir: str,
) -> None:
"""替换模块级单例:任务管理器 / Qdrant / upload_dirlifespan 建集合改空操作"""
async def _noop_ensure_collections(self: QdrantService) -> None:
return None
monkeypatch.setattr(QdrantService, "ensure_collections", _noop_ensure_collections)
monkeypatch.setattr(document_module, "_task_manager", manager)
monkeypatch.setattr(document_module, "_qdrant", qdrant)
monkeypatch.setattr(document_module.settings, "upload_dir", upload_dir)
class TestDocumentFileIntegration:
"""文件链接集成:上传闭环 + 文本入库无文件对照"""
async def test_upload_full_loop(
self,
monkeypatch: pytest.MonkeyPatch,
tmp_path: Path,
admin_headers: dict[str, str],
) -> None:
"""文件上传闭环:upload → wait_done done → 详情含 file → 下载返回原文"""
qdrant, ingester = await _make_env()
manager = IngestTaskManager(ingester, None, Settings())
_patch_app(monkeypatch, manager, qdrant, str(tmp_path / "uploads"))
filename = "上传测试.md"
original_content = "上传原文测试内容"
with TestClient(app) as client:
# 1. multipart 上传 .md 文件(变更类端点需 admin 认证头)
resp_upload = client.post(
"/api/v1/documents/upload",
files={"file": (filename, original_content.encode("utf-8"), "text/markdown")},
headers=admin_headers,
)
assert resp_upload.status_code == 202
body_upload = resp_upload.json()
assert body_upload["code"] == 0
assert body_upload["data"]["status"] == "pending"
task_id = body_upload["data"]["task_id"]
assert task_id
# 2. 等待入库终态:done + 结果完整
final = await manager.wait_done(task_id)
assert final["status"] == IngestTaskStatus.DONE
result = final["result"]
document_id = result["document_id"]
assert document_id
assert result["category"] == FAKE_CATEGORY
# 3. 文档详情:file 字段非空,含原文件名 / url / size
resp_detail = client.get(f"/api/v1/documents/{document_id}")
body_detail = resp_detail.json()
assert body_detail["code"] == 0
file_info = body_detail["data"]["file"]
assert file_info is not None
assert filename in file_info["filename"]
assert file_info["url"] == f"/api/v1/documents/{document_id}/file"
assert file_info["size_bytes"] > 0
# 4. 下载:200 + 响应体含上传原文 + Content-Disposition 含原文件名(中文按 RFC 5987 百分号编码,需 unquote
resp_file = client.get(f"/api/v1/documents/{document_id}/file")
assert resp_file.status_code == 200
assert original_content in resp_file.content.decode("utf-8")
disposition = resp_file.headers.get("content-disposition", "")
assert "attachment" in disposition
assert filename in unquote(disposition)
async def test_text_ingest_no_file(
self,
monkeypatch: pytest.MonkeyPatch,
tmp_path: Path,
admin_headers: dict[str, str],
) -> None:
"""纯文本入库(无 metadata):详情 file=None,下载端点返回 1004"""
qdrant, ingester = await _make_env()
manager = IngestTaskManager(ingester, None, Settings())
_patch_app(monkeypatch, manager, qdrant, str(tmp_path / "uploads"))
with TestClient(app) as client:
# 1. 纯文本入库(无 metadata
resp_post = client.post(
"/api/v1/documents",
json={"text": "纯文本入库无附件的对照内容", "title": "无文件文档"},
headers=admin_headers,
)
assert resp_post.status_code == 202
body_post = resp_post.json()
assert body_post["code"] == 0
task_id = body_post["data"]["task_id"]
# 2. 等待终态 done
final = await manager.wait_done(task_id)
assert final["status"] == IngestTaskStatus.DONE
document_id = final["result"]["document_id"]
# 3. 详情:file 为 None
resp_detail = client.get(f"/api/v1/documents/{document_id}")
body_detail = resp_detail.json()
assert body_detail["code"] == 0
assert body_detail["data"]["file"] is None
# 4. 下载端点:文档未关联文件 → 1004
resp_file = client.get(f"/api/v1/documents/{document_id}/file")
body_file = resp_file.json()
assert body_file["code"] == 1004
+49
View File
@@ -19,6 +19,8 @@ class FakeManager:
self.tasks = tasks or {} self.tasks = tasks or {}
self.task_id = task_id self.task_id = task_id
self.submitted: list[DocumentInput] = [] self.submitted: list[DocumentInput] = []
self.retry_result: str | None = None
self.delete_result: bool = False
async def submit(self, doc: DocumentInput) -> str: async def submit(self, doc: DocumentInput) -> str:
self.submitted.append(doc) self.submitted.append(doc)
@@ -27,6 +29,12 @@ class FakeManager:
async def get(self, task_id: str) -> dict[str, Any] | None: async def get(self, task_id: str) -> dict[str, Any] | None:
return self.tasks.get(task_id) return self.tasks.get(task_id)
async def retry(self, task_id: str) -> str | None:
return self.retry_result
async def delete(self, task_id: str) -> bool:
return self.delete_result
def _task(task_id: str, status: str, **extra: Any) -> dict[str, Any]: def _task(task_id: str, status: str, **extra: Any) -> dict[str, Any]:
"""构造一条任务记录(时间字段为固定 ISO8601 字符串)""" """构造一条任务记录(时间字段为固定 ISO8601 字符串)"""
@@ -159,3 +167,44 @@ def test_post_empty_text_creates_no_task(client: TestClient, monkeypatch: pytest
body = resp.json() body = resp.json()
assert body["code"] == 1001 assert body["code"] == 1001
assert manager.submitted == [] assert manager.submitted == []
def test_retry_task_returns_202(client: TestClient, monkeypatch: pytest.MonkeyPatch) -> None:
"""POST tasks/{id}/retryHTTP 202,返回新 task_id 与 pending 状态"""
manager = FakeManager()
manager.retry_result = "new-task-7"
_inject_manager(monkeypatch, manager)
resp = client.post("/api/v1/documents/tasks/t-failed/retry")
assert resp.status_code == 202
body = resp.json()
assert body["code"] == 0
assert body["data"]["task_id"] == "new-task-7"
assert body["data"]["status"] == "pending"
def test_retry_task_not_found(client: TestClient, monkeypatch: pytest.MonkeyPatch) -> None:
"""POST tasks/{id}/retrymanager 返回 None → code=1004"""
manager = FakeManager()
manager.retry_result = None
_inject_manager(monkeypatch, manager)
resp = client.post("/api/v1/documents/tasks/nope/retry")
body = resp.json()
assert body["code"] == 1004
def test_delete_task(client: TestClient, monkeypatch: pytest.MonkeyPatch) -> None:
"""DELETE tasks/{id}:删除成功 → deleted=true"""
manager = FakeManager()
manager.delete_result = True
_inject_manager(monkeypatch, manager)
resp = client.delete("/api/v1/documents/tasks/t-x")
body = resp.json()
assert body["code"] == 0
assert body["data"]["task_id"] == "t-x"
assert body["data"]["deleted"] is True
+143 -1
View File
@@ -7,8 +7,8 @@ from datetime import datetime
from typing import Any from typing import Any
from app.config import Settings from app.config import Settings
from app.core.dedup import DEDUP_KEY_PREFIX
from app.core.ingest_tasks import ( from app.core.ingest_tasks import (
DEDUP_KEY_PREFIX,
REDIS_KEY_PREFIX, REDIS_KEY_PREFIX,
IngestTaskManager, IngestTaskManager,
IngestTaskStatus, IngestTaskStatus,
@@ -91,6 +91,19 @@ class FakeRedis:
async def get_json(self, key: str) -> dict[str, Any] | None: async def get_json(self, key: str) -> dict[str, Any] | None:
return self.store.get(key) return self.store.get(key)
def _get_client(self) -> Any:
"""返回支持 scan_iter 的假客户端(供 list_tasks 扫描键)"""
store = self.store
class _FakeClient:
async def scan_iter(self, match: str = "*") -> Any:
prefix = match[:-1] if match.endswith("*") else match
for key in list(store.keys()):
if key.startswith(prefix):
yield key
return _FakeClient()
async def test_submit_returns_immediately_and_completes() -> None: async def test_submit_returns_immediately_and_completes() -> None:
"""submit 立即返回;任务后台跑完为 done,结果完整,阶段序列齐全,Redis 镜像同步""" """submit 立即返回;任务后台跑完为 done,结果完整,阶段序列齐全,Redis 镜像同步"""
@@ -317,3 +330,132 @@ async def test_dedup_lookup_failure_falls_back_to_normal_pipeline() -> None:
final = await manager.wait_done(task, timeout=5) final = await manager.wait_done(task, timeout=5)
assert final["status"] == IngestTaskStatus.DONE assert final["status"] == IngestTaskStatus.DONE
assert len(ingester.calls) == 1 assert len(ingester.calls) == 1
async def test_submit_records_filename_from_metadata_or_title() -> None:
"""submitfilename 优先 metadata.original_filename,其次 title"""
ingester = FakeIngester()
manager = IngestTaskManager(ingester, None, Settings())
# 有 original_filename
t1 = await manager.submit(
DocumentInput(text="x", title="t1", metadata={"original_filename": "report.pdf"})
)
await manager.wait_done(t1, timeout=5)
# 无 original_filename,回退 title
t2 = await manager.submit(DocumentInput(text="y", title="my-title"))
await manager.wait_done(t2, timeout=5)
items = await manager.list_tasks(limit=20)
by_id = {it["task_id"]: it for it in items}
assert by_id[t1]["filename"] == "report.pdf"
assert by_id[t2]["filename"] == "my-title"
async def test_list_tasks_merges_memory_and_redis_and_sorts() -> None:
"""list_tasks:合并内存与 Redis 镜像,去重,按 updated_at 降序"""
ingester = FakeIngester()
redis = FakeRedis()
manager = IngestTaskManager(ingester, redis, Settings())
# 内存任务 1
t1 = await manager.submit(DocumentInput(text="内容A", title="t1"))
await manager.wait_done(t1, timeout=5)
# 内存任务 2
t2 = await manager.submit(DocumentInput(text="内容B", title="t2"))
await manager.wait_done(t2, timeout=5)
# Redis 镜像中独有任务(不在内存里,模拟重启后只存在 Redis 的历史记录)
redis.store[f"{REDIS_KEY_PREFIX}redis-only-1"] = {
"task_id": "redis-only-1",
"status": "done",
"filename": "legacy.md",
"created_at": "2020-01-01T00:00:00+00:00",
"updated_at": "2020-01-01T00:00:00+00:00",
"result": {"document_id": "doc-legacy"},
"error": None,
}
items = await manager.list_tasks(limit=20)
# 共 3 条(内存 2 + Redis 独有 1,内存的 2 也已镜像到 Redis 但按 task_id 去重)
assert len(items) == 3
# 按 updated_at 降序:内存任务(当前时间)排在 Redis 旧任务前
ids = [it["task_id"] for it in items]
assert "redis-only-1" in ids
assert ids[-1] == "redis-only-1" # 最旧排最后
# Redis 独有任务提取 doc_id 与 filename
legacy = next(it for it in items if it["task_id"] == "redis-only-1")
assert legacy["filename"] == "legacy.md"
assert legacy["doc_id"] == "doc-legacy"
# 内存 done 任务也提取 doc_id
mem_item = next(it for it in items if it["task_id"] == t1)
assert mem_item["doc_id"] == "doc-1"
async def test_list_tasks_dedups_memory_and_redis_by_task_id() -> None:
"""list_tasks:内存与 Redis 都有的同一 task_id 仅保留内存版本(去重)"""
ingester = FakeIngester()
redis = FakeRedis()
manager = IngestTaskManager(ingester, redis, Settings())
task_id = await manager.submit(DocumentInput(text="唯一", title="t"))
await manager.wait_done(task_id, timeout=5)
# Redis 镜像中给同一 task_id 篡改一个旧 status,验证内存版本胜出
redis.store[f"{REDIS_KEY_PREFIX}{task_id}"]["status"] = "pending"
items = await manager.list_tasks(limit=20)
assert len(items) == 1
assert items[0]["task_id"] == task_id
assert items[0]["status"] == IngestTaskStatus.DONE # 内存版本(done
async def test_list_tasks_respects_limit() -> None:
"""list_taskslimit 截断返回条数"""
ingester = FakeIngester()
manager = IngestTaskManager(ingester, None, Settings())
for i in range(5):
t = await manager.submit(DocumentInput(text=f"c{i}", title=f"t{i}"))
await manager.wait_done(t, timeout=5)
items = await manager.list_tasks(limit=3)
assert len(items) == 3
async def test_list_tasks_without_redis_returns_memory_only() -> None:
"""Redis 不可用:list_tasks 仅返回内存任务"""
ingester = FakeIngester()
manager = IngestTaskManager(ingester, None, Settings())
t1 = await manager.submit(DocumentInput(text="x", title="t1"))
await manager.wait_done(t1, timeout=5)
items = await manager.list_tasks(limit=20)
assert len(items) == 1
assert items[0]["task_id"] == t1
assert items[0]["filename"] == "t1"
assert items[0]["doc_id"] == "doc-1"
async def test_list_tasks_extract_fields_for_in_progress_task() -> None:
"""进行中任务:doc_id 为 Nonedone 时才从 result.document_id 提取)"""
ingester = FakeIngester()
ingester.gate = asyncio.Event() # 阻塞任务使其停留在最后一个阶段
manager = IngestTaskManager(ingester, None, Settings(ingest_max_concurrency=1))
t1 = await manager.submit(
DocumentInput(text="x", title="t1", metadata={"original_filename": "a.txt"})
)
await asyncio.wait_for(ingester.started.wait(), timeout=1)
await asyncio.sleep(0.05)
items = await manager.list_tasks(limit=20)
assert len(items) == 1
item = items[0]
assert item["task_id"] == t1
# 任务已推进到 writing 阶段(gate 阻塞前最后一个 progress_cb
assert item["status"] == IngestTaskStatus.WRITING
assert item["doc_id"] is None # 未完成,doc_id 为 None
assert item["filename"] == "a.txt"
ingester.gate.set()
await manager.wait_done(t1, timeout=5)
+80 -4
View File
@@ -3,13 +3,27 @@
from typing import Any from typing import Any
import pytest import pytest
from qdrant_client import AsyncQdrantClient
from app.config import settings
from app.core.chunker import Chunker from app.core.chunker import Chunker
from app.core.ingestion import Ingester, IngestionError from app.core.ingestion import Ingester, IngestionError
from app.core.sparse import SparseEncoder from app.core.sparse import SparseEncoder
from app.models.document import DocumentInput, DocumentSummary, SummaryLevel from app.models.document import DocumentInput, DocumentSummary, SummaryLevel
from app.models.knowledge import CategoryResult from app.models.knowledge import CategoryResult
from app.services.qdrant import COLLECTION_L2, COLLECTION_L3 from app.services.qdrant import COLLECTION_L2, COLLECTION_L3, QdrantService
_DIM = settings.embedding_dimension
def _fake_vec(index: int, dim: int) -> list[float]:
"""构造确定性伪向量:第 0 维放 index 标识,其余补 0,长度对齐 dim"""
vec = [0.0] * dim
if dim > 0:
vec[0] = float(index)
if dim > 1:
vec[1] = 1.0
return vec
class FakeSummarizer: class FakeSummarizer:
@@ -37,14 +51,18 @@ class FakeClassifier:
class FakeEmbedding: class FakeEmbedding:
"""按输入数量返回伪向量的假 EmbeddingService,记录每次调用的文本""" """按输入数量返回伪向量的假 EmbeddingService,记录每次调用的文本
def __init__(self) -> None: dim 默认 2FakeQdrant 不校验维度);接真实 Qdrant 时需传 settings.embedding_dimension。
"""
def __init__(self, dim: int = 2) -> None:
self.dim = dim
self.calls: list[list[str]] = [] self.calls: list[list[str]] = []
async def embed(self, texts: list[str]) -> list[list[float]]: async def embed(self, texts: list[str]) -> list[list[float]]:
self.calls.append(list(texts)) self.calls.append(list(texts))
return [[float(i), 1.0] for i in range(len(texts))] return [_fake_vec(i, self.dim) for i in range(len(texts))]
class FakeQdrant: class FakeQdrant:
@@ -65,6 +83,7 @@ class FakeQdrant:
tags: list[str], tags: list[str],
dense_vector: list[float], dense_vector: list[float],
sparse_vector: Any = None, sparse_vector: Any = None,
metadata: dict[str, str] | None = None,
) -> None: ) -> None:
if self.fail_on == "l1": if self.fail_on == "l1":
raise RuntimeError("qdrant down") raise RuntimeError("qdrant down")
@@ -77,6 +96,7 @@ class FakeQdrant:
"tags": tags, "tags": tags,
"dense_vector": dense_vector, "dense_vector": dense_vector,
"sparse_vector": sparse_vector, "sparse_vector": sparse_vector,
"metadata": metadata,
} }
) )
@@ -271,3 +291,59 @@ class TestQdrantFailure:
assert len(qdrant.l1_calls) == 1 assert len(qdrant.l1_calls) == 1
assert {c for c, _ in qdrant.nodes_calls} == {COLLECTION_L2, COLLECTION_L3} assert {c for c, _ in qdrant.nodes_calls} == {COLLECTION_L2, COLLECTION_L3}
assert qdrant.chunks_calls == [] assert qdrant.chunks_calls == []
class TestMetadataIntegration:
"""L1 metadata 端到端:真实内存 Qdrant + 假总结/分类/向量化,验证 metadata 透传与 get_doc_detail.file"""
@pytest.fixture
async def real_service(self) -> QdrantService:
svc = QdrantService(client=AsyncQdrantClient(location=":memory:"))
await svc.ensure_collections()
return svc
def _make_real_ingester(self, service: QdrantService, summary: DocumentSummary) -> Ingester:
return Ingester(
summarizer=FakeSummarizer(summary), # type: ignore[arg-type]
classifier=FakeClassifier(_category()), # type: ignore[arg-type]
chunker=Chunker(),
embedding=FakeEmbedding(dim=_DIM), # type: ignore[arg-type]
sparse=SparseEncoder(),
qdrant=service,
)
async def test_ingest_with_metadata_populates_file_field(self, real_service: QdrantService) -> None:
"""DocumentInput 带 metadata 入库后,get_doc_detail.file 含正确信息"""
doc = DocumentInput(
text="这是一段用于测试 metadata 透传的正文内容。" * 5,
title="带文件元数据的文档",
metadata={
"raw_file_path": "/data/uploads/spec.md",
"original_filename": "spec.md",
"original_size_bytes": "5120",
},
)
ingester = self._make_real_ingester(real_service, _structured_summary())
result = await ingester.ingest(doc)
# FakeQdrant 已被真实 service 取代:l1_calls 不再可用,直接查 Qdrant
detail = await real_service.get_doc_detail(result.document_id)
assert detail is not None
assert detail["file"] == {
"filename": "spec.md",
"size_bytes": 5120,
"url": f"/api/v1/documents/{result.document_id}/file",
}
# L1 payload 也应带 metadata 字段
assert detail["l1"]["metadata"] == doc.metadata
async def test_ingest_without_metadata_file_is_none(self, real_service: QdrantService) -> None:
"""文本入库(metadata 为空 dict)→ get_doc_detail.file=None"""
doc = DocumentInput(text="纯文本入库,无文件元数据。" * 5, title="纯文本")
ingester = self._make_real_ingester(real_service, _structured_summary())
result = await ingester.ingest(doc)
detail = await real_service.get_doc_detail(result.document_id)
assert detail is not None
assert detail["file"] is None
assert detail["l1"]["metadata"] == {}
+149
View File
@@ -234,3 +234,152 @@ async def test_search_hybrid_rrf(service: QdrantService) -> None:
assert len(filtered) == 1 assert len(filtered) == 1
assert filtered[0].payload is not None assert filtered[0].payload is not None
assert filtered[0].payload["doc_id"] == "doc-h2" assert filtered[0].payload["doc_id"] == "doc-h2"
# ---------- L1 metadata 存储与读取 ----------
async def test_upsert_l1_metadata_roundtrip(service: QdrantService) -> None:
"""upsert_l1 写入 metadata 后,get_l1_metadata 返回写入的 dict"""
meta = {
"raw_file_path": "/data/uploads/report.pdf",
"original_filename": "年报.pdf",
"original_size_bytes": "2048",
}
await service.upsert_l1(
doc_id="doc-meta",
title="带元数据文档",
summary="总结",
category="tech",
tags=["x"],
dense_vector=_dense(0.9),
metadata=meta,
)
assert await service.get_l1_metadata("doc-meta") == meta
async def test_upsert_l1_without_metadata_stores_empty_dict(service: QdrantService) -> None:
"""upsert_l1 不传 metadata(旧签名)→ payload 存空 dictget_l1_metadata 返回 {}"""
await service.upsert_l1(
doc_id="doc-no-meta",
title="无元数据文档",
summary="总结",
category="tech",
tags=["x"],
dense_vector=_dense(0.9),
)
assert await service.get_l1_metadata("doc-no-meta") == {}
async def test_get_l1_metadata_doc_not_found(service: QdrantService) -> None:
"""文档不存在 → get_l1_metadata 返回 None"""
assert await service.get_l1_metadata("doc-missing") is None
async def test_get_l1_metadata_legacy_payload_without_field(service: QdrantService) -> None:
"""存量旧文档:payload 完全没有 metadata 字段(绕过 upsert_l1 直接写点)→ 返回 None 不报错"""
import uuid
from qdrant_client import models
point = models.PointStruct(
id=str(uuid.uuid5(uuid.NAMESPACE_URL, "legacy-doc:l1")),
vector={"dense": _dense(0.4)},
payload={"doc_id": "legacy-doc", "title": "旧文档", "category": "tech", "tags": [], "text": "旧总结"},
)
await service.client.upsert(collection_name=COLLECTION_L1, points=[point])
assert await service.get_l1_metadata("legacy-doc") is None
# ---------- get_doc_detail file 字段 ----------
async def test_get_doc_detail_file_field_with_raw_path(service: QdrantService) -> None:
"""L1 metadata 含 raw_file_path → file 字段含 filename/size_bytes/url"""
await service.upsert_l1(
doc_id="doc-file",
title="文件文档",
summary="总结",
category="tech",
tags=["x"],
dense_vector=_dense(0.9),
metadata={
"raw_file_path": "/data/uploads/report.pdf",
"original_filename": "年报.pdf",
"original_size_bytes": "4096",
},
)
detail = await service.get_doc_detail("doc-file")
assert detail is not None
file_info = detail["file"]
assert file_info == {
"filename": "年报.pdf",
"size_bytes": 4096,
"url": "/api/v1/documents/doc-file/file",
}
async def test_get_doc_detail_file_field_filename_fallback_to_path_name(service: QdrantService) -> None:
"""metadata 缺 original_filename 时,filename 回退为 raw_file_path 的 Path.name"""
await service.upsert_l1(
doc_id="doc-file-fb",
title="文件文档",
summary="总结",
category="tech",
tags=["x"],
dense_vector=_dense(0.9),
metadata={"raw_file_path": "/data/uploads/notes.md", "original_size_bytes": "100"},
)
detail = await service.get_doc_detail("doc-file-fb")
assert detail is not None
assert detail["file"]["filename"] == "notes.md"
assert detail["file"]["size_bytes"] == 100
async def test_get_doc_detail_file_none_when_no_metadata(service: QdrantService) -> None:
"""无 metadata(文本入库,存空 dict)→ file=None"""
await service.upsert_l1(
doc_id="doc-text",
title="纯文本文档",
summary="总结",
category="tech",
tags=["x"],
dense_vector=_dense(0.9),
)
detail = await service.get_doc_detail("doc-text")
assert detail is not None
assert detail["file"] is None
async def test_get_doc_detail_file_none_when_no_raw_path(service: QdrantService) -> None:
"""metadata 无 raw_file_path → file=None"""
await service.upsert_l1(
doc_id="doc-meta-no-path",
title="文档",
summary="总结",
category="tech",
tags=["x"],
dense_vector=_dense(0.9),
metadata={"original_filename": "x.pdf", "original_size_bytes": "10"},
)
detail = await service.get_doc_detail("doc-meta-no-path")
assert detail is not None
assert detail["file"] is None
async def test_get_doc_detail_file_none_for_legacy_doc(service: QdrantService) -> None:
"""存量旧文档(payload 无 metadata 字段,绕过 upsert_l1 直接写点)→ file=None 不报错"""
import uuid
from qdrant_client import models
point = models.PointStruct(
id=str(uuid.uuid5(uuid.NAMESPACE_URL, "legacy-detail:l1")),
vector={"dense": _dense(0.3)},
payload={"doc_id": "legacy-detail", "title": "旧文档", "category": "tech", "tags": [], "text": "旧总结"},
)
await service.client.upsert(collection_name=COLLECTION_L1, points=[point])
detail = await service.get_doc_detail("legacy-detail")
assert detail is not None
assert detail["file"] is None
assert detail["l1"]["doc_id"] == "legacy-detail"
+128
View File
@@ -0,0 +1,128 @@
"""RerankerService 单元测试(mock httpx,不发起真实网络请求)"""
from typing import Any
import httpx
import pytest
from app.config import settings
from app.services.reranker import OllamaRerankerService, create_reranker_service
def _make_response(status_code: int, data: dict | None = None) -> httpx.Response:
"""构造带 request 上下文的 httpx.Responseraise_for_status 依赖 request"""
request = httpx.Request("POST", "http://localhost:11434/api/rerank")
if data is None:
return httpx.Response(status_code, request=request)
return httpx.Response(status_code, json=data, request=request)
class _FakeAsyncClient:
"""httpx.AsyncClient 替代品:记录请求参数,返回预设响应或抛出预设异常"""
def __init__(
self,
captured: dict,
response: httpx.Response | None = None,
error: Exception | None = None,
**kwargs: Any,
) -> None:
self._captured = captured
self._response = response
self._error = error
captured["timeout"] = kwargs.get("timeout")
async def __aenter__(self) -> "_FakeAsyncClient":
return self
async def __aexit__(self, *args: object) -> bool:
return False
async def post(self, url: str, json: dict | None = None) -> httpx.Response:
self._captured["url"] = url
self._captured["json"] = json
if self._error is not None:
raise self._error
assert self._response is not None
return self._response
def _client(monkeypatch: pytest.MonkeyPatch, captured: dict, **kw: Any) -> None:
monkeypatch.setattr(httpx, "AsyncClient", lambda **k: _FakeAsyncClient(captured, **k, **kw))
class TestRerank:
async def test_returns_scores_aligned_to_documents(self, monkeypatch: pytest.MonkeyPatch) -> None:
captured: dict = {}
_client(
monkeypatch,
captured,
response=_make_response(
200,
{
"results": [
{"index": 0, "relevance_score": 0.2},
{"index": 1, "relevance_score": 0.9},
{"index": 2, "relevance_score": 0.5},
]
},
),
)
service = OllamaRerankerService(base_url="http://localhost:11434/", model="qwen3-reranker:0.6b")
scores = await service.rerank("查询", ["doc-a", "doc-b", "doc-c"])
assert scores == [0.2, 0.9, 0.5]
assert captured["url"] == "http://localhost:11434/api/rerank"
assert captured["json"] == {
"model": "qwen3-reranker:0.6b",
"query": "查询",
"documents": ["doc-a", "doc-b", "doc-c"],
}
async def test_empty_documents_returns_empty(self, monkeypatch: pytest.MonkeyPatch) -> None:
captured: dict = {}
_client(monkeypatch, captured, response=_make_response(200, {"results": []}))
service = OllamaRerankerService(base_url="http://localhost:11434", model="qwen3-reranker:0.6b")
scores = await service.rerank("查询", [])
assert scores == []
# 空文档不发起请求
assert "url" not in captured
async def test_partial_results_zero_fill_missing(self, monkeypatch: pytest.MonkeyPatch) -> None:
captured: dict = {}
# 仅返回 index 1index 0/2 缺失 → 补 0
_client(
monkeypatch,
captured,
response=_make_response(200, {"results": [{"index": 1, "relevance_score": 0.7}]}),
)
service = OllamaRerankerService(base_url="http://localhost:11434", model="qwen3-reranker:0.6b")
scores = await service.rerank("查询", ["a", "b", "c"])
assert scores == [0.0, 0.7, 0.0]
async def test_http_error_raises(self, monkeypatch: pytest.MonkeyPatch) -> None:
_client(monkeypatch, {}, response=_make_response(500))
service = OllamaRerankerService(base_url="http://localhost:11434", model="qwen3-reranker:0.6b")
with pytest.raises(httpx.HTTPStatusError):
await service.rerank("查询", ["a", "b"])
class TestCreateRerankerService:
def test_disabled_returns_none(self, monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setattr(settings, "reranker_enabled", False)
assert create_reranker_service() is None
def test_enabled_returns_service(self, monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setattr(settings, "reranker_enabled", True)
monkeypatch.setattr(settings, "reranker_model", "qwen3-reranker:0.6b")
monkeypatch.setattr(settings, "ollama_base_url", "http://ollama:11434")
monkeypatch.setattr(settings, "reranker_timeout", 30.0)
service = create_reranker_service()
assert isinstance(service, OllamaRerankerService)
assert service.model == "qwen3-reranker:0.6b"
+64 -1
View File
@@ -82,15 +82,29 @@ def _route(
) )
def _make_retriever(qdrant: FakeQdrant, route: RouteDecision) -> Retriever: def _make_retriever(qdrant: FakeQdrant, route: RouteDecision, reranker: Any = None) -> Retriever:
return Retriever( return Retriever(
qdrant=qdrant, # type: ignore[arg-type] qdrant=qdrant, # type: ignore[arg-type]
query_parser=FakeParser(route), # type: ignore[arg-type] query_parser=FakeParser(route), # type: ignore[arg-type]
embedding=FakeEmbedding(), embedding=FakeEmbedding(),
sparse_encoder=SparseEncoder(), sparse_encoder=SparseEncoder(),
reranker=reranker,
) )
class FakeReranker:
"""假 RerankerService:返回预设分数或抛出预设异常"""
def __init__(self, scores: list[float] | None = None, exc: Exception | None = None) -> None:
self.scores = scores if scores is not None else []
self.exc = exc
async def rerank(self, query: str, documents: list[str]) -> list[float]:
if self.exc is not None:
raise self.exc
return self.scores
def _calls(qdrant: FakeQdrant, collection: str) -> list[dict[str, Any]]: def _calls(qdrant: FakeQdrant, collection: str) -> list[dict[str, Any]]:
return [c for c in qdrant.calls if c["collection"] == collection] return [c for c in qdrant.calls if c["collection"] == collection]
@@ -348,3 +362,52 @@ class TestSearchApi:
resp = client.get("/api/v1/health") resp = client.get("/api/v1/health")
assert resp.status_code == 200 assert resp.status_code == 200
assert resp.json() == {"status": "ok"} assert resp.json() == {"status": "ok"}
class TestReranker:
"""重排启用时对 chunk 候选精排;关闭或失败则退化为 RRF 顺序"""
@staticmethod
def _full_qdrant() -> FakeQdrant:
return FakeQdrant(
{
COLLECTION_L1: [[_point("l1a", "d1")]],
COLLECTION_L2: [[_point("l2a", "d1", "章节A")]],
COLLECTION_L3: [[_point("l3a", "d1", "章节A")]],
COLLECTION_CHUNKS: [
[
_point("c1", "d1", "章节A"),
_point("c2", "d1", "章节A"),
_point("c3", "d1", "章节A"),
]
],
}
)
async def test_enabled_reorders_by_relevance(self):
"""重排按相关性分数降序重排,并将 score 写回相关性分数"""
reranker = FakeReranker(scores=[0.1, 0.9, 0.5]) # c1<c3<c2
retriever = _make_retriever(self._full_qdrant(), _route(), reranker=reranker)
resp = await retriever.search(SearchRequest(query="测试查询"))
# 重排后顺序为 c2, c3, c1
assert [h.text for h in resp.hits] == ["text-c2", "text-c3", "text-c1"]
assert resp.hits[0].score == 0.9
async def test_disabled_keeps_rrf_order(self):
"""重排未注入(关闭)时保持 RRF 融合原始顺序"""
retriever = _make_retriever(self._full_qdrant(), _route(), reranker=None)
resp = await retriever.search(SearchRequest(query="测试查询"))
assert [h.text for h in resp.hits] == ["text-c1", "text-c2", "text-c3"]
async def test_failure_falls_back_to_rrf(self):
"""重排调用异常时退化为 RRF 顺序,检索不中断"""
reranker = FakeReranker(exc=RuntimeError("ollama 不可用"))
retriever = _make_retriever(self._full_qdrant(), _route(), reranker=reranker)
resp = await retriever.search(SearchRequest(query="测试查询"))
assert [h.text for h in resp.hits] == ["text-c1", "text-c2", "text-c3"]
+247
View File
@@ -0,0 +1,247 @@
"""Spec Task 5:用户管理增强(PATCH 端点 + enabled 字段)全链路集成验证
内存 UserStore/SessionStore 经 conftest.auth_stores 夹具注入 app.api.deps 单例,
通过 TestClient 走真实 HTTP 链路(不跑 lifespan,无需真实 Redis/Qdrant),覆盖:
- 改角色生效:admin 创建 user → PATCH 改 admin → GET /auth/me 与重新登录均反映新角色
- 禁用用户:旧 token 立即失效(session 已清,1005);重新登录 1005("账号已禁用"
- 重新启用:PATCH enabled=True → login 恢复成功
- 最后 admin 保护:唯一 admin 降级/禁用自己 → 1001(且自身状态未变)
- PATCH 校验:空 body / 非法 role → 1001;不存在用户 → 1004;非 admin 调用 → 1006
"""
from typing import Any
from fastapi.testclient import TestClient
from app.main import app
def _login(client: TestClient, username: str, password: str) -> dict[str, Any]:
"""调登录接口并返回响应体"""
return client.post("/api/v1/auth/login", json={"username": username, "password": password}).json()
def _bearer(token: str) -> dict[str, str]:
"""构造 Authorization Bearer 请求头"""
return {"Authorization": f"Bearer {token}"}
def _admin_login(client: TestClient) -> dict[str, str]:
"""以 auth_stores 预置的 adminadmin/admin-pass-123)登录,返回 Bearer 请求头"""
resp = _login(client, "admin", "admin-pass-123")
assert resp["code"] == 0, f"admin 登录失败: {resp}"
return _bearer(resp["data"]["token"])
def _create_user(
client: TestClient,
admin_headers: dict[str, str],
username: str,
password: str,
role: str = "user",
) -> None:
"""admin 创建用户并断言成功"""
resp = client.post(
"/api/v1/auth/users",
json={"username": username, "password": password, "role": role},
headers=admin_headers,
).json()
assert resp["code"] == 0, f"创建用户 {username} 失败: {resp}"
class TestRoleChange:
"""改角色生效全链路"""
def test_role_change_takes_effect(self, auth_stores) -> None:
client = TestClient(app)
admin_headers = _admin_login(client)
# admin 创建 user 角色账号 alice
_create_user(client, admin_headers, "alice", "alice-pass-123", role="user")
# alice 登录,初始角色 user
alice_login = _login(client, "alice", "alice-pass-123")
assert alice_login["code"] == 0
assert alice_login["data"]["role"] == "user"
alice_headers = _bearer(alice_login["data"]["token"])
# 改角色前 GET /auth/me 反映 user 角色
me_before = client.get("/api/v1/auth/me", headers=alice_headers).json()
assert me_before["code"] == 0
assert me_before["data"]["role"] == "user"
# admin PATCH 改 alice 角色为 admin
patched = client.patch(
"/api/v1/auth/users/alice",
json={"role": "admin"},
headers=admin_headers,
).json()
assert patched["code"] == 0
assert patched["data"]["role"] == "admin"
# 同一 token 立即反映新角色(角色变更不清 session,用户记录实时读取)
me_after = client.get("/api/v1/auth/me", headers=alice_headers).json()
assert me_after["code"] == 0
assert me_after["data"]["role"] == "admin"
# 重新登录也反映新角色
relogin = _login(client, "alice", "alice-pass-123")
assert relogin["code"] == 0
assert relogin["data"]["role"] == "admin"
class TestDisableUser:
"""禁用用户:旧 session 清除 + 登录拒绝"""
def test_disable_user_clears_session_and_blocks_login(self, auth_stores) -> None:
client = TestClient(app)
admin_headers = _admin_login(client)
_create_user(client, admin_headers, "bob", "bob-pass-123")
# bob 登录拿 token
bob_login = _login(client, "bob", "bob-pass-123")
assert bob_login["code"] == 0
bob_headers = _bearer(bob_login["data"]["token"])
# admin 禁用 bob(清空其全部 session
disabled = client.patch(
"/api/v1/auth/users/bob",
json={"enabled": False},
headers=admin_headers,
).json()
assert disabled["code"] == 0
assert disabled["data"]["enabled"] is False
# bob 旧 token 调鉴权端点 → 1005(session 已清,凭证无效)
me = client.get("/api/v1/auth/me", headers=bob_headers).json()
assert me["code"] == 1005
# bob 重新登录 → 1005(账号已禁用)
relogin = _login(client, "bob", "bob-pass-123")
assert relogin["code"] == 1005
assert "禁用" in relogin["message"]
class TestReenableUser:
"""重新启用:login 恢复成功"""
def test_reenable_user_allows_login(self, auth_stores) -> None:
client = TestClient(app)
admin_headers = _admin_login(client)
_create_user(client, admin_headers, "carol", "carol-pass-123")
# 先禁用 carol,确认登录被拒
disabled = client.patch(
"/api/v1/auth/users/carol",
json={"enabled": False},
headers=admin_headers,
).json()
assert disabled["code"] == 0
assert _login(client, "carol", "carol-pass-123")["code"] == 1005
# 重新启用
enabled = client.patch(
"/api/v1/auth/users/carol",
json={"enabled": True},
headers=admin_headers,
).json()
assert enabled["code"] == 0
assert enabled["data"]["enabled"] is True
# login 恢复成功
relogin = _login(client, "carol", "carol-pass-123")
assert relogin["code"] == 0
assert relogin["data"]["username"] == "carol"
class TestLastAdminProtection:
"""最后 admin 保护:唯一 admin 不可降级/禁用"""
def test_cannot_demote_last_admin(self, auth_stores) -> None:
client = TestClient(app)
admin_headers = _admin_login(client)
# 唯一 adminadmin)尝试降级自己 role=user → 1001
resp = client.patch(
"/api/v1/auth/users/admin",
json={"role": "user"},
headers=admin_headers,
).json()
assert resp["code"] == 1001
# admin 未被降级,仍可访问 admin 专属端点
assert client.get("/api/v1/auth/users", headers=admin_headers).json()["code"] == 0
def test_cannot_disable_last_admin(self, auth_stores) -> None:
client = TestClient(app)
admin_headers = _admin_login(client)
# 唯一 admin 尝试禁用自己 enabled=False → 1001
resp = client.patch(
"/api/v1/auth/users/admin",
json={"enabled": False},
headers=admin_headers,
).json()
assert resp["code"] == 1001
# admin 未被禁用,旧 token 仍可用
assert client.get("/api/v1/auth/users", headers=admin_headers).json()["code"] == 0
class TestPatchValidation:
"""PATCH 端点校验:空 body / 非法 role / 不存在用户 / 非 admin"""
def test_empty_body_rejected(self, auth_stores) -> None:
client = TestClient(app)
admin_headers = _admin_login(client)
_create_user(client, admin_headers, "dave", "dave-pass-123")
# 空 bodyrole 与 enabled 均缺)→ 模型校验 1001
resp = client.patch(
"/api/v1/auth/users/dave",
json={},
headers=admin_headers,
).json()
assert resp["code"] == 1001
def test_invalid_role_rejected(self, auth_stores) -> None:
client = TestClient(app)
admin_headers = _admin_login(client)
_create_user(client, admin_headers, "dave", "dave-pass-123")
# role 非法(非 admin/user)→ 模型校验 1001
resp = client.patch(
"/api/v1/auth/users/dave",
json={"role": "superuser"},
headers=admin_headers,
).json()
assert resp["code"] == 1001
def test_nonexistent_user_rejected(self, auth_stores) -> None:
client = TestClient(app)
admin_headers = _admin_login(client)
# 不存在用户 → 1004
resp = client.patch(
"/api/v1/auth/users/ghost",
json={"role": "admin"},
headers=admin_headers,
).json()
assert resp["code"] == 1004
def test_non_admin_forbidden(self, auth_stores) -> None:
client = TestClient(app)
admin_headers = _admin_login(client)
_create_user(client, admin_headers, "eve", "eve-pass-123", role="user")
# eveuser 角色)登录后调 PATCH → 1006
eve_login = _login(client, "eve", "eve-pass-123")
assert eve_login["code"] == 0
eve_headers = _bearer(eve_login["data"]["token"])
resp = client.patch(
"/api/v1/auth/users/eve",
json={"role": "admin"},
headers=eve_headers,
).json()
assert resp["code"] == 1006
+96
View File
@@ -17,6 +17,7 @@ import pytest
from app.core.users import ( from app.core.users import (
UserExistsError, UserExistsError,
UserNotFoundError,
UserStore, UserStore,
UserStoreError, UserStoreError,
bootstrap_admin, bootstrap_admin,
@@ -225,6 +226,101 @@ class TestCountAdmins:
assert await store.count_admins() == 1 assert await store.count_admins() == 1
class TestEnabledField:
"""enabled 字段:默认 True、create 传参、get/list 回读、存量兼容"""
async def test_default_enabled_is_true(self):
record = await UserStore(FakeRedis()).create("alice", "password123")
assert record.enabled is True
async def test_create_with_enabled_false(self):
redis = FakeRedis()
store = UserStore(redis)
record = await store.create("alice", "password123", enabled=False)
assert record.enabled is False
# 持久化后回读仍为 False
assert (await store.get("alice")).enabled is False
# 落库 JSON 含 enabled 字段
assert json.loads(redis.store["user:alice"])["enabled"] is False
async def test_get_list_roundtrip_enabled(self):
store = UserStore(FakeRedis())
await store.create("alice", "password123", enabled=False)
await store.create("bob", "password123", role="admin")
assert (await store.get("alice")).enabled is False
assert (await store.get("bob")).enabled is True
records = {r.username: r for r in await store.list()}
assert records["alice"].enabled is False
assert records["bob"].enabled is True
async def test_legacy_record_without_enabled_defaults_true(self):
"""存量记录无 enabled 字段时按 True 兼容(get/list"""
redis = FakeRedis()
store = UserStore(redis)
# 直接写入无 enabled 字段的存量记录
redis.store["user:legacy"] = json.dumps(
{
"username": "legacy",
"role": "user",
"password_hash": "hash",
"salt": "00" * 16,
"must_change_password": False,
"created_at": "2024-01-01T00:00:00+00:00",
}
)
record = await store.get("legacy")
assert record is not None
assert record.enabled is True
records = await store.list()
assert records[0].enabled is True
class TestUpdateUser:
"""update_user:角色/启用状态更新与校验"""
async def test_update_role(self):
store = UserStore(FakeRedis())
await store.create("alice", "password123")
updated = await store.update_user("alice", role="admin")
assert updated.role == "admin"
assert updated.enabled is True # 未改动
# 持久化
record = await store.get("alice")
assert record is not None
assert record.role == "admin"
async def test_update_enabled(self):
store = UserStore(FakeRedis())
await store.create("alice", "password123", role="admin")
updated = await store.update_user("alice", enabled=False)
assert updated.enabled is False
assert updated.role == "admin" # 未改动
async def test_update_both(self):
store = UserStore(FakeRedis())
await store.create("alice", "password123")
updated = await store.update_user("alice", role="admin", enabled=False)
assert updated.role == "admin"
assert updated.enabled is False
async def test_user_not_found_raises(self):
with pytest.raises(UserNotFoundError):
await UserStore(FakeRedis()).update_user("nobody", role="admin")
async def test_invalid_role_raises_value_error(self):
store = UserStore(FakeRedis())
await store.create("alice", "password123")
with pytest.raises(ValueError):
await store.update_user("alice", role="superuser")
async def test_no_fields_noop(self):
store = UserStore(FakeRedis())
await store.create("alice", "password123", role="admin")
updated = await store.update_user("alice")
assert updated.role == "admin"
assert updated.enabled is True
class TestBootstrapAdmin: class TestBootstrapAdmin:
"""空库引导创建默认管理员""" """空库引导创建默认管理员"""