Commit Graph

22 Commits

Author SHA1 Message Date
hailin 6476bd868f feat(llm-gateway): 新增对外 LLM API 代理服务 — 完整的监管注入、内容审查和管理后台
## 新增微服务: llm-gateway (端口 3008)

对外提供与 Anthropic/OpenAI 完全兼容的 API 接口,中间拦截实现:
- API Key 认证:由我们分配 Key 给外部用户,SHA-256 哈希存储
- System Prompt 注入:在请求转发前注入监管合规内容(支持 prepend/append)
- 内容审查过滤:对用户消息进行关键词/正则匹配,支持 block/warn/log 三种动作
- 用量记录:异步批量写入,跟踪 token 消耗和费用估算
- 审计日志:记录每次请求的来源 IP、过滤状态、注入状态等
- 速率限制:基于内存滑动窗口的 RPM 限制

### 技术选型
- Fastify (非 NestJS):纯代理场景无需 DI 容器,路由开销 ~2ms
- SSE 流式管道:零缓冲直通,支持 Anthropic streaming 和 OpenAI streaming
- 规则缓存:30 秒 TTL,避免每次请求查库

### API 端点
- POST /v1/messages — Anthropic Messages API 代理(流式+非流式)
- POST /v1/embeddings — OpenAI Embeddings API 代理
- POST /v1/chat/completions — OpenAI Chat Completions API 代理
- GET /health — 健康检查

## 数据库 (5 张新表)

- gateway_api_keys: 外部用户 API Key(权限、限速、预算、过期时间)
- gateway_injection_rules: 监管内容注入规则(位置、匹配模型、匹配 Key)
- gateway_content_rules: 内容审查规则(关键词/正则、block/warn/log)
- gateway_usage_logs: Token 用量记录(按 Key、模型、提供商统计)
- gateway_audit_logs: 请求审计日志(IP、过滤状态、注入状态)

## Admin 后端 (conversation-service)

4 个 NestJS 控制器,挂载在 /conversations/admin/gateway/ 下:
- AdminGatewayKeysController: Key 的 CRUD + toggle
- AdminGatewayInjectionRulesController: 注入规则 CRUD + toggle
- AdminGatewayContentRulesController: 内容审查规则 CRUD + toggle
- AdminGatewayDashboardController: 仪表盘汇总、用量查询、审计日志查询

5 个 ORM 实体文件对应 5 张数据库表。

## Admin 前端 (admin-client)

新增 features/llm-gateway 模块,Tabs 布局包含 5 个管理面板:
- API Key Tab: 创建/删除/启停 Key,创建时一次性显示完整 Key
- 注入规则 Tab: 配置监管内容(前置/追加到 system prompt)
- 内容审查 Tab: 配置关键词/正则过滤规则
- 用量统计 Tab: 查看 token 消耗、费用、响应时间
- 审计日志 Tab: 查看请求记录、过滤命中、注入状态

菜单项: GatewayOutlined + "LLM 网关",位于"系统总监"和"数据分析"之间。

## 基础设施

- docker-compose.yml: 新增 llm-gateway 服务定义
- kong.yml: 新增 /v1/messages、/v1/embeddings、/v1/chat/completions 路由
  - 超时设置 300 秒(LLM 长响应)
  - CORS 新增 X-Api-Key、anthropic-version、anthropic-beta 头
- init-db.sql: 新增 5 张 gateway 表的建表语句

## 架构说明

内部服务(conversation-service、knowledge-service、evolution-service)继续直连 API,
llm-gateway 仅服务外部用户。两者通过共享 PostgreSQL 数据库关联配置。

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 22:32:25 -08:00
hailin 8e4bd95dda feat(agents): add Collection Expert specialist + admin directive system
## Part A: Collection Expert Specialist Agent (7th specialist)
- New specialist: CollectionExpertService (Haiku 4.5, maxTurns 2)
  - Analyzes user info completeness against 12-item weighted checklist
  - Identifies missing fields, recommends next questions
  - Category-specific priority adjustments (QMAS/GEP/IANG/TTPS/CIES/TechTAS)
  - Tools: search_knowledge, get_user_context
  - Admin directive injection: loads active directives from DB before each run
- Prompt: collection-expert-prompt.ts (completeness calc, validation rules, JSON output)
- Coordinator integration: invoke_collection_expert tool + case in executeAgentTool
- System prompt: section 2.6 usage guide, section 4.3 optional invocation reference

## Part B: Admin Directive System (parallel to assessment directives)
- ORM: CollectionDirectiveORM (collection_directives table)
  - Types: general, priority, category, validation
  - Multi-tenant with tenant_id + enabled indexes
- SQL: CREATE TABLE collection_directives in init-db.sql
- Controller: /conversations/admin/collection-directives (10 REST endpoints)
  - CRUD + toggle + reset + preview + AI chat
- Chat Service: CollectionDirectiveChatService (Haiku 4.5 tool loop)
  - 5 tools: list/create/update/delete/reset directives
  - mutated flag for frontend cache invalidation

## Part C: Frontend Admin-Client
- Feature module: features/collection-config/ (5 files)
  - API client, React Query hooks, Config page, Chat drawer
  - Directive types: 通用指令/优先级调整/类别配置/验证规则
- Route: /collection-config in App.tsx
- Sidebar: FormOutlined icon, label '收集指令' in MainLayout.tsx

Files: 11 new, 9 modified | Backend + frontend compile clean

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 20:07:33 -08:00
hailin 9feb03153b feat(agents): add admin assessment directive system for dynamic prompt injection
Admins can now write natural-language directives that get injected into the
assessment expert's system prompt. Directives are stored in DB, loaded per
execution, and support incremental additions, toggling, and full reset.

Backend:
- New assessment_directives table + ORM entity
- Admin CRUD API at /conversations/admin/assessment-directives
- buildAssessmentExpertPrompt() accepts optional adminDirectives param
- AssessmentExpertService loads active directives from DB before each execution
- Fail-safe: missing repo/tenant context → default prompt (no directives)

Frontend (admin-client):
- New "评估指令" page with table, create/edit modals, toggle switches
- Prompt preview panel showing assembled directive text
- Reset-to-default with confirmation
- React Query hooks for all CRUD operations

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 18:21:21 -08:00
hailin 06226a3d74 fix(db): drop files_user_id_fkey — anonymous users may not exist in users table
The conversations table has no FK on user_id, but files had one, causing
500 errors on file upload when the anonymous user wasn't registered.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 08:11:37 -08:00
hailin cf2fd07ead fix(file-service): sync tenantId in domain entity and add migration
FileORM had tenant_id column but FileEntity domain class was missing it,
causing "column FileORM.tenant_id does not exist" errors on production.

- Add tenantId to FileEntity (constructor, create, fromPersistence)
- Pass tenantId in repository toEntity() mapping
- Add idempotent migration script for files.tenant_id + indexes

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 04:21:43 -08:00
hailin 4ac1fc4f88 fix(db): add multi-tenancy migration for conversation-service tables
- Add tenant_id column to conversations, messages, token_usages tables
- Create standalone migration SQL script for production deployment
- Add agent_executions table to init-db.sql for new installations
- Fix MessageORM created_at nullable mismatch with database schema
- Backfill existing data with default tenant ID

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 08:39:10 -08:00
hailin e1e9ba1a77 feat(multi-tenant): add super admin module and user profile pages
- Add SuperAdmin module for tenant management (CRUD, suspend/activate)
- Add tenant management REST API (/super-admin/tenants/*)
- Add user profile menu in ChatSidebar with dropdown
- Add ProfilePage and BindPhonePage for user account management
- Update init-db.sql with tenant_id columns for all 16 tables
- Add database seed script (scripts/seed.ts) with ts-node
- Integrate db:seed into deploy.sh rebuild command

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-26 06:41:11 -08:00
hailin 7975982fc3 feat(user): add user profile and contact management
- Add UserProfile entity with immigration-specific fields:
  - Basic info (name, birth date, nationality, current location)
  - Immigration intent (target countries, types, timeline)
  - Education records with WES evaluation flag
  - Work records with NOC codes
  - Language scores (IELTS, TOEFL, etc.)
  - Family members info
  - Financial info for investment immigration
  - Profile completion percentage calculation

- Add UserContact entity for identity binding:
  - Support multiple contact types (EMAIL, WECHAT, WHATSAPP, TELEGRAM, LINE)
  - Verification code flow with expiration
  - Notification settings (paid feature)
  - Notification types: POLICY_UPDATE, DEADLINE_REMINDER, etc.

- Add API endpoints:
  - GET/PUT /users/me/profile/* for profile sections
  - GET/POST/PUT/DELETE /users/me/contacts for contact management
  - POST /users/me/contacts/:type/verification for verification flow
  - POST/PUT/DELETE /users/me/contacts/:type/notifications

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-25 19:32:17 -08:00
hailin 65c0bdd17c feat(payment): add transaction reliability and idempotency support
- Add TransactionService for atomic database operations with optimistic lock retry
- Implement pessimistic locking in payment callback handling to prevent race conditions
- Add idempotency check via transactionId unique index to prevent duplicate processing
- Add version columns to PaymentORM and OrderORM for optimistic locking
- Add composite indexes for performance (order_status, transaction_id)
- Optimize connection pool settings for both payment and conversation services
- Update init-db.sql with version columns and new indexes

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-25 07:00:01 -08:00
hailin 0992523876 fix(init-db): add V2 consulting fields to conversations table
Add missing V2 fields to init-db.sql to match conversation.entity.ts:
- consulting_stage
- consulting_state
- collected_info
- recommended_programs
- conversion_path
- device_info

Also add indexes for the new fields.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-24 19:43:27 -08:00
hailin 849a4a3099 feat(conversation): add token usage tracking for API cost analysis
- Add TokenUsageEntity to store per-request token consumption
- Add TokenUsageService with cost calculation and statistics APIs
  - Record input/output/cache tokens per API call
  - Calculate estimated cost based on Claude pricing
  - Provide user/conversation/global stats aggregation
  - Support daily stats and top users ranking
- Integrate token tracking in ClaudeAgentService
  - Track latency, tool calls, response length
  - Accumulate tokens across tool loop iterations
- Add token_usages table to init-db.sql with proper indexes

This enables:
- Per-user token consumption tracking
- Cost analysis and optimization
- Future billing/quota features

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-23 08:23:58 -08:00
hailin 925283d8e6 fix(schema): support anonymous users in conversations and user_memories
- Remove foreign key constraint from conversations.user_id
- Remove foreign key constraint from user_memories.user_id
- Add FACT and INTENT to user_memories.memory_type enum
- Both tables now support anonymous users (UUID without registration)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-23 05:28:59 -08:00
hailin 0f56cea96a fix(schema): sync ORM entities with database schema
- Add missing 'type' column to MessageORM entity
- Add 'TEXT_WITH_ATTACHMENTS' to messages.type CHECK constraint
  (matches MessageType enum in conversation-service)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-23 05:00:38 -08:00
hailin 9ae2feff09 fix(db): correct admin password hash in init-db.sql
The previous hash was not working correctly with bcrypt.compare().
Updated to a verified working hash for password 'admin123'.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-23 04:40:47 -08:00
hailin d4925719fc feat(multimodal): add file upload and image support for chat
- Add MinIO object storage to docker-compose infrastructure
- Create file-service microservice for upload management with presigned URLs
- Add files table to database schema
- Update nginx and Kong for MinIO proxy routes
- Implement file upload UI in chat InputArea with drag-and-drop
- Add attachment preview in MessageBubble component
- Update conversation-service to handle multimodal messages
- Add Claude Vision API integration for image analysis

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-10 05:34:41 -08:00
hailin 141b45bec2 chore(db): sync init-db.sql with complete postgres schema
Sync scripts/init-db.sql with infrastructure/docker/services/postgres/init.sql
to include all required database columns:
- users: fingerprint, nickname columns for anonymous auth support
- conversations: ended_at column for tracking conversation end time
- messages: type enum column (USER, ASSISTANT, SYSTEM)

This ensures the schema files are consistent and include all columns
needed by the application entities.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 21:15:53 -08:00
hailin f06e6ee76e feat(scripts): add unified deployment management script
Commands:
- start/stop/restart/status - service lifecycle
- logs - view service logs
- build/pull - build and deploy
- health - health check all services
- kong - Kong routes/services/plugins management
- db - PostgreSQL/Redis/Neo4j access and backup
- clean - cleanup images/volumes

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 19:45:47 -08:00
hailin fd4f23accc refactor: move Kong config to kong/ directory
Organize infrastructure configs properly:
- nginx/ for Nginx configuration
- kong/ for Kong configuration

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 19:34:04 -08:00
hailin 2512848d6c refactor(kong): consolidate Kong configuration into single init script
- Update init-kong.sh with complete service/route configuration
- Add CORS plugin configuration with all required headers
- Add timeout settings (120s for conversation-service)
- Simplify docker-compose kong-init to use the script
- Add kong-migrations service for database bootstrap

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 19:27:26 -08:00
hailin 572796f753 feat(kong): add Kong API gateway initialization script
Configures routes for all microservices:
- user-service: /api/v1/users, /api/v1/auth
- payment-service: /api/v1/payments, /api/v1/subscriptions
- knowledge-service: /api/v1/knowledge
- conversation-service: /api/v1/conversations, /api/v1/messages
- evolution-service: /api/v1/evolution

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 19:24:31 -08:00
hailin 885e84f823 fix: 修复管理员默认密码哈希
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 07:58:04 -08:00
hailin a7add8ff90 Initial commit: iConsulting 香港移民咨询智能客服系统
项目架构:
- Monorepo (pnpm + Turborepo)
- 后端: NestJS 微服务 + Claude Agent SDK
- 前端: React + Vite + Ant Design

包含服务:
- conversation-service: 对话服务 (Claude AI)
- user-service: 用户认证服务
- payment-service: 支付服务 (支付宝/微信/Stripe)
- knowledge-service: 知识库服务 (RAG + Neo4j)
- evolution-service: 自我进化服务
- web-client: 用户前端
- admin-client: 管理后台

基础设施:
- PostgreSQL + Redis + Neo4j
- Kong API Gateway
- Nginx 反向代理
- Docker Compose 部署配置

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 00:01:12 -08:00