Add docling (Python document parsing service) to all deploy.sh operations:
- SERVICE_PORTS, DOCKER_SERVICES maps
- build, rebuild, start, stop, restart commands
- start_all_backend (ordered before knowledge-service, which depends on it)
- Help text and examples updated
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add IBM Docling as a Python FastAPI microservice for high-quality document
parsing with table structure recognition (TableFormer ~94% accuracy) and
OCR support, replacing pdf-parse/mammoth as the primary text extractor.
Architecture:
- New docling-service (Python FastAPI, port 3007) in Docker network
- knowledge-service calls docling-service via HTTP POST multipart/form-data
- Graceful fallback: if Docling fails, falls back to pdf-parse/mammoth
- Text/Markdown files skip Docling (no benefit for plain text)
Changes:
- New: packages/services/docling-service/ (main.py, Dockerfile, requirements.txt)
- docker-compose.yml: add docling-service, wire DOCLING_SERVICE_URL to
knowledge-service, add missing FILE_SERVICE_URL to conversation-service
- text-extraction.service.ts: inject ConfigService, add extractViaDocling()
with automatic fallback to legacy extractors
- .env.example: add FILE_SERVICE_PORT/URL and DOCLING_SERVICE_PORT/URL
Inter-service communication map:
conversation-service → file-service (FILE_SERVICE_URL, attachments)
conversation-service → knowledge-service (KNOWLEDGE_SERVICE_URL, RAG)
knowledge-service → docling-service (DOCLING_SERVICE_URL, document parsing)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Enforce Claude API file size limits at upload time with user-friendly
error messages:
- Images: max 5MB (Claude API hard limit)
- PDF: max 25MB (32MB request limit minus headroom)
- Other documents: max 50MB (general upload limit)
Replaced duplicate ALLOWED_TYPES/MAX_FILE_SIZE in InputArea with shared
validateFile() from fileService, showing alert() on rejection.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Claude API enforces a hard 5MB limit per image (not 20MB as previously
set). PDFs have a 32MB total request limit; set individual PDF cap to
25MB to leave room for prompt/messages. The downloadAsBase64 method now
accepts a per-type maxSize parameter.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
MinIO presigned URLs use Docker-internal hostname (minio:9000), making
them inaccessible from both Claude API servers and user browsers.
Changes:
- file-service: add /files/:id/content and /files/:id/thumbnail proxy
endpoints that stream file data from MinIO
- file-service: toResponseDto now returns API proxy paths instead of
MinIO presigned URLs
- coordinator: buildAttachmentBlocks now downloads files via file-service
internal API (http://file-service:3006) and converts to base64 for
Claude API (images, PDFs) or embeds text content directly
- Configurable FILE_SERVICE_URL env var for service-to-service calls
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Claude API cannot fetch arbitrary URLs. Text-based attachments (txt, csv,
json, md) are now downloaded via their presigned MinIO URL and embedded
directly as text blocks. PDF uses Claude's native document block. Added
50KB size limit with truncation for large text files.
buildMessages() is now async to support text content fetching.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1. Coordinator now sends all attachment types to Claude:
- Images → native image blocks (existing)
- PDF → native document blocks (Claude PDF support)
- Text files (txt, csv, json, md) → text blocks with filename
Extracted common buildAttachmentBlocks() helper.
2. File-service generates thumbnails on image upload:
- Uses sharp to resize to 400x400 max (inside fit, no upscale)
- Output as WebP at 80% quality for smaller file size
- Stored in MinIO under thumbnails/ prefix
- Generated for both direct upload and presigned URL confirm
- Non-blocking: thumbnail failure doesn't break upload
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
FileORM had tenant_id column but FileEntity domain class was missing it,
causing "column FileORM.tenant_id does not exist" errors on production.
- Add tenantId to FileEntity (constructor, create, fromPersistence)
- Pass tenantId in repository toEntity() mapping
- Add idempotent migration script for files.tenant_id + indexes
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The model was putting internal notes like "引导回移民话题" in the followUp
field instead of actual user-facing questions. Two fixes:
1. Schema: describe followUp as "必须以？结尾，禁止填写内部策略备注"
2. agent-loop: only yield followUp if it contains ？or ? (question mark)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
claude-sonnet-4-20250514 does not support output_config (structured outputs).
Changed coordinator model to claude-sonnet-4-5-20250929 which supports it.
Specialist agents remain on their original models (no output_config needed).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- findForEvolution() now excludes DELETED conversations (should not learn from deleted data)
- getConversation() rejects DELETED conversations for user-facing operations (sendMessage, getMessages, etc.)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The delete conversation endpoint was a no-op — it verified ownership but
never actually modified the record. Users saw conversations disappear
(frontend optimistic removal) but they reappeared on refresh.
Changes:
- conversation.entity.ts: Add DELETED status, softDelete() and isDeleted()
- conversation.service.ts: Call softDelete() + update instead of no-op
- conversation-postgres.repository.ts: Exclude DELETED conversations
from findByUserId() queries so they don't appear in user's list
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
AdminConversationController's GET /:id was intercepting requests to
AdminEvaluationRuleController (matching "evaluation-rules" as an id param).
Similarly, DELETE /:id was matching "cache" as an id.
Changes:
- conversation.module.ts: Register AdminMcpController and
AdminEvaluationRuleController before AdminConversationController
(more specific prefixes must come first in NestJS)
- admin-evaluation-rule.controller.ts: Move static routes (POST /test,
DELETE /cache) before dynamic routes (GET/:id, DELETE/:id)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The AdminEvaluationRuleController in ConversationModule needs the
EVALUATION_RULE_REPOSITORY token. Even though AgentsModule is @Global(),
Symbol-based providers must be explicitly exported to be available
in other modules.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds tenant_id UUID column to user_memories, system_experiences,
knowledge_articles, and knowledge_chunks tables with default tenant
backfill and indexes. Migration already applied to production DB.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
claude-haiku-3-5-20241022 returns 404 on the proxy. Updated to
claude-haiku-4-5-20251001 in agent configs and context injector.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The mapEventToStreamChunk was mapping both 'usage' (per-turn) and 'end'
(final) events to type 'end', causing the gateway to emit multiple
stream_end events. This made the frontend create a separate message
bubble (with its own bot avatar) for each agent loop turn.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
WebSocket gateway was missing AsyncLocalStorage tenant context setup,
causing 'Tenant context not set' error on every message. Now extracts
tenantId from handshake and wraps handleMessage in tenantContext.runAsync().
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add tenant_id column to conversations, messages, token_usages tables
- Create standalone migration SQL script for production deployment
- Add agent_executions table to init-db.sql for new installations
- Fix MessageORM created_at nullable mismatch with database schema
- Backfill existing data with default tenant ID
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Replace IKnowledgeClient TypeScript interface (erased at runtime) with KnowledgeClientService concrete class injection
- Fix method signatures to match KnowledgeClientService API:
- getUserMemories() → getUserTopMemories(), field type → memoryType
- retrieveForPrompt(query, userId) → retrieveForPrompt({ query, userId })
- getRelevantExperiences(query, n) → searchExperiences({ query, limit: n }), field type → experienceType
- Remove unused ContextData import
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add x-admin-id header to API interceptor from auth store
- Required for super admin tenant management APIs
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Fix toDto method to handle statDate being string or Date
- Fix getTrendData method to handle statDate being string or Date
- PostgreSQL date type returns string, not Date object
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Add is_super_admin column to admins table in migration script
- Add index for is_super_admin column
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Update AdminPostgresRepository.findByUsername to support super admin
- Add fallback to find super admin by username and isSuperAdmin flag
- Add is_super_admin column to admins table
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Fix nginx admin location try_files to use relative path for alias directive
- Add database migrations for multi-tenancy support in user-service
- Add tenant_id column to users, user_profiles, user_contacts, verification_codes tables
- Create migration rollback scripts and documentation
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Add tenants feature module with Clean Architecture structure
- Create tenantsApi for super admin endpoints
- Add React Query hooks for tenant CRUD operations
- Implement TenantsPage with statistics, list, and modals
- Add tenant route and sidebar menu item
- Support create/edit tenant, suspend/activate, admin management
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add SuperAdmin module for tenant management (CRUD, suspend/activate)
- Add tenant management REST API (/super-admin/tenants/*)
- Add user profile menu in ChatSidebar with dropdown
- Add ProfilePage and BindPhonePage for user account management
- Update init-db.sql with tenant_id columns for all 16 tables
- Add database seed script (scripts/seed.ts) with ts-node
- Integrate db:seed into deploy.sh rebuild command
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Use SimpleTenantFinder for evolution-service middleware (same as other services)
- Keep TenantFinderService available via TenantModule for super-admin
- Add useExternalFinder option to TenantContextModule
- Remove @Global() decorator from TenantContextModule class
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
All services were providing TenantContextService directly without
making it global, causing DI resolution failures in child modules.
Now using TenantContextModule.forRoot() which exports TenantContextService
globally so all repositories can access it.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
npm install was clearing the @iconsulting/shared folder that was
copied before it. Moving the COPY command after npm install ensures
the shared package remains in node_modules.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- deploy.sh deploy now uses --no-cache by default to ensure
@iconsulting/shared package is correctly bundled into images
- Add --cache option if user wants to use Docker cache for faster builds
- This fixes the "Cannot find module '@iconsulting/shared'" error
that occurred when Docker used cached images without the latest shared code
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
hailin