Commit Graph

91 Commits

Author SHA1 Message Date
hailin 8e2c44edd0 feat(deploy): add rebuild command with --no-cache support
Add do_rebuild function to rebuild Docker images without cache.
Supports rebuilding individual services or all services at once.

Usage:
  ./deploy.sh rebuild conversation  # rebuild single service
  ./deploy.sh rebuild all           # rebuild all services

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-24 18:40:36 -08:00
hailin aaf43155d9 fix(deploy): include nginx and kong in full-reset
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-24 18:27:39 -08:00
hailin a54b82cab3 fix(deploy): full-reset now only resets data, not rebuild
- Remove image deletion and rebuild steps
- Only delete database volumes and restart services
- TypeORM synchronize:true handles schema creation
- Much faster: no recompilation needed

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-24 08:16:07 -08:00
hailin 7d9b87ef3c fix(conversation): use synchronize:true for 100% reliable schema sync
- Remove migration-based approach which kept failing
- Enable synchronize:true to auto-sync Entity with database
- Schema will always match Entity definition on startup

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-24 08:13:41 -08:00
hailin fc6078e4f8 fix(conversation): enable auto-migration on app startup
- Add migrations path to TypeORM config
- Set migrationsRun: true to run pending migrations on startup
- This ensures V2 columns are created automatically when app starts

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-24 08:12:50 -08:00
hailin b70035ad2f fix(migration): add production migration support for Docker
- Add data-source.prod.ts for compiled JS migrations
- Add migration:run:prod script to package.json
- Update deploy.sh to try prod migration first, fallback to dev
- Keep SQL fallback in full-reset as safety net with proper indexes

This ensures migrations work in Docker where ts-node may not be available.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-24 08:05:14 -08:00
hailin 90917b3ac8 fix(deploy): add V2 consulting columns in full-reset
- Add SQL to create consulting_stage, consulting_state, collected_info,
  recommended_programs, conversion_path, device_info columns
- Runs after migration step in full-reset to ensure V2 features work

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-24 08:02:35 -08:00
hailin 15ff2f1c3c feat(deploy): add full-reset command for complete system reset
- Add do_full_reset function that deletes all data and rebuilds system
- Preserves SSL certificates (stored in /etc/letsencrypt/)
- Requires explicit YES confirmation before proceeding
- Includes: stop services, clean Docker, rebuild, migrate, restart

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-24 07:17:56 -08:00
hailin 9f2bdee8d9 feat(conversation): integrate ClaudeAgentServiceV2 for consulting workflow
- Switch ConversationService to use ClaudeAgentServiceV2
- Pass consultingState and deviceInfo from conversation to context
- Handle state_update chunks and save updated state to database
- Move dotenv to dependencies for migration runtime

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-24 06:43:19 -08:00
hailin c0a9710943 feat(conversation): add TypeORM migration scripts and data-source config
- Add migration:run, migration:revert, migration:generate scripts
- Create data-source.ts for TypeORM CLI
- Add dotenv, ts-node, tsconfig-paths dependencies

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-24 06:38:41 -08:00
hailin cd5399eac3 feat(agent): implement consulting strategy engine with V2 agent service
- Add 8-stage consulting workflow (greeting → handoff)
- Create StrategyEngineService for state management and transitions
- Add ClaudeAgentServiceV2 with integrated strategy guidance
- Support old user recognition via get_user_context tool
- Add device info (IP, fingerprint) for new user icebreaking
- Extend ConversationEntity with consulting state fields
- Add database migration for new JSONB columns

Stages: greeting, needs_discovery, info_collection, assessment,
recommendation, objection_handling, conversion, handoff

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-24 06:32:07 -08:00
hailin 8352578bd3 fix(conversation): add explicit varchar type for intentType column
TypeORM requires explicit type for nullable string columns

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-23 08:30:03 -08:00
hailin dd66c3a892 fix(conversation): resolve TypeScript type errors in token tracking
- Fix Usage type cast by using unknown intermediate type
- Add PricingTier interface and proper Record type for PRICING

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-23 08:27:03 -08:00
hailin 849a4a3099 feat(conversation): add token usage tracking for API cost analysis
- Add TokenUsageEntity to store per-request token consumption
- Add TokenUsageService with cost calculation and statistics APIs
  - Record input/output/cache tokens per API call
  - Calculate estimated cost based on Claude pricing
  - Provide user/conversation/global stats aggregation
  - Support daily stats and top users ranking
- Integrate token tracking in ClaudeAgentService
  - Track latency, tool calls, response length
  - Accumulate tokens across tool loop iterations
- Add token_usages table to init-db.sql with proper indexes

This enables:
- Per-user token consumption tracking
- Cost analysis and optimization
- Future billing/quota features

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-23 08:23:58 -08:00
hailin c768e2aa53 fix(agent): stricter max_tokens calculation for response length control
- Reduce tokensPerChar from 2 to 1.8 for more accurate Chinese token estimation
- Use min() instead of max() to enforce upper limits on token counts
- CHAT: max 200 tokens (was min 256)
- SIMPLE_QUERY: max 600 tokens (was min 512)
- CLARIFICATION: max 300 tokens (was min 256)
- CONFIRMATION: max 400 tokens (was min 384)
- DEEP_CONSULTATION: 800-1600 tokens (was 1024-4096)
- ACTION_NEEDED: 500-1000 tokens (was 768-2048)

This should result in more concise AI responses that better match
the intent classifier's suggested length limits.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-23 08:02:34 -08:00
hailin d9b4c72894 feat(agent): implement 3-layer architecture for better response quality
Implement a three-layer architecture to improve AI response quality:

Layer 1 - Intent Classifier (intent-classifier.ts):
- Classifies user intent into 6 types: SIMPLE_QUERY, DEEP_CONSULTATION,
  ACTION_NEEDED, CHAT, CLARIFICATION, CONFIRMATION
- Determines suggested response length based on intent type
- Detects follow-up questions and extracts entities (visa types, etc.)
- Uses keyword matching for fast classification (no API calls)

Layer 2 - ReAct Agent (system-prompt.ts):
- Adds ReAct thinking framework to system prompt
- 4-step process: Understand -> Evaluate -> Act -> Generate
- Emphasizes concise responses, avoids redundant phrases
- Injects intent classification results to guide response strategy

Layer 3 - Response Gate (response-gate.ts):
- Quality checks: length, relevance, redundancy, completeness, tone
- Logs gate results for analysis and future optimization
- Can trim responses and remove redundant expressions

Integration (claude-agent.service.ts):
- Integrates all 3 layers in sendMessage flow
- Dynamically adjusts max_tokens based on intent type
- Collects full response for gate analysis

Documentation:
- Added AGENT_THREE_LAYER_ARCHITECTURE.md with detailed design docs

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-23 07:51:19 -08:00
hailin ad0f904f98 fix(knowledge): add pgvector transformer for TypeORM embedding columns
TypeORM doesn't natively support pgvector type. Add custom transformer
to convert between JavaScript arrays and pgvector string format [1,2,3].

Fixes: invalid input syntax for type vector errors

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-23 07:12:28 -08:00
hailin 71b98c2d07 fix(docker): pass OPENAI_BASE_URL to knowledge-service container
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-23 05:46:03 -08:00
hailin 91f8792110 feat(embedding): add OpenAI proxy support for IP-based URLs
- Add OPENAI_BASE_URL configuration to .env.example
- Update EmbeddingService to disable TLS verification for IP-based proxy URLs
- Mirror the same proxy handling pattern used in Anthropic API

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-23 05:42:22 -08:00
hailin 925283d8e6 fix(schema): support anonymous users in conversations and user_memories
- Remove foreign key constraint from conversations.user_id
- Remove foreign key constraint from user_memories.user_id
- Add FACT and INTENT to user_memories.memory_type enum
- Both tables now support anonymous users (UUID without registration)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-23 05:28:59 -08:00
hailin 10a2449d05 fix(conversation): use VARCHAR instead of enum for consistency with init-db.sql
- Change MessageEntity.role from enum to VARCHAR(20)
- Change MessageEntity.type from enum to VARCHAR(30)
- Change ConversationEntity.status from enum to VARCHAR(20)
- Add nullable: true to userId to match database schema
- Add length constraints to match database schema
- Convert enums to const objects with type exports for type safety

This ensures TypeORM entities match the database schema exactly,
avoiding potential issues with enum type creation in production.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-23 05:06:36 -08:00
hailin 0f56cea96a fix(schema): sync ORM entities with database schema
- Add missing 'type' column to MessageORM entity
- Add 'TEXT_WITH_ATTACHMENTS' to messages.type CHECK constraint
  (matches MessageType enum in conversation-service)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-23 05:00:38 -08:00
hailin fcffaebdc9 docs: add link to Agent Evaluation Report in README
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-23 04:56:27 -08:00
hailin 2069a3cf0a fix(evolution): resolve pgvector type conflict in SystemExperienceORM
The embedding column was declared as float[] but the database uses
VECTOR(1536) from pgvector. TypeORM doesn't natively support pgvector
types, causing 500 errors when querying the system_experiences table.

Fixed by:
- Changed column type to 'text' with select: false
- This prevents TypeORM from trying to select/map the vector column
- The embedding field is only used for similarity searches via raw SQL

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-23 04:46:18 -08:00
hailin 9ae2feff09 fix(db): correct admin password hash in init-db.sql
The previous hash was not working correctly with bcrypt.compare().
Updated to a verified working hash for password 'admin123'.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-23 04:40:47 -08:00
hailin 8b99021e0e feat(deploy): auto-init database with pgvector support
- Change PostgreSQL image from postgres:15-alpine to pgvector/pgvector:pg15
- Add init_database() function to auto-detect and initialize database
- Add 'db init' command for manual database initialization
- Database init runs automatically on first 'start all' deployment
- Includes admin account creation and pgvector extension verification

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-23 04:28:51 -08:00
hailin 632e27213a fix(deploy): update start_kong for DB-less mode
Kong is configured to use DB-less mode with declarative config
(kong/kong.yml), so remove the obsolete kong-database and
migrations steps from the deploy script.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-23 04:10:35 -08:00
hailin d5b751fdf8 chore: add real-time tools API key configs to .env.example
Add optional configuration for:
- GOOGLE_SEARCH_API_KEY + GOOGLE_CSE_ID (for web_search tool)
- EXCHANGE_RATE_API_KEY (optional, v4 API is free without key)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-23 00:53:37 -08:00
hailin 36ce824db8 docs: add comprehensive agent tools documentation
Document all 10 tools available to the immigration consulting agent:
- 6 knowledge & memory tools (search, off-topic check, assessment, payment, memory, context)
- 4 real-time tools (datetime, web search, exchange rate, news)

Includes input parameters, return formats, implementation details, and configuration.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-23 00:49:10 -08:00
hailin 4c125f3276 feat(agent): add 4 real-time tools for enhanced agent capabilities
Add the following real-time tools to ImmigrationToolsService:
- get_current_datetime: Get current date/time with timezone support
- web_search: Search internet for latest immigration news/policies (Google CSE)
- get_exchange_rate: Query real-time currency exchange rates (for investment immigration)
- fetch_immigration_news: Fetch latest immigration announcements

All tools include graceful degradation with fallback responses when external APIs are unavailable.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-23 00:46:55 -08:00
hailin 022b1eb608 docs: add QMAS (优才计划) reference image
添加优才计划相关参考图片

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-22 23:47:03 -08:00
hailin 911132ab3e feat(agent): upgrade to Level 3 with real RAG, Memory and Evolution integration
## Summary
Upgrade iConsulting from Level 2 (48 points) to Level 3 (68 points) by
implementing real service-to-service integration between conversation-service
and knowledge-service.

## New Files
- knowledge-client.service.ts: HTTP client for knowledge-service APIs
- knowledge.module.ts: NestJS module for KnowledgeClientService
- AGENT_EVALUATION_REPORT.md: Agent capability evaluation report
- LEVEL3_UPGRADE_PLAN.md: Upgrade plan and completion report

## Changes

### RAG Integration
- search_knowledge tool now calls /api/v1/knowledge/retrieve
- check_off_topic tool calls /api/v1/knowledge/check-off-topic
- Results include real vector similarity search from knowledge base

### Memory Integration
- save_user_memory writes to PostgreSQL + Neo4j via knowledge-service
- collect_assessment_info saves user data to long-term memory
- generate_payment records payment intent to user memory
- New get_user_context tool retrieves user's historical memories

### Evolution Integration
- getAccumulatedExperience() fetches approved system experiences
- sendMessage() dynamically injects experiences into system prompt
- System learns from approved experiences across all conversations

## Expected Score Improvement
| Dimension  | Before | After | Delta |
|------------|--------|-------|-------|
| Tool Use   | 14/20  | 18/20 | +4    |
| Memory     | 12/20  | 16/20 | +4    |
| RAG        | 10/20  | 16/20 | +6    |
| Evolution  | 8/20   | 14/20 | +6    |
| Total      | 48     | 68    | +20   |

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-22 23:45:58 -08:00
hailin 4dbd6f075f docs: reorganize documentation and add architecture document
Changes:
- Create docs/ directory for centralized documentation
- Move DEVELOPMENT_GUIDE.md from root to docs/
- Add comprehensive ARCHITECTURE.md with:
  - System overview and architecture diagrams (ASCII)
  - Microservices responsibilities (6 services)
  - Technology stack details
  - Data flow diagrams (user conversation, admin evolution)
  - Production deployment architecture
  - Kong API routing table
  - Port mapping reference
  - Database schema overview (PostgreSQL, Neo4j, MinIO)
  - Security design notes
- Update README.md with documentation index table

Documentation structure:
  docs/
  ├── ARCHITECTURE.md      - System architecture reference
  └── DEVELOPMENT_GUIDE.md - Development guide and code examples

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-22 23:05:41 -08:00
hailin e6e69f15ce fix(admin): correct 401 redirect path to include /admin prefix
The API interceptor was redirecting to /login on 401 errors, but since
admin-client is deployed under /admin/, it should redirect to /admin/login.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-10 07:57:43 -08:00
hailin 731323ad7c fix(admin): add basename to BrowserRouter for /admin/ deployment
The router needs basename="/admin" to work correctly when deployed
under the /admin/ subpath.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-10 07:50:22 -08:00
hailin cb0b8c6ea9 fix(admin): configure base path for /admin/ deployment
- Add base: '/admin/' to vite.config.ts for proper asset paths
- Replace vite.svg favicon with inline SVG emoji icon
- Fixes 404 errors when accessing admin panel at /admin/ path

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-10 07:35:52 -08:00
hailin 5ff53b8ba7 fix(chat): align input area elements vertically centered
- Change flex container from items-end to items-center
- Use top-1/2 -translate-y-1/2 for send button vertical centering
- Adjust paperclip button padding for consistent sizing

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-10 06:49:20 -08:00
hailin 3d120e1ce3 fix(chat): hide textarea scrollbar when content fits
- Set overflow-hidden by default on textarea
- Only show scrollbar when content exceeds max height (200px)
- Fix scrollbar appearing on empty input

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-10 06:44:59 -08:00
hailin 8a39505ee6 feat(chat): add paste and drag-drop file upload support
- Add clipboard paste handler for images and files (Ctrl+V / Cmd+V)
- Add drag-and-drop zone with visual feedback
- Update placeholder text to inform users about new features
- Improve file upload UX with drop overlay

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-10 06:22:59 -08:00
hailin 2570e4add9 fix(file-service): specify explicit column types for TypeORM entities
Fix DataTypeNotSupportedError by explicitly specifying PostgreSQL column types
for nullable fields that TypeORM was incorrectly inferring as Object type.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-10 06:05:36 -08:00
hailin d4925719fc feat(multimodal): add file upload and image support for chat
- Add MinIO object storage to docker-compose infrastructure
- Create file-service microservice for upload management with presigned URLs
- Add files table to database schema
- Update nginx and Kong for MinIO proxy routes
- Implement file upload UI in chat InputArea with drag-and-drop
- Add attachment preview in MessageBubble component
- Update conversation-service to handle multimodal messages
- Add Claude Vision API integration for image analysis

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-10 05:34:41 -08:00
hailin 7adbaaa871 fix(db): use POSTGRES_* env vars in knowledge and evolution services
These services were using DB_HOST, DB_USER etc. but docker-compose
sets POSTGRES_HOST, POSTGRES_USER etc.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-10 02:57:39 -08:00
hailin c98cae2e39 fix(payment): use PORT env variable instead of PAYMENT_SERVICE_PORT
Payment service was listening on wrong port (3004) because it used
PAYMENT_SERVICE_PORT which wasn't set in docker-compose.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-10 02:48:44 -08:00
hailin 224e1fb509 fix(health): exclude /health endpoint from API prefix
The health check endpoint should be at /health not /api/v1/health
for Docker health checks to work properly.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-10 02:30:24 -08:00
hailin 223aa25af1 fix(docker): add health check endpoints and fix IPv6 issue
- Add /health endpoints to all NestJS services (user, payment, knowledge, conversation, evolution)
- Fix nginx healthcheck to use 127.0.0.1 instead of localhost (IPv6 issue)
- Add healthcheck configuration to docker-compose for all backend services
- Use start_period to allow services time to initialize before health checks

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-10 02:13:42 -08:00
hailin 93050b6889 perf(claude): enable Prompt Caching for ~90% cost savings on system prompt 2026-01-10 01:42:33 -08:00
hailin c914693f94 feat(web): add markdown rendering for AI responses 2026-01-10 01:33:00 -08:00
hailin 0cd667b5c8 fix(websocket): use singleton socket to prevent disconnection on re-render 2026-01-10 01:26:49 -08:00
hailin bd65a431aa fix(websocket): add debug logging for connection issues 2026-01-10 01:24:11 -08:00
hailin d073bd5a9d fix(websocket): configure Socket.IO path for nginx proxy 2026-01-10 01:12:58 -08:00