hailin
/
it0
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
442 Commits 1 Branch 0 Tags 9.5 MiB
Commit Graph

12 Commits

Author SHA1 Message Date
hailin 8865985019 feat(agent-instance-chat): 实现用户与自己的 OpenClaw 智能体直接对话功能 
## 功能概述
用户可在「我的智能体」页面点击运行中的 OpenClaw 实例卡片，
直接打开与该智能体的专属对话页面，完整复用 iAgent 的聊天 UI
（流式输出、工具时间线、审批卡片、语音输入等），同时保证
iAgent 对话完全不受影响。

## 架构设计
- 使用 Riverpod ProviderScope 子作用域覆盖 chatRemoteDatasourceProvider
  / chatProvider / sessionListProvider，实现 iAgent 与实例对话的
  provider 完全隔离，无任何共享状态。
- OpenClaw bridge 采用已有的 /task-async 异步回调模式：
    Flutter → POST /api/v1/agent/instances/:id/tasks（立即返回 sessionId/taskId）
    → 订阅 WS /ws/agent（等待事件）
    → Bridge 完成后 POST /api/v1/agent/instances/openclaw-app-callback（公开端点）
    → 后端发 WS text+completed 事件 → Flutter 收到回复
- 每个实例的会话通过 agent_sessions.agent_instance_id 字段隔离，
  会话抽屉只显示当前实例的历史记录。

## 后端变更
### packages/shared/database/src/migrations/013-add-agent-instance-id-to-sessions.sql
- 新增迁移：ALTER TABLE agent_sessions ADD COLUMN agent_instance_id UUID NULL
- 为按实例过滤会话建立索引

### packages/services/agent-service/src/domain/entities/agent-session.entity.ts
- 新增可选字段 agentInstanceId: string（对应 agent_instance_id 列）
- iAgent 会话该字段为 null；实例聊天会话存储对应的 instance UUID

### packages/services/agent-service/src/infrastructure/repositories/session.repository.ts
- 新增 findByInstanceId(tenantId, agentInstanceId) 方法
- 用于 GET /instances/:id/sessions 按实例过滤会话列表

### packages/services/agent-service/src/interfaces/rest/controllers/agent.controller.ts
新增三个端点（注意：已知存在以下待修复问题，见后续 fix commit）：
1. POST /api/v1/agent/instances/:instanceId/tasks
   - 校验 instance 归属（userId 匹配）和 running 状态
   - 创建会话（engineType='openclaw'，携带 agentInstanceId）
   - 保存用户消息到 conversation_messages 表
   - 向 OpenClaw bridge POST /task-async，sessionKey=it0:{sessionId}
   - 立即返回 { sessionId, taskId }，Flutter 订阅 WS 等待回调
2. GET /api/v1/agent/instances/:instanceId/sessions
   - 返回该实例的会话列表（含 title/status/时间戳）
3. POST /api/v1/agent/instances/openclaw-app-callback（公开端点，无 JWT）
   - bridge 完成后回调此端点
   - 成功：发 WS text+completed 事件，保存 assistant 消息，更新 task 状态
   - 失败/超时：发 WS error 事件，标记 task 为 FAILED
- 注入 AgentInstanceRepository 依赖
- 新增私有方法 createInstanceSession()

### packages/gateway/config/kong.yml
- 新增 openclaw-app-callback-public service（无 JWT 插件）
- 路由：POST /api/v1/agent/instances/openclaw-app-callback
- 必须在 agent-service 之前声明，确保路由优先匹配（同 wecom-public 模式）

## Flutter 变更
### it0_app/lib/core/config/api_endpoints.dart
- 新增 instanceTasks(instanceId) 和 instanceSessions(instanceId) 静态方法

### it0_app/lib/features/chat/presentation/pages/chat_page.dart
- 新增可选参数 agentName（默认 null = iAgent 模式）
- agentName != null 时：AppBar 显示智能体名称，隐藏语音通话按钮
- 不传 agentName 时行为与原来完全一致，iAgent 功能零影响

### it0_app/lib/features/my_agents/presentation/pages/my_agents_page.dart
- _InstanceCard 新增 onTap 回调参数
- 卡片用 Material+InkWell 包裹，支持圆角水波纹点击效果
- 新增 _openInstanceChat() 顶层函数：
    running → 滑入式跳转到 AgentInstanceChatPage
    其他状态 → SnackBar 提示（部署中/已停止/错误）
- 导入 AgentInstanceChatPage

### it0_app/lib/features/agent_instance_chat/（新建功能模块）
data/datasources/agent_instance_chat_remote_datasource.dart:
- AgentInstanceChatDatasource implements ChatRemoteDatasource
- 通过组合模式包装 ChatRemoteDatasource 委托所有通用操作
- 覆盖 createTask → POST /api/v1/agent/instances/:id/tasks
- 覆盖 listSessions → GET /api/v1/agent/instances/:id/sessions（仅当前实例会话）

presentation/pages/agent_instance_chat_page.dart:
- AgentInstanceChatPage(instance: AgentInstance)
- ProviderScope 子作用域覆盖三个 provider 实现完全隔离：
    chatRemoteDatasourceProvider → AgentInstanceChatDatasource
    chatProvider → 独立 ChatNotifier 实例（与 iAgent 零共享）
    sessionListProvider → 仅当前实例的会话列表
- child: ChatPage(agentName: instance.name) 完整复用 UI

## 已知待修复问题（下一个 commit）
1. [安全] 鉴权检查逻辑：if (userId && ...) 应为 if (!userId || ...)
2. [可靠性] fetch 未处理 HTTP 4xx/5xx 错误，任务可能永久挂起
3. [可靠性] bridge 回调无超时机制，bridge 崩溃后任务永久 RUNNING
4. [UX] robotStateProvider 未在子 ProviderScope 覆盖，头像动画反映 iAgent 状态
5. [UX] 实例聊天附件 UI 未禁用，上传附件被静默丢弃
6. [UX] 语音消息在实例模式下错误路由到 iAgent 引擎（非 OpenClaw）
7. [DB] 002 模板未加 agent_instance_id 列，新租户缺失此字段

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-09 19:30:38 -07:00
hailin 7d5840c245 feat(openclaw): Phase 1 — server pool + agent instance deployment infrastructure 
## inventory-service
- New: pool_servers table (public schema, platform-admin managed)
- New: PoolServer entity, PoolServerRepository, PoolServerController
- CRUD endpoints at /api/v1/inventory/pool-servers
- Internal /deploy-creds endpoint (x-internal-api-key protected) for SSH key retrieval
- increment/decrement endpoints for capacity tracking

## agent-service
- New: agent_instances table (tenant schema)
- New: AgentInstance entity, AgentInstanceRepository, AgentInstanceController
- New: AgentInstanceDeployService — SSH-based docker deployment
  - Queries pool server availability from inventory-service
  - AES-256 encrypts OpenClaw gateway token at rest
  - Allocates host ports in range 20000-29999
  - Fires docker run for it0hub/openclaw-bridge:latest
  - Async deploy with error capture
- Added ssh2 dependency for SSH execution
- Added INVENTORY_SERVICE_URL, INTERNAL_API_KEY, VAULT_MASTER_KEY to docker-compose

## openclaw-bridge (new package)
- packages/openclaw-bridge/ — custom Docker image
- Two processes via supervisord: OpenClaw gateway + IT0 Bridge (Node.js)
- IT0 Bridge exposes REST API on port 3000:
  GET /health, GET /status, POST /task, GET /sessions, GET /metrics
- Connects to OpenClaw gateway at ws://127.0.0.1:18789 via WebSocket RPC
- Sends heartbeat to IT0 agent-service every 60s
- Dockerfile: multi-stage build (openclaw source + bridge TS compilation)

## Web Admin
- New: /server-pool page — list/add/edit/delete pool servers with capacity bars
- New: /openclaw-instances page — cross-tenant instance monitoring with status filter
- Sidebar: added 服务器池 (Database icon) + OpenClaw 实例 (Boxes icon) to platform_admin nav

## Flutter App
- my_agents_page: rewritten to show real AgentInstance data from /api/v1/agent/instances
- Added AgentInstance model with status-driven UI (running/deploying/stopped/error)
- Status badges with color coding + spinner for deploying state
- Summary chips showing running vs stopped counts
- api_endpoints.dart: added agentInstances endpoint

## Design docs
- OPENCLAW_INTEGRATION_PLAN.md: complete architecture document with all confirmed decisions

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-07 11:11:21 -08:00
hailin d5930ff4c8 feat(app): redesign navigation — floating robot FAB + 4-tab layout 
- Add animated robot avatar widget (CustomPainter, 5 states: idle/thinking/executing/speaking/alert)
- Add FloatingRobotFab that mirrors chatProvider AgentStatus as robot animation state
- Replace 5-tab nav (dashboard/chat/tasks/alerts/settings) with 4-tab (home/my-agents/billing/profile)
- Chat is now pushed full-screen from the robot FAB with slide-up transition
- HomePage: active agent status card + official agent horizontal scroll + quick tips
- MyAgentsPage: empty state with 3-step guide + template grid; shows list when agents exist
- ProfilePage: merged settings + prominent billing entry (replaces old SettingsPage as tab)
- ChatPage AppBar: robot avatar replaces plain text title, reflects real-time agent state
- Add agentConfigs endpoint to ApiEndpoints

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-07 09:42:17 -08:00
hailin 71ea80972d feat(auth): add SMS OTP verification for phone registration and login 
- auth-service: add SmsService (Aliyun SMS) + RedisProvider for OTP storage
- POST /api/v1/auth/sms/send — send OTP (rate limited 1/min per phone)
- POST /api/v1/auth/sms/verify — verify OTP only
- POST /api/v1/auth/login/otp — passwordless login with phone + OTP
- register endpoint now requires smsCode when registering with phone
- Web Admin register page: add OTP input + 60s countdown button for phone mode
- Flutter login page: add 验证码登录 tab with phone + OTP flow
- SMS enabled via ALIYUN_ACCESS_KEY_ID/SECRET + SMS_ENABLED=true env vars
- Falls back to mock mode (logs code) when env vars not set

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-07 06:43:27 -08:00
hailin 8a48e92970 fix: use domain names for API access, China IP for LiveKit 
Flutter app now uses https://it0api.szaiai.com (nginx reverse proxy)
instead of direct IP:port. LiveKit URL uses China IP 14.215.128.96
for lower latency from domestic mobile clients.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-01 21:44:25 -08:00
hailin 68ee2516d5 fix: use host networking for voice services to eliminate docker-proxy overhead 
Bridge mode created 600+ docker-proxy processes for LiveKit's UDP port-range
mappings (30000-30100, 50000-50200). Switch livekit-server, voice-agent, and
voice-service to network_mode: host for zero-overhead networking.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-01 19:58:32 -08:00
hailin 94a14b3104 feat: migrate voice call from WebSocket/PCM to LiveKit WebRTC 
实时语音对话架构迁移：WebSocket → LiveKit WebRTC

## 背景
原语音通话架构基于 FastAPI WebSocket 传输原始 PCM，管道串行执行
（VAD → 批量STT → Agent → 攒句 → 批量TTS），首音频延迟约 6 秒。
迁移到 LiveKit Agents 框架后，利用 WebRTC 传输 + 流水线并行，
预期延迟降至 1.5-2 秒。

## 架构
Flutter App ←── WebRTC (Opus/UDP) ──→ LiveKit Server ←──→ Voice Agent
  livekit_client                      (自部署, Go)       (Python, LiveKit Agents SDK)
                                                          ├─ VAD (Silero)
                                                          ├─ STT (faster-whisper / OpenAI)
                                                          ├─ LLM (自定义插件 → agent-service)
                                                          └─ TTS (Kokoro / OpenAI)

关键设计：LLM 不直接调用 Claude API，而是通过自定义插件代理到现有
agent-service，保留 Tool Use、会话历史、租户隔离等能力。

## 新增服务

### voice-agent (packages/services/voice-agent/)
LiveKit Agent Worker，包含：
- agent.py: 入口，prewarm() 预加载模型，entrypoint() 编排会话
- plugins/agent_llm.py: 自定义 LLM 插件，代理 agent-service API
  - POST /api/v1/agent/tasks 创建任务
  - WS /ws/agent 订阅流式事件 (stream_event)
  - 跨轮复用 session_id 保持对话上下文
- plugins/whisper_stt.py: 本地 faster-whisper STT (批量识别)
- plugins/kokoro_tts.py: 本地 Kokoro-82M TTS (24kHz PCM)
- config.py: pydantic-settings 配置

### LiveKit Server (deploy/docker/)
- livekit.yaml: 信令端口 7880, RTC TCP 7881, UDP 50000-50200
- docker-compose.yml: 新增 livekit-server + voice-agent 容器

### LiveKit Token 端点
- voice-service/src/api/livekit_token.py:
  POST /api/v1/voice/livekit/token
  生成 Room JWT，嵌入 auth_header 到 AgentDispatch metadata

## Flutter 客户端改造
- agent_call_page.dart: 从 ~814 行简化到 ~380 行
  - 替换: WebSocketChannel, AudioRecorder, PcmPlayer, 手动心跳/重连
  - 使用: Room.connect(), setMicrophoneEnabled(true), LiveKit 事件监听
  - 波形动画改用 participant.audioLevel
- pubspec.yaml: 添加 livekit_client: ^2.3.0
- app_config.dart: 增加 livekitUrl 字段
- api_endpoints.dart: 增加 livekitToken 端点

## 配置说明 (环境变量)
- STT_PROVIDER: local (默认, faster-whisper) / openai
- TTS_PROVIDER: local (默认, Kokoro) / openai
- WHISPER_MODEL: base (默认) / small / medium / large
- WHISPER_LANGUAGE: zh (默认)
- KOKORO_VOICE: zf_xiaoxiao (默认)
- DEVICE: cpu (默认) / cuda

## 不变的部分
- agent-service: 完全不改，voice-agent 通过现有 API 调用
- voice-service 核心: pipeline/STT/TTS/VAD 保留 (Twilio 备用)
- Kong 网关: 现有路由不变
- 数据库: 无 schema 变更

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-28 08:55:33 -08:00
hailin f5d9b1f04f feat: add app upgrade system with self-hosted APK update support 
- Add core/updater module: version checker, download manager (resumable + SHA-256),
  APK installer, app market detector, self-hosted updater with progress dialogs
- Add Android native MethodChannels for APK installation and market detection
- Add FileProvider config and REQUEST_INSTALL_PACKAGES permission
- Wire UpdateService singleton into main.dart initialization
- Add auto-check on home entry with cooldown + app resume detection
- Add manual "检查更新" button and dynamic version display in settings
- Fix chat page: bottom overflow, bash spinner persistence, collapsible results
- Merge standing orders into tasks page as second tab

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-26 22:35:01 -08:00
hailin 092a561867 feat: 完成 iAgent App 三大功能 + 修复租户上下文 
## 功能一：设置页（完整实现）
- 新增浅色主题（lightTheme），支持深色/浅色/跟随系统三种模式
- app.dart 接入 themeMode 动态切换
- 设置页完整重写：个人信息编辑、修改密码、主题切换、通知开关
- 新增 settings_remote_datasource 对接后端 admin/settings API
- settings_providers 新增 AccountProfileNotifier 管理远程个人资料

## 功能二：语音通话（音频集成）
- 添加 flutter_sound 依赖，创建 PcmPlayer 流式 PCM 播放器
- agent_call_page 替换空壳：真实麦克风采集（record + GTCRN 降噪）
- 真实 PCM 16kHz 流式播放，基于 RMS 能量驱动波形动画
- 修复 WebSocket URL 路径：/ws/voice/ → /api/v1/voice/ws/
- voice_repository_impl 支持后端返回相对路径自动拼接

## 功能三：推送通知（WebSocket MVP）
- 添加 flutter_local_notifications + socket_io_client 依赖
- 创建 AppNotification 实体、NotificationService（Socket.IO 连接 comm-service）
- 通知 providers：列表管理 + 未读计数
- 登录后自动连接通知服务，登出断开
- 底部导航 Alerts 标签添加未读角标（Badge）
- AndroidManifest 添加 POST_NOTIFICATIONS 权限
- main.dart 初始化本地通知插件

## 修复：租户上下文未初始化（500错误）
- 根因：登录后未设置 currentTenantIdProvider，导致 X-Tenant-Id 头缺失
- Flutter 端：login() 成功后从 JWT 设置 tenantId，logout 时清除
- 后端：tenant-context.middleware 增加 JWT tenantId 回退逻辑
- AuthUser 模型新增 tenantId 字段解析

新增 5 个文件，修改 16 个文件，添加 3 个依赖包

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-23 01:10:52 -08:00
hailin 51430adafc fix: 修复登录问题 — 重置用户密码 + 配置生产环境API地址 
问题排查过程:
  1. 用户 hailin@it0.com 存在于 public.users，is_active=true
  2. 直接调用 auth-service login 接口返回 401 Invalid credentials
  3. 确认是密码不匹配 — 将密码重置为 admin123 (与 admin 账号相同)
  4. 重置后登录成功，Kong Gateway 路由也正常

App配置修改:
  - development: 端口从 8000 改为 18000 (匹配 Kong 映射)
  - production: 指向服务器 http://154.84.135.121:18000
  - 默认使用 production 配置 (之前是 development)

登录凭据:
  - admin@it0.com / admin123 (管理员)
  - hailin@it0.com / admin123 (运维员，请登录后修改密码)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-22 16:56:38 -08:00
hailin a568558585 feat: replace speech_to_text with GTCRN ML noise reduction + backend STT 
Replace traditional on-device speech_to_text with a modern pipeline:
- Record audio via `record` package with hardware noise suppression
- Apply GTCRN neural denoising (sherpa-onnx, ICASSP 2024, 48K params)
- Trim silence, POST to backend /voice/transcribe (faster-whisper)

Changes:
- Add /transcribe endpoint to voice-service for audio file upload
- Add SpeechEnhancer wrapper for sherpa-onnx GTCRN model (523KB)
- Rewrite chat_page.dart voice input: record → denoise → transcribe
- Keep NoiseReducer.trimSilence for silence removal only
- Upgrade record to v6.2.0, add sherpa_onnx, path_provider

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-22 07:59:15 -08:00
hailin 00f8801d51 Initial commit: IT0 AI-powered server cluster operations platform 
Full-stack monorepo with DDD + Clean Architecture:
- Backend: 7 NestJS microservices + 5 shared libraries (TypeScript)
- Mobile: Flutter app with Riverpod (Dart)
- Web Admin: Next.js dashboard with Zustand + React Query
- Voice: Python voice service (STT/TTS/VAD)
- Infra: Docker Compose, K8s manifests, Turborepo build

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-08 22:54:37 -08:00