hailin
/
it0
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
241 Commits 1 Branch 0 Tags 8 MiB
Commit Graph

8 Commits

Author SHA1 Message Date
hailin 8a48e92970 fix: use domain names for API access, China IP for LiveKit 
Flutter app now uses https://it0api.szaiai.com (nginx reverse proxy)
instead of direct IP:port. LiveKit URL uses China IP 14.215.128.96
for lower latency from domestic mobile clients.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-01 21:44:25 -08:00
hailin 68ee2516d5 fix: use host networking for voice services to eliminate docker-proxy overhead 
Bridge mode created 600+ docker-proxy processes for LiveKit's UDP port-range
mappings (30000-30100, 50000-50200). Switch livekit-server, voice-agent, and
voice-service to network_mode: host for zero-overhead networking.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-01 19:58:32 -08:00
hailin 94a14b3104 feat: migrate voice call from WebSocket/PCM to LiveKit WebRTC 
实时语音对话架构迁移：WebSocket → LiveKit WebRTC

## 背景
原语音通话架构基于 FastAPI WebSocket 传输原始 PCM，管道串行执行
（VAD → 批量STT → Agent → 攒句 → 批量TTS），首音频延迟约 6 秒。
迁移到 LiveKit Agents 框架后，利用 WebRTC 传输 + 流水线并行，
预期延迟降至 1.5-2 秒。

## 架构
Flutter App ←── WebRTC (Opus/UDP) ──→ LiveKit Server ←──→ Voice Agent
  livekit_client                      (自部署, Go)       (Python, LiveKit Agents SDK)
                                                          ├─ VAD (Silero)
                                                          ├─ STT (faster-whisper / OpenAI)
                                                          ├─ LLM (自定义插件 → agent-service)
                                                          └─ TTS (Kokoro / OpenAI)

关键设计：LLM 不直接调用 Claude API，而是通过自定义插件代理到现有
agent-service，保留 Tool Use、会话历史、租户隔离等能力。

## 新增服务

### voice-agent (packages/services/voice-agent/)
LiveKit Agent Worker，包含：
- agent.py: 入口，prewarm() 预加载模型，entrypoint() 编排会话
- plugins/agent_llm.py: 自定义 LLM 插件，代理 agent-service API
  - POST /api/v1/agent/tasks 创建任务
  - WS /ws/agent 订阅流式事件 (stream_event)
  - 跨轮复用 session_id 保持对话上下文
- plugins/whisper_stt.py: 本地 faster-whisper STT (批量识别)
- plugins/kokoro_tts.py: 本地 Kokoro-82M TTS (24kHz PCM)
- config.py: pydantic-settings 配置

### LiveKit Server (deploy/docker/)
- livekit.yaml: 信令端口 7880, RTC TCP 7881, UDP 50000-50200
- docker-compose.yml: 新增 livekit-server + voice-agent 容器

### LiveKit Token 端点
- voice-service/src/api/livekit_token.py:
  POST /api/v1/voice/livekit/token
  生成 Room JWT，嵌入 auth_header 到 AgentDispatch metadata

## Flutter 客户端改造
- agent_call_page.dart: 从 ~814 行简化到 ~380 行
  - 替换: WebSocketChannel, AudioRecorder, PcmPlayer, 手动心跳/重连
  - 使用: Room.connect(), setMicrophoneEnabled(true), LiveKit 事件监听
  - 波形动画改用 participant.audioLevel
- pubspec.yaml: 添加 livekit_client: ^2.3.0
- app_config.dart: 增加 livekitUrl 字段
- api_endpoints.dart: 增加 livekitToken 端点

## 配置说明 (环境变量)
- STT_PROVIDER: local (默认, faster-whisper) / openai
- TTS_PROVIDER: local (默认, Kokoro) / openai
- WHISPER_MODEL: base (默认) / small / medium / large
- WHISPER_LANGUAGE: zh (默认)
- KOKORO_VOICE: zf_xiaoxiao (默认)
- DEVICE: cpu (默认) / cuda

## 不变的部分
- agent-service: 完全不改，voice-agent 通过现有 API 调用
- voice-service 核心: pipeline/STT/TTS/VAD 保留 (Twilio 备用)
- Kong 网关: 现有路由不变
- 数据库: 无 schema 变更

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-28 08:55:33 -08:00
hailin f5d9b1f04f feat: add app upgrade system with self-hosted APK update support 
- Add core/updater module: version checker, download manager (resumable + SHA-256),
  APK installer, app market detector, self-hosted updater with progress dialogs
- Add Android native MethodChannels for APK installation and market detection
- Add FileProvider config and REQUEST_INSTALL_PACKAGES permission
- Wire UpdateService singleton into main.dart initialization
- Add auto-check on home entry with cooldown + app resume detection
- Add manual "检查更新" button and dynamic version display in settings
- Fix chat page: bottom overflow, bash spinner persistence, collapsible results
- Merge standing orders into tasks page as second tab

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-26 22:35:01 -08:00
hailin 092a561867 feat: 完成 iAgent App 三大功能 + 修复租户上下文 
## 功能一：设置页（完整实现）
- 新增浅色主题（lightTheme），支持深色/浅色/跟随系统三种模式
- app.dart 接入 themeMode 动态切换
- 设置页完整重写：个人信息编辑、修改密码、主题切换、通知开关
- 新增 settings_remote_datasource 对接后端 admin/settings API
- settings_providers 新增 AccountProfileNotifier 管理远程个人资料

## 功能二：语音通话（音频集成）
- 添加 flutter_sound 依赖，创建 PcmPlayer 流式 PCM 播放器
- agent_call_page 替换空壳：真实麦克风采集（record + GTCRN 降噪）
- 真实 PCM 16kHz 流式播放，基于 RMS 能量驱动波形动画
- 修复 WebSocket URL 路径：/ws/voice/ → /api/v1/voice/ws/
- voice_repository_impl 支持后端返回相对路径自动拼接

## 功能三：推送通知（WebSocket MVP）
- 添加 flutter_local_notifications + socket_io_client 依赖
- 创建 AppNotification 实体、NotificationService（Socket.IO 连接 comm-service）
- 通知 providers：列表管理 + 未读计数
- 登录后自动连接通知服务，登出断开
- 底部导航 Alerts 标签添加未读角标（Badge）
- AndroidManifest 添加 POST_NOTIFICATIONS 权限
- main.dart 初始化本地通知插件

## 修复：租户上下文未初始化（500错误）
- 根因：登录后未设置 currentTenantIdProvider，导致 X-Tenant-Id 头缺失
- Flutter 端：login() 成功后从 JWT 设置 tenantId，logout 时清除
- 后端：tenant-context.middleware 增加 JWT tenantId 回退逻辑
- AuthUser 模型新增 tenantId 字段解析

新增 5 个文件，修改 16 个文件，添加 3 个依赖包

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-23 01:10:52 -08:00
hailin 51430adafc fix: 修复登录问题 — 重置用户密码 + 配置生产环境API地址 
问题排查过程:
  1. 用户 hailin@it0.com 存在于 public.users，is_active=true
  2. 直接调用 auth-service login 接口返回 401 Invalid credentials
  3. 确认是密码不匹配 — 将密码重置为 admin123 (与 admin 账号相同)
  4. 重置后登录成功，Kong Gateway 路由也正常

App配置修改:
  - development: 端口从 8000 改为 18000 (匹配 Kong 映射)
  - production: 指向服务器 http://154.84.135.121:18000
  - 默认使用 production 配置 (之前是 development)

登录凭据:
  - admin@it0.com / admin123 (管理员)
  - hailin@it0.com / admin123 (运维员，请登录后修改密码)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-22 16:56:38 -08:00
hailin a568558585 feat: replace speech_to_text with GTCRN ML noise reduction + backend STT 
Replace traditional on-device speech_to_text with a modern pipeline:
- Record audio via `record` package with hardware noise suppression
- Apply GTCRN neural denoising (sherpa-onnx, ICASSP 2024, 48K params)
- Trim silence, POST to backend /voice/transcribe (faster-whisper)

Changes:
- Add /transcribe endpoint to voice-service for audio file upload
- Add SpeechEnhancer wrapper for sherpa-onnx GTCRN model (523KB)
- Rewrite chat_page.dart voice input: record → denoise → transcribe
- Keep NoiseReducer.trimSilence for silence removal only
- Upgrade record to v6.2.0, add sherpa_onnx, path_provider

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-22 07:59:15 -08:00
hailin 00f8801d51 Initial commit: IT0 AI-powered server cluster operations platform 
Full-stack monorepo with DDD + Clean Architecture:
- Backend: 7 NestJS microservices + 5 shared libraries (TypeScript)
- Mobile: Flutter app with Riverpod (Dart)
- Web Admin: Next.js dashboard with Zustand + React Query
- Voice: Python voice service (STT/TTS/VAD)
- Infra: Docker Compose, K8s manifests, Turborepo build

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-08 22:54:37 -08:00