hailin/it0 - it0 - AI Wolves Team

Commit Graph

Author	SHA1	Message	Date
hailin	c9ee93fffd	feat(instance-chat): full multimodal attachment support via OpenClaw bridge After verifying that the OpenClaw gateway's chat.send WebSocket RPC accepts an 'attachments' array (confirmed from openclaw/openclaw source and documentation), implement end-to-end image/file attachment support for instance chat: Bridge (openclaw-client.ts): - chatSendAndWait() now accepts optional `attachments[]` parameter - Passes attachments to chat.send RPC only when non-empty Bridge (index.ts): - /task-async accepts `attachments[]` from request body - Forwards to chatSendAndWait unchanged Backend (agent.controller.ts): - executeInstanceTask() accepts IT0 attachment format { base64Data, mediaType, fileName? } - Converts to OpenClaw format { name, mimeType, media: "data:..." } - Saves attachments to conversation history via contextService - Forwards to bridge via bridgeAttachments spread Flutter (agent_instance_chat_remote_datasource.dart): - createTask() now includes attachments in POST body when present Flutter (chat_page.dart): - Reverted Fix 5 (disabled button) — attachment button fully enabled in instance mode since the bridge now supports it Attachment format (OpenClaw wire): { name: string, mimeType: string, media: "data:<mime>;base64,<data>" } Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-03-09 21:18:14 -07:00
hailin	ea3cbf64a5	feat(agent): complete instance-chat robustness fixes (Fix 2-6) Fix 2 — Callback timeout wiring: - Store callbackTimer in pendingCallbackTimers Map after creation - handleOpenClawAppCallback clears the timer immediately on arrival, preventing spurious "timeout" errors when the bridge replies in time Fix 3 — Provider scope isolation: - Override agentStatusProvider and robotStateProvider in child ProviderScope so the robot avatar/FAB reflects the instance chat state, not iAgent's Fix 4 — Voice routing to OpenClaw: - AgentInstanceChatDatasource.sendVoiceMessage() now calls transcribeAudio() then routes the transcript through instance-specific createTask() endpoint, ensuring voice messages reach the user's OpenClaw agent, not iAgent Fix 5 — Attachment UI in instance mode: - Attachment button shown as disabled (onPressed: null) with explanatory tooltip ("附件功能暂不支持智能体对话") when agentName != null - Prevents misleading UX where attachments appear to work but are silently dropped by the OpenClaw bridge Fix 6 — DB schema template: - Add agent_instance_id UUID NULL to agent_sessions table in migration 002 (tenant schema template) so new tenants get the column from creation - Add covering index idx_agent_sessions_instance for efficient instance queries All TypeScript and Flutter analyze checks pass clean. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-03-09 19:49:36 -07:00
hailin	8865985019	feat(agent-instance-chat): 实现用户与自己的 OpenClaw 智能体直接对话功能 ## 功能概述用户可在「我的智能体」页面点击运行中的 OpenClaw 实例卡片，直接打开与该智能体的专属对话页面，完整复用 iAgent 的聊天 UI （流式输出、工具时间线、审批卡片、语音输入等），同时保证 iAgent 对话完全不受影响。 ## 架构设计 - 使用 Riverpod ProviderScope 子作用域覆盖 chatRemoteDatasourceProvider / chatProvider / sessionListProvider，实现 iAgent 与实例对话的 provider 完全隔离，无任何共享状态。 - OpenClaw bridge 采用已有的 /task-async 异步回调模式： Flutter → POST /api/v1/agent/instances/:id/tasks（立即返回 sessionId/taskId） → 订阅 WS /ws/agent（等待事件） → Bridge 完成后 POST /api/v1/agent/instances/openclaw-app-callback（公开端点） → 后端发 WS text+completed 事件 → Flutter 收到回复 - 每个实例的会话通过 agent_sessions.agent_instance_id 字段隔离，会话抽屉只显示当前实例的历史记录。 ## 后端变更 ### packages/shared/database/src/migrations/013-add-agent-instance-id-to-sessions.sql - 新增迁移：ALTER TABLE agent_sessions ADD COLUMN agent_instance_id UUID NULL - 为按实例过滤会话建立索引 ### packages/services/agent-service/src/domain/entities/agent-session.entity.ts - 新增可选字段 agentInstanceId: string（对应 agent_instance_id 列） - iAgent 会话该字段为 null；实例聊天会话存储对应的 instance UUID ### packages/services/agent-service/src/infrastructure/repositories/session.repository.ts - 新增 findByInstanceId(tenantId, agentInstanceId) 方法 - 用于 GET /instances/:id/sessions 按实例过滤会话列表 ### packages/services/agent-service/src/interfaces/rest/controllers/agent.controller.ts 新增三个端点（注意：已知存在以下待修复问题，见后续 fix commit）： 1. POST /api/v1/agent/instances/:instanceId/tasks - 校验 instance 归属（userId 匹配）和 running 状态 - 创建会话（engineType='openclaw'，携带 agentInstanceId） - 保存用户消息到 conversation_messages 表 - 向 OpenClaw bridge POST /task-async，sessionKey=it0:{sessionId} - 立即返回 { sessionId, taskId }，Flutter 订阅 WS 等待回调 2. GET /api/v1/agent/instances/:instanceId/sessions - 返回该实例的会话列表（含 title/status/时间戳） 3. POST /api/v1/agent/instances/openclaw-app-callback（公开端点，无 JWT） - bridge 完成后回调此端点 - 成功：发 WS text+completed 事件，保存 assistant 消息，更新 task 状态 - 失败/超时：发 WS error 事件，标记 task 为 FAILED - 注入 AgentInstanceRepository 依赖 - 新增私有方法 createInstanceSession() ### packages/gateway/config/kong.yml - 新增 openclaw-app-callback-public service（无 JWT 插件） - 路由：POST /api/v1/agent/instances/openclaw-app-callback - 必须在 agent-service 之前声明，确保路由优先匹配（同 wecom-public 模式） ## Flutter 变更 ### it0_app/lib/core/config/api_endpoints.dart - 新增 instanceTasks(instanceId) 和 instanceSessions(instanceId) 静态方法 ### it0_app/lib/features/chat/presentation/pages/chat_page.dart - 新增可选参数 agentName（默认 null = iAgent 模式） - agentName != null 时：AppBar 显示智能体名称，隐藏语音通话按钮 - 不传 agentName 时行为与原来完全一致，iAgent 功能零影响 ### it0_app/lib/features/my_agents/presentation/pages/my_agents_page.dart - _InstanceCard 新增 onTap 回调参数 - 卡片用 Material+InkWell 包裹，支持圆角水波纹点击效果 - 新增 _openInstanceChat() 顶层函数： running → 滑入式跳转到 AgentInstanceChatPage 其他状态 → SnackBar 提示（部署中/已停止/错误） - 导入 AgentInstanceChatPage ### it0_app/lib/features/agent_instance_chat/（新建功能模块） data/datasources/agent_instance_chat_remote_datasource.dart: - AgentInstanceChatDatasource implements ChatRemoteDatasource - 通过组合模式包装 ChatRemoteDatasource 委托所有通用操作 - 覆盖 createTask → POST /api/v1/agent/instances/:id/tasks - 覆盖 listSessions → GET /api/v1/agent/instances/:id/sessions（仅当前实例会话） presentation/pages/agent_instance_chat_page.dart: - AgentInstanceChatPage(instance: AgentInstance) - ProviderScope 子作用域覆盖三个 provider 实现完全隔离： chatRemoteDatasourceProvider → AgentInstanceChatDatasource chatProvider → 独立 ChatNotifier 实例（与 iAgent 零共享） sessionListProvider → 仅当前实例的会话列表 - child: ChatPage(agentName: instance.name) 完整复用 UI ## 已知待修复问题（下一个 commit） 1. [安全] 鉴权检查逻辑：if (userId && ...) 应为 if (!userId \|\| ...) 2. [可靠性] fetch 未处理 HTTP 4xx/5xx 错误，任务可能永久挂起 3. [可靠性] bridge 回调无超时机制，bridge 崩溃后任务永久 RUNNING 4. [UX] robotStateProvider 未在子 ProviderScope 覆盖，头像动画反映 iAgent 状态 5. [UX] 实例聊天附件 UI 未禁用，上传附件被静默丢弃 6. [UX] 语音消息在实例模式下错误路由到 iAgent 引擎（非 OpenClaw） 7. [DB] 002 模板未加 agent_instance_id 列，新租户缺失此字段 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-03-09 19:30:38 -07:00

Author

SHA1

Message

Date

hailin

c9ee93fffd

feat(instance-chat): full multimodal attachment support via OpenClaw bridge

After verifying that the OpenClaw gateway's chat.send WebSocket RPC
accepts an 'attachments' array (confirmed from openclaw/openclaw source
and documentation), implement end-to-end image/file attachment support
for instance chat:

Bridge (openclaw-client.ts):
- chatSendAndWait() now accepts optional `attachments[]` parameter
- Passes attachments to chat.send RPC only when non-empty

Bridge (index.ts):
- /task-async accepts `attachments[]` from request body
- Forwards to chatSendAndWait unchanged

Backend (agent.controller.ts):
- executeInstanceTask() accepts IT0 attachment format
  { base64Data, mediaType, fileName? }
- Converts to OpenClaw format { name, mimeType, media: "data:..." }
- Saves attachments to conversation history via contextService
- Forwards to bridge via bridgeAttachments spread

Flutter (agent_instance_chat_remote_datasource.dart):
- createTask() now includes attachments in POST body when present

Flutter (chat_page.dart):
- Reverted Fix 5 (disabled button) — attachment button fully enabled
  in instance mode since the bridge now supports it

Attachment format (OpenClaw wire):
  { name: string, mimeType: string, media: "data:<mime>;base64,<data>" }

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-03-09 21:18:14 -07:00

hailin

ea3cbf64a5

feat(agent): complete instance-chat robustness fixes (Fix 2-6)

Fix 2 — Callback timeout wiring:
- Store callbackTimer in pendingCallbackTimers Map after creation
- handleOpenClawAppCallback clears the timer immediately on arrival,
  preventing spurious "timeout" errors when the bridge replies in time

Fix 3 — Provider scope isolation:
- Override agentStatusProvider and robotStateProvider in child ProviderScope
  so the robot avatar/FAB reflects the instance chat state, not iAgent's

Fix 4 — Voice routing to OpenClaw:
- AgentInstanceChatDatasource.sendVoiceMessage() now calls transcribeAudio()
  then routes the transcript through instance-specific createTask() endpoint,
  ensuring voice messages reach the user's OpenClaw agent, not iAgent

Fix 5 — Attachment UI in instance mode:
- Attachment button shown as disabled (onPressed: null) with explanatory
  tooltip ("附件功能暂不支持智能体对话") when agentName != null
- Prevents misleading UX where attachments appear to work but are silently
  dropped by the OpenClaw bridge

Fix 6 — DB schema template:
- Add agent_instance_id UUID NULL to agent_sessions table in migration 002
  (tenant schema template) so new tenants get the column from creation
- Add covering index idx_agent_sessions_instance for efficient instance queries

All TypeScript and Flutter analyze checks pass clean.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-03-09 19:49:36 -07:00

hailin

8865985019

feat(agent-instance-chat): 实现用户与自己的 OpenClaw 智能体直接对话功能

## 功能概述
用户可在「我的智能体」页面点击运行中的 OpenClaw 实例卡片，
直接打开与该智能体的专属对话页面，完整复用 iAgent 的聊天 UI
（流式输出、工具时间线、审批卡片、语音输入等），同时保证
iAgent 对话完全不受影响。

## 架构设计
- 使用 Riverpod ProviderScope 子作用域覆盖 chatRemoteDatasourceProvider
  / chatProvider / sessionListProvider，实现 iAgent 与实例对话的
  provider 完全隔离，无任何共享状态。
- OpenClaw bridge 采用已有的 /task-async 异步回调模式：
    Flutter → POST /api/v1/agent/instances/:id/tasks（立即返回 sessionId/taskId）
    → 订阅 WS /ws/agent（等待事件）
    → Bridge 完成后 POST /api/v1/agent/instances/openclaw-app-callback（公开端点）
    → 后端发 WS text+completed 事件 → Flutter 收到回复
- 每个实例的会话通过 agent_sessions.agent_instance_id 字段隔离，
  会话抽屉只显示当前实例的历史记录。

## 后端变更
### packages/shared/database/src/migrations/013-add-agent-instance-id-to-sessions.sql
- 新增迁移：ALTER TABLE agent_sessions ADD COLUMN agent_instance_id UUID NULL
- 为按实例过滤会话建立索引

### packages/services/agent-service/src/domain/entities/agent-session.entity.ts
- 新增可选字段 agentInstanceId: string（对应 agent_instance_id 列）
- iAgent 会话该字段为 null；实例聊天会话存储对应的 instance UUID

### packages/services/agent-service/src/infrastructure/repositories/session.repository.ts
- 新增 findByInstanceId(tenantId, agentInstanceId) 方法
- 用于 GET /instances/:id/sessions 按实例过滤会话列表

### packages/services/agent-service/src/interfaces/rest/controllers/agent.controller.ts
新增三个端点（注意：已知存在以下待修复问题，见后续 fix commit）：
1. POST /api/v1/agent/instances/:instanceId/tasks
   - 校验 instance 归属（userId 匹配）和 running 状态
   - 创建会话（engineType='openclaw'，携带 agentInstanceId）
   - 保存用户消息到 conversation_messages 表
   - 向 OpenClaw bridge POST /task-async，sessionKey=it0:{sessionId}
   - 立即返回 { sessionId, taskId }，Flutter 订阅 WS 等待回调
2. GET /api/v1/agent/instances/:instanceId/sessions
   - 返回该实例的会话列表（含 title/status/时间戳）
3. POST /api/v1/agent/instances/openclaw-app-callback（公开端点，无 JWT）
   - bridge 完成后回调此端点
   - 成功：发 WS text+completed 事件，保存 assistant 消息，更新 task 状态
   - 失败/超时：发 WS error 事件，标记 task 为 FAILED
- 注入 AgentInstanceRepository 依赖
- 新增私有方法 createInstanceSession()

### packages/gateway/config/kong.yml
- 新增 openclaw-app-callback-public service（无 JWT 插件）
- 路由：POST /api/v1/agent/instances/openclaw-app-callback
- 必须在 agent-service 之前声明，确保路由优先匹配（同 wecom-public 模式）

## Flutter 变更
### it0_app/lib/core/config/api_endpoints.dart
- 新增 instanceTasks(instanceId) 和 instanceSessions(instanceId) 静态方法

### it0_app/lib/features/chat/presentation/pages/chat_page.dart
- 新增可选参数 agentName（默认 null = iAgent 模式）
- agentName != null 时：AppBar 显示智能体名称，隐藏语音通话按钮
- 不传 agentName 时行为与原来完全一致，iAgent 功能零影响

### it0_app/lib/features/my_agents/presentation/pages/my_agents_page.dart
- _InstanceCard 新增 onTap 回调参数
- 卡片用 Material+InkWell 包裹，支持圆角水波纹点击效果
- 新增 _openInstanceChat() 顶层函数：
    running → 滑入式跳转到 AgentInstanceChatPage
    其他状态 → SnackBar 提示（部署中/已停止/错误）
- 导入 AgentInstanceChatPage

### it0_app/lib/features/agent_instance_chat/（新建功能模块）
data/datasources/agent_instance_chat_remote_datasource.dart:
- AgentInstanceChatDatasource implements ChatRemoteDatasource
- 通过组合模式包装 ChatRemoteDatasource 委托所有通用操作
- 覆盖 createTask → POST /api/v1/agent/instances/:id/tasks
- 覆盖 listSessions → GET /api/v1/agent/instances/:id/sessions（仅当前实例会话）

presentation/pages/agent_instance_chat_page.dart:
- AgentInstanceChatPage(instance: AgentInstance)
- ProviderScope 子作用域覆盖三个 provider 实现完全隔离：
    chatRemoteDatasourceProvider → AgentInstanceChatDatasource
    chatProvider → 独立 ChatNotifier 实例（与 iAgent 零共享）
    sessionListProvider → 仅当前实例的会话列表
- child: ChatPage(agentName: instance.name) 完整复用 UI

## 已知待修复问题（下一个 commit）
1. [安全] 鉴权检查逻辑：if (userId && ...) 应为 if (!userId || ...)
2. [可靠性] fetch 未处理 HTTP 4xx/5xx 错误，任务可能永久挂起
3. [可靠性] bridge 回调无超时机制，bridge 崩溃后任务永久 RUNNING
4. [UX] robotStateProvider 未在子 ProviderScope 覆盖，头像动画反映 iAgent 状态
5. [UX] 实例聊天附件 UI 未禁用，上传附件被静默丢弃
6. [UX] 语音消息在实例模式下错误路由到 iAgent 引擎（非 OpenClaw）
7. [DB] 002 模板未加 agent_instance_id 列，新租户缺失此字段

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-03-09 19:30:38 -07:00

3 Commits