After container starts, sed-replace api.anthropic.com with iConsulting LLM gateway URL
in all models.generated.js files (ANTHROPIC_BASE_URL env alone is not enough since
baseUrl is hardcoded). Also create missing AGENTS.md template symlink so OpenClaw
does not 500 on workspace init.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two bugs fixed:
1. findByDingTalkUserId now filters status != 'removed' so a re-bound new instance
is not shadowed by an old removed one with the same DingTalk user ID.
2. When an agent is deleted (removed), its dingtalkUserId is cleared so the
DingTalk ID is freed for reuse by the next binding.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
OpenClaw runs as node user (uid 1000) but the host directory was created as root,
causing EACCES when the container tried to create /home/node/.openclaw/workspace.
Now mkdir workspace/ and chown -R 1000:1000 before starting the container.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Text sessions were not passing sessionId to SystemPromptBuilder, causing
Claude to use the `initiate_dingtalk_binding` custom tool (claude_api only).
When the engine is claude_agent_sdk, this tool does not exist → 404.
Fix: pass session.id as sessionId to systemPromptBuilder.build() in
agent.controller.ts. Claude will now use the wget oauth-trigger endpoint
for ALL session types (text and voice), which works with every engine.
Also: store userId (staffId) as the DingTalk binding ID when resolvable,
falling back to openId. Bot messages deliver senderStaffId which matches
userId, not openId — this prevents the "binding not found" routing failure.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Problem: sendGreeting() was passing openId as `userIds` to batchSend, but
the API requires the enterprise staffId (userId). This caused HTTP 400
"staffId.notExisted" for every OAuth-bound greeting.
Fix:
1. completeOAuthBinding now resolves unionId → userId via
oapi.dingtalk.com/topapi/user/getbyunionid with corp app token.
Non-fatal: if the user has no enterprise context, greeting is skipped
with a clear log explaining why (no Contact.User.Read permission or
user is not an enterprise member).
2. sendGreeting accepts userId (staffId) and openId separately; uses
the correct staffId for batchSend. If userId is undefined, emits a
WARN and skips (user gets greeting on first message instead).
3. routeToAgent now tries senderStaffId as fallback if senderId lookup
misses — handles edge cases where DingTalk delivers staffId in senderId.
4. Added detailed logging: all three IDs (openId, unionId, userId) are
logged at binding time so future issues are immediately diagnosable.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Flutter:
- my_agents_page: refresh agent list on every My Agents tab tap
(ref.invalidate in ScaffoldWithNav.onDestinationSelected)
- chat_page + my_agents_page: activate AudioSession before launching OAuth
browser so iOS keeps network connections alive in background; deactivate
when app resumes or binding polling completes
agent-service deploy:
- Write openclaw.json with correct gateway token and auth-profiles.json
with API key BEFORE starting the container, so OpenClaw and bridge
always agree on the auth token (fixes token_mismatch on new deployments)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
openclaw-bridge:
- index.ts: /task endpoint now calls chatSendAndWait() with idempotencyKey
(removes broken timeoutSeconds param; uses caller-supplied msgId for dedup)
- openclaw-client.ts: added onEvent() subscription + chatSendAndWait() that
subscribes to 'chat' WS events, waits for state='final' matching runId,
and extracts text from the message payload
dingtalk-router:
- After OAuth binding completes, sends a proactive greeting to the user via
DingTalk batchSend API (/v1.0/robot/oToMessages/batchSend) introducing the
agent by name and explaining what it can do
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- DingTalk binding UX replaced with OAuth one-tap flow:
- GET /api/v1/agent/channels/dingtalk/oauth/init returns OAuth URL
- GET /api/v1/agent/channels/dingtalk/oauth/callback (public, no JWT)
exchanges code+state for openId, saves binding, returns HTML page
- oauthStates Map with 10-min TTL; state validated before exchange
- msg.senderId (openId) aligned with OAuth openId for consistent routing
- CODE_TTL_MS extended from 5→15 min (fallback code method preserved)
- Kong: dingtalk-oauth-public service declared before agent-service
so callback path matches without JWT plugin
- Voice sessions: use stored session.systemPrompt + voice rules;
allowedTools includes Bash so Claude can call internal APIs
- Flutter _DingTalkBindSheet: OAuth-first UX with code-based fallback
phases: idle→loadingOAuth→waitingOAuth→success + polling every 2s
- docker-compose: IT0_BASE_URL env var for agent-service (redirect URI)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add GET /api/v1/agent/instances/user/:userId endpoint so Claude can
look up the caller's agent instances without knowing the ID upfront
- Update SystemPromptBuilder DingTalk section with centralized binding
flow (one-time code via iAgent DingTalk bot, no per-instance creds)
- VoiceSessionController.startVoiceSession now extracts userId from JWT
and builds a full iAgent system prompt (userId + DingTalk instructions)
so Claude knows who is speaking and how to call the binding API
- VoiceSessionManager.executeTurn now uses the session's stored system
prompt (base context + voice rules) and allows the Bash tool so Claude
can call internal APIs via wget during voice conversations
User flow: speak "帮我绑定钉钉" → Claude lists instances → generates
code via POST /api/v1/agent/channels/dingtalk/bind/:id → speaks code
letter-by-letter → user sends code in DingTalk → binding completes.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Critical fixes:
- ws.on('message') fully wrapped in try/catch — uncaught exception in
wsSend() no longer propagates to EventEmitter boundary and crashes process
- wsSend() helper: checks readyState === OPEN before send(), never throws
- Stale-WS guard: close/message events from old WS ignored after reconnect
(ws !== this.ws check); terminateCurrentWs() closes old WS before new one
- Queue tail: .catch(() => {}) appended to guarantee promise always resolves,
preventing permanently dead queue tail from silently dropping future tasks
- DISCONNECT frame handler: force-close + reconnect immediately
High fixes:
- sessionWebhookExpiredTime unit auto-detection: values < 1e11 treated as
seconds (×1000), values >= 1e11 treated as ms — prevents always-blocked reply
- httpsPost response capped at 256 KB to prevent memory spike on bad response
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- agent-instance.controller.ts: accept dingTalkClientId/dingTalkClientSecret
in POST /instances body, forward to deploy service
- system-prompt-builder.ts: add DingTalk 5-step binding guide for iAgent
so the AI can walk users through connecting their DingTalk account
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
supervisord uses %(ENV_IT0_AGENT_SERVICE_URL)s expansion which fails
if the var is not present, crashing the entire supervisor process.
Add AGENT_SERVICE_PUBLIC_URL config and inject it via docker run -e.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
agent_instances is in public schema — no tenant context needed.
Fixes 'Tenant context not initialized' when iAgent calls internal API via Bash.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- SystemPromptBuilder: add userId/userEmail to context, expose internal API curl commands for OpenClaw creation
- agent.controller.ts: extract userId from JWT, build system prompt via SystemPromptBuilder so iAgent knows current user
- agent.module.ts: register SystemPromptBuilder as provider
- agent-instance.entity.ts: make serverHost/sshUser nullable (pool mode doesn't set these upfront)
- DB: ALTER TABLE agent_instances DROP NOT NULL on server_host/ssh_user
Now iAgent can create 小龙虾 instances autonomously when user asks in natural language.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The it0hub org doesn't exist on Docker Hub. Switch to hailin168/openclaw-bridge:latest
which was built and pushed from openclaw source + IT0 bridge.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- agent-instance.controller: POST :id/heartbeat — bridge calls this every 60s;
auto-transitions status from deploying→running when gateway is confirmed connected
- system-prompt-builder: teach iAgent about OpenClaw deployment capability:
create/list/stop/remove instance API endpoints, when to trigger deployment,
and what to tell users about channel connectivity (Telegram/WhatsApp etc.)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Flutter: language='auto' omits the language field → backend receives none
- Backend: no language field → passes undefined to STT service
- STT service: language=undefined → omits language param from Whisper request
- Whisper auto-detects language per utterance when no hint is provided
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Node 18 native fetch (undici) ignores https.Agent, causing fetch failed
on the self-signed proxy at 67.223.119.33:8443. Switch to https.request
with rejectUnauthorized: false which works reliably.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
OPENAI_BASE_URL=https://67.223.119.33:8443/v1 already includes /v1,
so the URL was being built as .../v1/v1/audio/transcriptions.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
multer was only transitively available; pnpm strict mode blocks it.
Also adds @types/multer for TypeScript compilation.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add POST /api/v1/agent/transcribe endpoint (STT only, no agent trigger)
- Add transcribeAudio() to chat datasource and provider
- VoiceMicButton now fills the text input field with transcript;
user reviews and sends manually
- Add OPENAI_API_KEY/OPENAI_BASE_URL to agent-service in docker-compose
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
New endpoint: POST /api/v1/agent/sessions/:sessionId/voice-message
- Accepts multipart/form-data audio file (any format Whisper supports)
- Transcribes via OpenAI Whisper API (routed through existing proxy)
- If a task is currently running in the session → hard-interrupts it first
(same cancel+inject pattern as text inject, triggered by voice command)
- Otherwise → starts a fresh task with the transcript
- Returns { sessionId, taskId, transcript } so client can subscribe to WS stream
This enables WhatsApp-style push-to-talk and doubles as an async voice
interrupt into any active agent workflow, bypassing the need for speaker
diarization (whoever presses record owns the message).
New files:
infrastructure/stt/openai-stt.service.ts — OpenAI Whisper client,
manually builds multipart/form-data, supports self-signed proxy cert
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements a two-level abort controller design to support real-time
interruption when the user speaks while the agent is still responding:
sessionAbortController (session-scoped)
- Created once when startSession() is called
- Fired only by terminateSession() (user hangs up)
- Propagated into each turn via addEventListener
turnAbort (per-turn, stored as handle.currentTurnAbort)
- Created fresh at the start of each executeTurn() call
- Stored on the VoiceSessionHandle so injectMessage() can abort it
- When a new inject arrives while a turn is running, injectMessage()
calls turnAbort.abort() BEFORE enqueuing the new message
Interruption flow:
1. User speaks mid-response → LiveKit stops TTS playback (client-side)
2. STT utterance → POST voice/inject → injectMessage() fires
3. handle.currentTurnAbort.abort() called → sets aborted flag
4. for-await loop checks turnAbort.signal.aborted on next SDK event → break
5. catch block NOT reached (break ≠ exception) → no error event emitted
6. finally block saves partial text with "[中断]" suffix to history
7. New message dequeued → fresh executeTurn() starts immediately
Why no "Agent error" message plays to the user:
- break exits the for-await loop silently, not via exception
- The catch block's error-event emission is guarded by err?.name !== 'AbortError'
AND requires an actual exception; a plain break never enters catch
- Empty or partial responses are filtered by `if response:` in agent.py
Also update module-level JSDoc with full architecture explanation covering
the long-lived run loop design, two-level abort hierarchy, tenant context
injection pattern, and SDK session resume across turns.
Update agent.py module docstring to document voice session lifecycle and
interruption flow for future maintainers.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace the per-turn POST /tasks approach for voice calls with a
long-lived agent run loop tied to the call lifecycle:
agent-service:
- Add AsyncQueue<T> utility for blocking message relay
- Add VoiceSessionManager: spawns one background run loop per voice call,
accepts injected messages, terminates cleanly on hangup
- Add VoiceSessionController with 3 endpoints:
POST /api/v1/agent/sessions/voice/start (call start)
POST /api/v1/agent/sessions/:id/voice/inject (each speech turn)
DELETE /api/v1/agent/sessions/:id/voice (user hung up)
- Register VoiceSessionManager + VoiceSessionController in agent.module.ts
voice-agent:
- AgentServiceLLM: add start_voice_session(), terminate_voice_session(),
inject_text_message() (voice/inject-aware), _do_inject_voice()
- AgentServiceLLMStream._run(): use voice/inject path when voice session
is active; fall back to per-task POST for text-chat / non-SDK engines
- entrypoint(): call start_voice_session() after session.start();
register _on_room_disconnect that calls terminate_voice_session()
so the agent is always killed when the user hangs up
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two issues fixed:
1. agent.controller.ts — on the FIRST task of each session, write title+voiceMode
into session.metadata so the client can display a meaningful conversation title:
- Text sessions: metadata.title = first 40 chars of user prompt
- Voice sessions: metadata.title = '' + metadata.voiceMode = true
(Flutter renders these as '语音对话 M/D HH:mm')
titleSet flag prevents overwriting the title on subsequent turns of the same session.
2. session.controller.ts — listSessions() now returns a DTO instead of the raw entity.
systemPrompt is an internal engine instruction and is explicitly excluded from the
response. The client receives { id, status, engineType, metadata, createdAt, updatedAt }.
1. Remove on_enter greeting entirely (no more race condition)
2. voice-agent sends voiceMode: true when engine_type is claude_agent_sdk
3. AgentController.runTaskStream() filters thinking, tool_use, tool_result
events in voice mode — only text, completed, error reach the client
4. Detailed logging: each event logged with [FILTERED-voice] tag when skipped
Claude API mode is completely unaffected (voiceMode defaults to false).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Claude API supports up to 32MB PDFs; base64 encoding adds ~33% overhead.
50mb body limit covers the maximum single-document upload case.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
PDF files were incorrectly wrapped as type:'image' content blocks,
causing Claude API to reject them as "Invalid image data".
- conversation-context.service: check mediaType for application/pdf,
use type:'document' block (Anthropic native PDF support) instead
- claude-agent-sdk-engine: detect both 'image' and 'document' blocks
when deciding to build multimodal SDK prompt
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The direct `import * as express from 'express'` caused a
MODULE_NOT_FOUND error in the Docker production image since express
is only available as a transitive dependency via @nestjs/platform-express.
Use NestExpressApplication.useBodyParser() which is the official NestJS API.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- SDK engine now constructs AsyncIterable<SDKUserMessage> with image
content blocks when attachments are present in conversationHistory,
using the SDK's native multimodal prompt format
- CLI engine logs a warning when images are detected, since the `-p`
flag only accepts text (upstream Claude CLI limitation)
- Both SDK and API engines now fully support multimodal image input
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Two major features in this commit:
1. Streaming Markdown Rendering Optimization
- Replace deprecated flutter_markdown with gpt_markdown (active, AI-optimized)
- Real-time markdown rendering during streaming (was showing raw syntax)
- Solid block cursor (█) instead of AnimationController blink
- 80ms token throttle buffer reducing rebuilds from per-token to ~12.5/sec
- RepaintBoundary isolation for markdown widget repaints
- StreamTextWidget simplified from StatefulWidget to StatelessWidget
2. Multimodal Image Input (camera + gallery + display)
- Flutter: image_picker for gallery/camera, base64 encoding, attachment
preview strip with delete, thumbnails in sent messages
- Data layer: List<String>? → List<Map<String, dynamic>>? for structured
attachment payloads through datasource/repository/usecase
- ChatAttachment model with base64Data, mediaType, fileName
- ChatMessage entity + ChatMessageModel both support attachments field
- Backend DTO, Entity (JSONB), Controller, ConversationContextService
all extended to receive, store, and reconstruct Anthropic image
content blocks in loadContext()
- Claude API engine skips duplicate user message when history already
ends with multimodal content blocks
- NestJS body parser limit raised to 10MB for base64 image payloads
- Android CAMERA permission added to manifest
- Image.memory uses cacheWidth/cacheHeight for memory efficiency
- Max 5 images per message enforced in UI
Data flow:
ImagePicker → base64Encode → ChatAttachment → POST body →
DB (JSONB) → loadContext → Anthropic image content blocks → Claude API
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
6 rounds of systematic audit identified and fixed 14 bugs across
backend controller and Flutter client:
## Backend (agent.controller.ts)
Security & Tenant Isolation:
- Add @TenantId + ForbiddenException check to cancelTask, injectMessage,
approveCommand — all 4 write endpoints now enforce tenant isolation
- Add tenantId check on session reuse in executeTask to prevent
cross-tenant session hijacking
Architecture & Correctness:
- Extract shared runTaskStream() from inline fire-and-forget block,
used by both executeTask and injectMessage to reduce duplication
- Use session.engineType (not getActiveEngine()) in cancelTask,
injectMessage, approveCommand — fixes wrong-engine-cancel when
global engine config is switched after task creation
- Add concurrent task prevention: executeTask checks for existing
RUNNING task on same session and cancels it before starting new one
- Add runningTasks Map to track task promises, awaitTaskCleanup()
helper with 3s timeout for inject to wait for partial text save
- captureSdkSessionId() captures SDK session ID into metadata
without DB save (callers persist), preventing fire-and-forget race
Cancel/Reject Improvements:
- cancelTask: idempotent (returns early if already CANCELLED/COMPLETED),
session stays 'active' (was 'cancelled'), emits cancelled WS event
- approveCommand reject: session stays 'active' (was 'cancelled'),
now emits cancelled WS event so Flutter stream listeners clean up
- approveCommand approved: collect text events and save assistant
response to conversation history on completion (was missing)
Minor:
- task.result! non-null assertion → task.result ?? 'Unknown error'
- Add findRunningBySessionId() to TaskRepository
## Flutter
API Contract Fix:
- approveCommand: route changed from /api/v1/ops/approvals/:id/approve
to /api/v1/agent/tasks/:id/approve with {approved: true} body
- rejectCommand: route changed from /api/v1/ops/approvals/:id/reject
to /api/v1/agent/tasks/:id/approve with {approved: false} body
Resource Management:
- ChatNotifier.dispose() now disconnects WebSocket to prevent
connection leak when navigating away from chat
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Backend (agent-engine.port.ts):
- Add `cancelled` event type: emitted when a task is cancelled (user-initiated
or injection), so Flutter can close the old stream cleanly
- Add `task_info` event type: emitted after inject to pass the new taskId to
the client, enabling cancel/re-inject on the replacement task
Flutter (features/chat/):
- ChatState: track current `taskId` alongside `sessionId`; clear on completion
or error
- Handle `TaskInfoEvent`: update taskId in state when server issues a new task
- Handle `CancelledEvent`: treat as stream termination (agentStatus → idle)
- MessageType.interrupted: new UI node (warning style) for mid-stream cancels
- _inject(): send text as an inject request while streaming; backend cancels
the current task and starts a new one with the injected message
- Input area: during streaming, hint changes to "追加指令...", Enter key calls
_inject() instead of _send(), and both inject-send + stop buttons are shown
- isAwaitingApproval kept separate from isStreaming so approval flow is not
blocked by inject mode
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Previously AgentSkillService wrote skills to public.agent_skills (TypeORM
entity with tenantId column filter), while ClaudeAgentSdkEngine read from
it0_t_{tenantId}.skills (per-tenant schema). The two tables were never
connected, so any skill added via the CRUD API was invisible to the agent.
This fix:
- Rewrites AgentSkillService to use DataSource + raw SQL against the
per-tenant schema it0_t_{tenantId}.skills
- Maps API fields: script→content, enabled→is_active
- Removes AgentSkillRepository and AgentSkill entity from module (no longer needed)
- CRUD API response shape is unchanged (fields mapped back to script/enabled)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Load active skills from the tenant's schema `skills` table and append
them to the system prompt before passing to the Claude Agent SDK. This
closes the gap where skills existed in the DB but were never surfaced
to the agent during task execution.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace prompt-prefix workaround with SDK's native resume mechanism.
Each tenant gets isolated HOME directory (/data/claude-tenants/{tenantId})
to prevent cross-tenant session file mixing. SDK session IDs are persisted
in session.metadata for cross-request resume support.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Implement DB-based conversation message storage (engine-agnostic) that
works across both Claude API and Agent SDK engines. Add ChatGPT/Claude-style
conversation history drawer in Flutter with date-grouped session list,
session switching, and new chat functionality.
Backend: entity, repository, context service, migration 004, session/message
API endpoints. Flutter: ConversationDrawer, sessionId flow from backend
response via SessionInfoEvent, session list/switch/delete support.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
hailin