Root cause of "Bridge call failed" errors: bridge /task endpoint defaults
to 25s agent reply timeout, but LLM calls through the iConsulting gateway
can take 30-60s. Fix: pass timeoutSeconds=55 explicitly in POST body.
Also add batchSend fallback in routeToAgent: if the sessionWebhook has
expired by the time the LLM replies (user sent a message, LLM took >30s,
webhook window closed), the reply is now sent via proactive batchSend
using senderStaffId instead of being silently dropped.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When the voice agent triggers DingTalk OAuth, the user leaves the app
to authorize in DingTalk/browser, causing the LiveKit participant to
disconnect. The voice-agent then calls DELETE /voice to terminate the
session — but the user intends to return after completing OAuth.
Fix: mark the session as "oauth_pending" in VoiceSessionController when
oauth-trigger fires. If terminateVoiceSession is called while the flag
is active (10-min grace), suppress the terminate and return 200 OK so
the voice-agent exits cleanly. The session stays alive; when the user
returns to the voice screen, voice/start + inject auto-resume it.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two binding paths store different DingTalk ID types:
- OAuth binding stores staffId (resolved via unionId→userId at auth time)
- Code binding stores senderId ($:LWCP_v1:$... format from bot message)
DingTalk Stream API senderId != OAuth openId (different encodings), so
primary lookup by senderId always missed OAuth-bound instances, requiring
a fallback every time. Reverse the lookup order: try senderStaffId first
(direct hit for OAuth binding), fall back to senderId (code binding).
Also add MAX_RESPONSE_BYTES cap to httpPostJson — previously uncapped
unlike the DingTalk API helpers which already had the 256KB guard.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
OpenClaw daemon checks ANTHROPIC_API_KEY env var on startup. We were passing
CLAUDE_API_KEY which openclaw ignores, so it fell back to auth-profiles.json
containing the raw Anthropic key, causing 401 from iConsulting LLM gateway.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
OpenClaw reads API key from auth-profiles.json. Was writing raw Anthropic key
sk-ant-api03-... which gateway doesn't recognize. Must use effectiveApiKey
(sk-gw-oc-... gateway key) so authentication with iConsulting LLM gateway succeeds.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
After container starts, sed-replace api.anthropic.com with iConsulting LLM gateway URL
in all models.generated.js files (ANTHROPIC_BASE_URL env alone is not enough since
baseUrl is hardcoded). Also create missing AGENTS.md template symlink so OpenClaw
does not 500 on workspace init.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two bugs fixed:
1. findByDingTalkUserId now filters status != 'removed' so a re-bound new instance
is not shadowed by an old removed one with the same DingTalk user ID.
2. When an agent is deleted (removed), its dingtalkUserId is cleared so the
DingTalk ID is freed for reuse by the next binding.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
OpenClaw runs as node user (uid 1000) but the host directory was created as root,
causing EACCES when the container tried to create /home/node/.openclaw/workspace.
Now mkdir workspace/ and chown -R 1000:1000 before starting the container.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Text sessions were not passing sessionId to SystemPromptBuilder, causing
Claude to use the `initiate_dingtalk_binding` custom tool (claude_api only).
When the engine is claude_agent_sdk, this tool does not exist → 404.
Fix: pass session.id as sessionId to systemPromptBuilder.build() in
agent.controller.ts. Claude will now use the wget oauth-trigger endpoint
for ALL session types (text and voice), which works with every engine.
Also: store userId (staffId) as the DingTalk binding ID when resolvable,
falling back to openId. Bot messages deliver senderStaffId which matches
userId, not openId — this prevents the "binding not found" routing failure.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Problem: sendGreeting() was passing openId as `userIds` to batchSend, but
the API requires the enterprise staffId (userId). This caused HTTP 400
"staffId.notExisted" for every OAuth-bound greeting.
Fix:
1. completeOAuthBinding now resolves unionId → userId via
oapi.dingtalk.com/topapi/user/getbyunionid with corp app token.
Non-fatal: if the user has no enterprise context, greeting is skipped
with a clear log explaining why (no Contact.User.Read permission or
user is not an enterprise member).
2. sendGreeting accepts userId (staffId) and openId separately; uses
the correct staffId for batchSend. If userId is undefined, emits a
WARN and skips (user gets greeting on first message instead).
3. routeToAgent now tries senderStaffId as fallback if senderId lookup
misses — handles edge cases where DingTalk delivers staffId in senderId.
4. Added detailed logging: all three IDs (openId, unionId, userId) are
logged at binding time so future issues are immediately diagnosable.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Flutter:
- my_agents_page: refresh agent list on every My Agents tab tap
(ref.invalidate in ScaffoldWithNav.onDestinationSelected)
- chat_page + my_agents_page: activate AudioSession before launching OAuth
browser so iOS keeps network connections alive in background; deactivate
when app resumes or binding polling completes
agent-service deploy:
- Write openclaw.json with correct gateway token and auth-profiles.json
with API key BEFORE starting the container, so OpenClaw and bridge
always agree on the auth token (fixes token_mismatch on new deployments)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
openclaw-bridge:
- index.ts: /task endpoint now calls chatSendAndWait() with idempotencyKey
(removes broken timeoutSeconds param; uses caller-supplied msgId for dedup)
- openclaw-client.ts: added onEvent() subscription + chatSendAndWait() that
subscribes to 'chat' WS events, waits for state='final' matching runId,
and extracts text from the message payload
dingtalk-router:
- After OAuth binding completes, sends a proactive greeting to the user via
DingTalk batchSend API (/v1.0/robot/oToMessages/batchSend) introducing the
agent by name and explaining what it can do
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- DingTalk binding UX replaced with OAuth one-tap flow:
- GET /api/v1/agent/channels/dingtalk/oauth/init returns OAuth URL
- GET /api/v1/agent/channels/dingtalk/oauth/callback (public, no JWT)
exchanges code+state for openId, saves binding, returns HTML page
- oauthStates Map with 10-min TTL; state validated before exchange
- msg.senderId (openId) aligned with OAuth openId for consistent routing
- CODE_TTL_MS extended from 5→15 min (fallback code method preserved)
- Kong: dingtalk-oauth-public service declared before agent-service
so callback path matches without JWT plugin
- Voice sessions: use stored session.systemPrompt + voice rules;
allowedTools includes Bash so Claude can call internal APIs
- Flutter _DingTalkBindSheet: OAuth-first UX with code-based fallback
phases: idle→loadingOAuth→waitingOAuth→success + polling every 2s
- docker-compose: IT0_BASE_URL env var for agent-service (redirect URI)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add GET /api/v1/agent/instances/user/:userId endpoint so Claude can
look up the caller's agent instances without knowing the ID upfront
- Update SystemPromptBuilder DingTalk section with centralized binding
flow (one-time code via iAgent DingTalk bot, no per-instance creds)
- VoiceSessionController.startVoiceSession now extracts userId from JWT
and builds a full iAgent system prompt (userId + DingTalk instructions)
so Claude knows who is speaking and how to call the binding API
- VoiceSessionManager.executeTurn now uses the session's stored system
prompt (base context + voice rules) and allows the Bash tool so Claude
can call internal APIs via wget during voice conversations
User flow: speak "帮我绑定钉钉" → Claude lists instances → generates
code via POST /api/v1/agent/channels/dingtalk/bind/:id → speaks code
letter-by-letter → user sends code in DingTalk → binding completes.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Critical fixes:
- ws.on('message') fully wrapped in try/catch — uncaught exception in
wsSend() no longer propagates to EventEmitter boundary and crashes process
- wsSend() helper: checks readyState === OPEN before send(), never throws
- Stale-WS guard: close/message events from old WS ignored after reconnect
(ws !== this.ws check); terminateCurrentWs() closes old WS before new one
- Queue tail: .catch(() => {}) appended to guarantee promise always resolves,
preventing permanently dead queue tail from silently dropping future tasks
- DISCONNECT frame handler: force-close + reconnect immediately
High fixes:
- sessionWebhookExpiredTime unit auto-detection: values < 1e11 treated as
seconds (×1000), values >= 1e11 treated as ms — prevents always-blocked reply
- httpsPost response capped at 256 KB to prevent memory spike on bad response
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- agent-instance.controller.ts: accept dingTalkClientId/dingTalkClientSecret
in POST /instances body, forward to deploy service
- system-prompt-builder.ts: add DingTalk 5-step binding guide for iAgent
so the AI can walk users through connecting their DingTalk account
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
supervisord uses %(ENV_IT0_AGENT_SERVICE_URL)s expansion which fails
if the var is not present, crashing the entire supervisor process.
Add AGENT_SERVICE_PUBLIC_URL config and inject it via docker run -e.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
agent_instances is in public schema — no tenant context needed.
Fixes 'Tenant context not initialized' when iAgent calls internal API via Bash.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- SystemPromptBuilder: add userId/userEmail to context, expose internal API curl commands for OpenClaw creation
- agent.controller.ts: extract userId from JWT, build system prompt via SystemPromptBuilder so iAgent knows current user
- agent.module.ts: register SystemPromptBuilder as provider
- agent-instance.entity.ts: make serverHost/sshUser nullable (pool mode doesn't set these upfront)
- DB: ALTER TABLE agent_instances DROP NOT NULL on server_host/ssh_user
Now iAgent can create 小龙虾 instances autonomously when user asks in natural language.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The it0hub org doesn't exist on Docker Hub. Switch to hailin168/openclaw-bridge:latest
which was built and pushed from openclaw source + IT0 bridge.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- agent-instance.controller: POST :id/heartbeat — bridge calls this every 60s;
auto-transitions status from deploying→running when gateway is confirmed connected
- system-prompt-builder: teach iAgent about OpenClaw deployment capability:
create/list/stop/remove instance API endpoints, when to trigger deployment,
and what to tell users about channel connectivity (Telegram/WhatsApp etc.)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Flutter: language='auto' omits the language field → backend receives none
- Backend: no language field → passes undefined to STT service
- STT service: language=undefined → omits language param from Whisper request
- Whisper auto-detects language per utterance when no hint is provided
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Node 18 native fetch (undici) ignores https.Agent, causing fetch failed
on the self-signed proxy at 67.223.119.33:8443. Switch to https.request
with rejectUnauthorized: false which works reliably.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
OPENAI_BASE_URL=https://67.223.119.33:8443/v1 already includes /v1,
so the URL was being built as .../v1/v1/audio/transcriptions.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
multer was only transitively available; pnpm strict mode blocks it.
Also adds @types/multer for TypeScript compilation.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add POST /api/v1/agent/transcribe endpoint (STT only, no agent trigger)
- Add transcribeAudio() to chat datasource and provider
- VoiceMicButton now fills the text input field with transcript;
user reviews and sends manually
- Add OPENAI_API_KEY/OPENAI_BASE_URL to agent-service in docker-compose
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
New endpoint: POST /api/v1/agent/sessions/:sessionId/voice-message
- Accepts multipart/form-data audio file (any format Whisper supports)
- Transcribes via OpenAI Whisper API (routed through existing proxy)
- If a task is currently running in the session → hard-interrupts it first
(same cancel+inject pattern as text inject, triggered by voice command)
- Otherwise → starts a fresh task with the transcript
- Returns { sessionId, taskId, transcript } so client can subscribe to WS stream
This enables WhatsApp-style push-to-talk and doubles as an async voice
interrupt into any active agent workflow, bypassing the need for speaker
diarization (whoever presses record owns the message).
New files:
infrastructure/stt/openai-stt.service.ts — OpenAI Whisper client,
manually builds multipart/form-data, supports self-signed proxy cert
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements a two-level abort controller design to support real-time
interruption when the user speaks while the agent is still responding:
sessionAbortController (session-scoped)
- Created once when startSession() is called
- Fired only by terminateSession() (user hangs up)
- Propagated into each turn via addEventListener
turnAbort (per-turn, stored as handle.currentTurnAbort)
- Created fresh at the start of each executeTurn() call
- Stored on the VoiceSessionHandle so injectMessage() can abort it
- When a new inject arrives while a turn is running, injectMessage()
calls turnAbort.abort() BEFORE enqueuing the new message
Interruption flow:
1. User speaks mid-response → LiveKit stops TTS playback (client-side)
2. STT utterance → POST voice/inject → injectMessage() fires
3. handle.currentTurnAbort.abort() called → sets aborted flag
4. for-await loop checks turnAbort.signal.aborted on next SDK event → break
5. catch block NOT reached (break ≠ exception) → no error event emitted
6. finally block saves partial text with "[中断]" suffix to history
7. New message dequeued → fresh executeTurn() starts immediately
Why no "Agent error" message plays to the user:
- break exits the for-await loop silently, not via exception
- The catch block's error-event emission is guarded by err?.name !== 'AbortError'
AND requires an actual exception; a plain break never enters catch
- Empty or partial responses are filtered by `if response:` in agent.py
Also update module-level JSDoc with full architecture explanation covering
the long-lived run loop design, two-level abort hierarchy, tenant context
injection pattern, and SDK session resume across turns.
Update agent.py module docstring to document voice session lifecycle and
interruption flow for future maintainers.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace the per-turn POST /tasks approach for voice calls with a
long-lived agent run loop tied to the call lifecycle:
agent-service:
- Add AsyncQueue<T> utility for blocking message relay
- Add VoiceSessionManager: spawns one background run loop per voice call,
accepts injected messages, terminates cleanly on hangup
- Add VoiceSessionController with 3 endpoints:
POST /api/v1/agent/sessions/voice/start (call start)
POST /api/v1/agent/sessions/:id/voice/inject (each speech turn)
DELETE /api/v1/agent/sessions/:id/voice (user hung up)
- Register VoiceSessionManager + VoiceSessionController in agent.module.ts
voice-agent:
- AgentServiceLLM: add start_voice_session(), terminate_voice_session(),
inject_text_message() (voice/inject-aware), _do_inject_voice()
- AgentServiceLLMStream._run(): use voice/inject path when voice session
is active; fall back to per-task POST for text-chat / non-SDK engines
- entrypoint(): call start_voice_session() after session.start();
register _on_room_disconnect that calls terminate_voice_session()
so the agent is always killed when the user hangs up
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two issues fixed:
1. agent.controller.ts — on the FIRST task of each session, write title+voiceMode
into session.metadata so the client can display a meaningful conversation title:
- Text sessions: metadata.title = first 40 chars of user prompt
- Voice sessions: metadata.title = '' + metadata.voiceMode = true
(Flutter renders these as '语音对话 M/D HH:mm')
titleSet flag prevents overwriting the title on subsequent turns of the same session.
2. session.controller.ts — listSessions() now returns a DTO instead of the raw entity.
systemPrompt is an internal engine instruction and is explicitly excluded from the
response. The client receives { id, status, engineType, metadata, createdAt, updatedAt }.
1. Remove on_enter greeting entirely (no more race condition)
2. voice-agent sends voiceMode: true when engine_type is claude_agent_sdk
3. AgentController.runTaskStream() filters thinking, tool_use, tool_result
events in voice mode — only text, completed, error reach the client
4. Detailed logging: each event logged with [FILTERED-voice] tag when skipped
Claude API mode is completely unaffected (voiceMode defaults to false).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Claude API supports up to 32MB PDFs; base64 encoding adds ~33% overhead.
50mb body limit covers the maximum single-document upload case.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
PDF files were incorrectly wrapped as type:'image' content blocks,
causing Claude API to reject them as "Invalid image data".
- conversation-context.service: check mediaType for application/pdf,
use type:'document' block (Anthropic native PDF support) instead
- claude-agent-sdk-engine: detect both 'image' and 'document' blocks
when deciding to build multimodal SDK prompt
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The direct `import * as express from 'express'` caused a
MODULE_NOT_FOUND error in the Docker production image since express
is only available as a transitive dependency via @nestjs/platform-express.
Use NestExpressApplication.useBodyParser() which is the official NestJS API.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- SDK engine now constructs AsyncIterable<SDKUserMessage> with image
content blocks when attachments are present in conversationHistory,
using the SDK's native multimodal prompt format
- CLI engine logs a warning when images are detected, since the `-p`
flag only accepts text (upstream Claude CLI limitation)
- Both SDK and API engines now fully support multimodal image input
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Two major features in this commit:
1. Streaming Markdown Rendering Optimization
- Replace deprecated flutter_markdown with gpt_markdown (active, AI-optimized)
- Real-time markdown rendering during streaming (was showing raw syntax)
- Solid block cursor (█) instead of AnimationController blink
- 80ms token throttle buffer reducing rebuilds from per-token to ~12.5/sec
- RepaintBoundary isolation for markdown widget repaints
- StreamTextWidget simplified from StatefulWidget to StatelessWidget
2. Multimodal Image Input (camera + gallery + display)
- Flutter: image_picker for gallery/camera, base64 encoding, attachment
preview strip with delete, thumbnails in sent messages
- Data layer: List<String>? → List<Map<String, dynamic>>? for structured
attachment payloads through datasource/repository/usecase
- ChatAttachment model with base64Data, mediaType, fileName
- ChatMessage entity + ChatMessageModel both support attachments field
- Backend DTO, Entity (JSONB), Controller, ConversationContextService
all extended to receive, store, and reconstruct Anthropic image
content blocks in loadContext()
- Claude API engine skips duplicate user message when history already
ends with multimodal content blocks
- NestJS body parser limit raised to 10MB for base64 image payloads
- Android CAMERA permission added to manifest
- Image.memory uses cacheWidth/cacheHeight for memory efficiency
- Max 5 images per message enforced in UI
Data flow:
ImagePicker → base64Encode → ChatAttachment → POST body →
DB (JSONB) → loadContext → Anthropic image content blocks → Claude API
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
6 rounds of systematic audit identified and fixed 14 bugs across
backend controller and Flutter client:
## Backend (agent.controller.ts)
Security & Tenant Isolation:
- Add @TenantId + ForbiddenException check to cancelTask, injectMessage,
approveCommand — all 4 write endpoints now enforce tenant isolation
- Add tenantId check on session reuse in executeTask to prevent
cross-tenant session hijacking
Architecture & Correctness:
- Extract shared runTaskStream() from inline fire-and-forget block,
used by both executeTask and injectMessage to reduce duplication
- Use session.engineType (not getActiveEngine()) in cancelTask,
injectMessage, approveCommand — fixes wrong-engine-cancel when
global engine config is switched after task creation
- Add concurrent task prevention: executeTask checks for existing
RUNNING task on same session and cancels it before starting new one
- Add runningTasks Map to track task promises, awaitTaskCleanup()
helper with 3s timeout for inject to wait for partial text save
- captureSdkSessionId() captures SDK session ID into metadata
without DB save (callers persist), preventing fire-and-forget race
Cancel/Reject Improvements:
- cancelTask: idempotent (returns early if already CANCELLED/COMPLETED),
session stays 'active' (was 'cancelled'), emits cancelled WS event
- approveCommand reject: session stays 'active' (was 'cancelled'),
now emits cancelled WS event so Flutter stream listeners clean up
- approveCommand approved: collect text events and save assistant
response to conversation history on completion (was missing)
Minor:
- task.result! non-null assertion → task.result ?? 'Unknown error'
- Add findRunningBySessionId() to TaskRepository
## Flutter
API Contract Fix:
- approveCommand: route changed from /api/v1/ops/approvals/:id/approve
to /api/v1/agent/tasks/:id/approve with {approved: true} body
- rejectCommand: route changed from /api/v1/ops/approvals/:id/reject
to /api/v1/agent/tasks/:id/approve with {approved: false} body
Resource Management:
- ChatNotifier.dispose() now disconnects WebSocket to prevent
connection leak when navigating away from chat
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
hailin