 hailin
|
00a0ac3820
|
feat(agents): add admin-configurable evaluation gate for agent loop quality control
Add a configurable evaluation gate system that allows administrators to
define quality rules per consulting stage. The gate checks are executed
programmatically before the agent loop returns a response to the user.
## Architecture
- **Zero-config safe**: Empty rules table = no checks = current behavior preserved
- **Callback-based decoupling**: agent-loop.ts receives an optional callback,
stays decoupled from database layer
- **Max 1 retry**: On RETRY/SUPPLEMENT failure, recurse once without gate to
prevent infinite loops
- **Error-tolerant**: Gate exceptions are caught and logged, never block responses
## New files
- `database/migrations/20260206_add_evaluation_rules.sql` — DB migration
- `domain/entities/evaluation-rule.entity.ts` — Domain entity with 6 rule types
(FIELD_COMPLETENESS, ASSESSMENT_QUALITY, RESPONSE_LENGTH, MUST_CONTAIN,
STAGE_MIN_TURNS, CONVERSION_SIGNAL) and 4 failure actions (RETRY, SUPPLEMENT,
WARN_AND_PASS, ESCALATE)
- `domain/repositories/evaluation-rule.repository.interface.ts` — Repository contract
- `infrastructure/database/postgres/entities/evaluation-rule.orm.ts` — TypeORM ORM entity
- `infrastructure/database/postgres/repositories/evaluation-rule.repository.ts` — Repository impl
- `infrastructure/agents/coordinator/evaluation-gate.service.ts` — Core evaluation engine
with 5-minute rule cache, per-rule-type evaluators, severity-based action resolution,
and feedback message builder for model retry
- `application/dtos/evaluation-rule.dto.ts` — Create/Update/Test DTOs
- `adapters/inbound/admin-evaluation-rule.controller.ts` — Admin CRUD API with 8 endpoints:
list, get, create, update, delete, toggle, test (dry-run), clear cache
## Modified files
- `agent.types.ts` — Add optional `evaluationGate` callback to `AgentLoopParams`
- `stream.types.ts` — Add `EvaluationWarningEvent`, `'evaluating'` phase
- `agent-loop.ts` — Insert gate check at termination point (line 315)
- `coordinator-agent.service.ts` — Inject EvaluationGateService, build callback,
handle `evaluation_warning` event in StreamChunk mapping
- `agents.module.ts` — Register EvaluationRuleORM, repository, EvaluationGateService
- `conversation.module.ts` — Register AdminEvaluationRuleController
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
2026-02-06 18:56:52 -08:00 |