Commit Graph

9 Commits

Author SHA1 Message Date
hailin fff56e8baa fix(mining-admin): 修复 CDC 事件缺少 eventId 的问题
- 在 normalizeServiceEvent 中添加对多种 id 字段的支持
- 当事件缺少 id 时,使用 aggregateId + timestamp 生成备用 ID
- 在 withIdempotency 中添加 event.id 验证,避免创建无效记录
- 修复驼峰格式事件可能没有 id 字段的问题

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-20 22:09:34 -08:00
hailin 5e16adc1ec fix(mining-admin): use outbox_id for event idempotency key
The outbox_events table uses outbox_id as the primary key column name
(mapped from id in Prisma). When Debezium captures changes, the message
contains outbox_id field, not id. This caused all events to have
undefined eventId, resulting in duplicate detection treating all events
as duplicates after the first one.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 21:41:54 -08:00
hailin 70135938c4 feat(mining-admin): implement transactional idempotent consumer for 100% exactly-once semantics
- Use Prisma $transaction with Serializable isolation level
- Insert idempotency record FIRST, then execute business logic
- Unique constraint violation (P2002) indicates duplicate event
- All operations atomic - either fully commit or fully rollback
- Modified all handlers to accept transaction client parameter
- Removed old non-atomic isEventProcessed/recordProcessedEvent methods

This ensures 100% data consistency for CDC synchronization, which is
critical for financial data where any error is catastrophic.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 19:11:30 -08:00
hailin 577f626972 fix(cdc): implement idempotent consumer pattern for reliable CDC sync
- Use (sourceTopic, eventId) as composite unique key in processed_events
- Add sourceTopic to ServiceEvent for globally unique idempotency key
- Wrap all handlers with withIdempotency() for duplicate event detection
- Fix ID collision issue between different service outbox tables

This implements the industry-standard CDC exactly-once semantics pattern.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 13:31:10 -08:00
hailin bf5a16939f fix(mining-admin-service): ignore Debezium heartbeat messages
- Add isHeartbeatMessage to detect heartbeat messages (only have ts_ms field)
- Skip processing for heartbeat messages to avoid unnecessary warnings

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 07:44:27 -08:00
hailin 30e1867eb0 fix(mining-admin-service): properly handle Debezium outbox CDC events
- Add isDebeziumOutboxEvent to detect outbox table CDC messages
- Add handleDebeziumOutboxEvent to extract service event from payload.after
- Fix CDC consumer not recognizing events from contribution-service outbox

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 07:39:27 -08:00
hailin 0d06080760 fix(mining-admin): 兼容 Debezium outbox 消息格式
问题:Debezium 产生的 outbox 消息使用下划线命名(event_type,
aggregate_type),而代码期望驼峰格式(eventType, aggregateType)

解决方案:
- isServiceEvent() 同时检查两种命名格式
- 新增 normalizeServiceEvent() 转换 Debezium 格式到驼峰格式
- 解析 payload JSON 字符串

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 01:10:24 -08:00
hailin 63d73af135 refactor(cdc): 统一使用 Debezium CDC 进行数据同步
1. contribution-service:
   - 添加 identity topic 订阅,全量同步 1.0 用户数据
   - 修改 fromBeginning 为 true,首次启动全量同步

2. mining-admin-service:
   - 将 Outbox 事件改为 Debezium CDC 监听 outbox_events 表
   - 修改 fromBeginning 为 true,首次启动全量同步

3. 新增 5 个 2.0 服务的 Debezium connector 配置:
   - auth-outbox-connector.json
   - contribution-outbox-connector.json
   - mining-outbox-connector.json
   - trading-outbox-connector.json
   - mining-wallet-outbox-connector.json

4. 更新 register-connectors.sh 脚本

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 23:19:34 -08:00
hailin 15a5fb6c14 feat(mining-admin-service): 添加CDC同步和完整用户管理API
## Prisma Schema 更新
- 添加 CDC 同步表:SyncedUser, SyncedContributionAccount, SyncedMiningAccount, SyncedTradingAccount
- 添加系统数据同步表:SyncedMiningConfig, SyncedDailyMiningStat, SyncedDayKLine, SyncedCirculationPool
- 添加 CDC 进度跟踪:CdcSyncProgress, ProcessedEvent

## CDC 消费者模块
- CdcConsumerService: Kafka 消费者,支持 Debezium CDC 和服务间事件
- CdcSyncService: 同步处理器,从 auth/contribution/mining/trading 服务同步数据

## 新增 API 端点
### 用户管理 (/api/v1/users)
- GET /users - 用户列表(分页、搜索、过滤)
- GET /users/:accountSequence - 用户详情
- GET /users/:accountSequence/contributions - 算力记录
- GET /users/:accountSequence/mining-records - 挖矿记录
- GET /users/:accountSequence/orders - 交易订单

### 系统账户 (/api/v1/system-accounts)
- GET /system-accounts - 系统账户列表
- GET /system-accounts/summary - 系统账户汇总

### 仪表盘增强 (/api/v1/dashboard)
- GET /dashboard - 统计数据(新增用户/算力/挖矿/交易统计)
- GET /dashboard/realtime - 实时数据
- GET /dashboard/stats - 统计数据(别名)

## Docker Compose 更新
- 添加 Kafka 依赖和 CDC topic 配置
- 添加与 auth-service 的依赖关系

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-10 20:36:21 -08:00