fix(debezium): 添加 heartbeat.action.query 防止 WAL 无限积压

问题根因:
- 5个 Debezium outbox connector 配置了 heartbeat.interval.ms=10000
  但缺少 heartbeat.action.query,导致低写入频率的数据库
  (auth/contribution/mining-wallet) 无法向 PostgreSQL 回传
  confirmed_flush_lsn 确认位点
- PostgreSQL 无法回收旧 WAL,磁盘占用持续增长至 293GB

修复方案 (参考 Debezium 创始人 Gunnar Morling 推荐):
- 使用 pg_logical_emit_message() 代替心跳表方案
- 每10秒向 WAL 写入逻辑解码消息,绕过 publication 过滤
- 无需建表、无需改 table.include.list、无需改 publication
- 不产生额外 Kafka 消息,对消费端零影响
- 要求 PostgreSQL >= 14 (当前 16.11)

影响范围: 全部5个 outbox connector 配置文件
操作方式: PUT /connectors/{name}/config 热更新,触发 task 重启

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
hailin 2026-02-15 02:44:38 -08:00
parent ab9212cefa
commit 54eb472faa
5 changed files with 5 additions and 0 deletions

View File

@ -35,6 +35,7 @@
"transforms.route.replacement": "cdc.auth.outbox", "transforms.route.replacement": "cdc.auth.outbox",
"heartbeat.interval.ms": "10000", "heartbeat.interval.ms": "10000",
"heartbeat.action.query": "SELECT pg_logical_emit_message(false, 'heartbeat', now()::varchar)",
"snapshot.mode": "initial", "snapshot.mode": "initial",

View File

@ -35,6 +35,7 @@
"transforms.route.replacement": "cdc.contribution.outbox", "transforms.route.replacement": "cdc.contribution.outbox",
"heartbeat.interval.ms": "10000", "heartbeat.interval.ms": "10000",
"heartbeat.action.query": "SELECT pg_logical_emit_message(false, 'heartbeat', now()::varchar)",
"snapshot.mode": "initial", "snapshot.mode": "initial",

View File

@ -35,6 +35,7 @@
"transforms.route.replacement": "cdc.mining.outbox", "transforms.route.replacement": "cdc.mining.outbox",
"heartbeat.interval.ms": "10000", "heartbeat.interval.ms": "10000",
"heartbeat.action.query": "SELECT pg_logical_emit_message(false, 'heartbeat', now()::varchar)",
"snapshot.mode": "initial", "snapshot.mode": "initial",

View File

@ -35,6 +35,7 @@
"transforms.route.replacement": "cdc.mining-wallet.outbox", "transforms.route.replacement": "cdc.mining-wallet.outbox",
"heartbeat.interval.ms": "10000", "heartbeat.interval.ms": "10000",
"heartbeat.action.query": "SELECT pg_logical_emit_message(false, 'heartbeat', now()::varchar)",
"snapshot.mode": "initial", "snapshot.mode": "initial",

View File

@ -35,6 +35,7 @@
"transforms.route.replacement": "cdc.trading.outbox", "transforms.route.replacement": "cdc.trading.outbox",
"heartbeat.interval.ms": "10000", "heartbeat.interval.ms": "10000",
"heartbeat.action.query": "SELECT pg_logical_emit_message(false, 'heartbeat', now()::varchar)",
"snapshot.mode": "initial", "snapshot.mode": "initial",