fix(debezium): 添加 heartbeat.action.query 防止 WAL 无限积压
问题根因:
- 5个 Debezium outbox connector 配置了 heartbeat.interval.ms=10000
但缺少 heartbeat.action.query,导致低写入频率的数据库
(auth/contribution/mining-wallet) 无法向 PostgreSQL 回传
confirmed_flush_lsn 确认位点
- PostgreSQL 无法回收旧 WAL,磁盘占用持续增长至 293GB
修复方案 (参考 Debezium 创始人 Gunnar Morling 推荐):
- 使用 pg_logical_emit_message() 代替心跳表方案
- 每10秒向 WAL 写入逻辑解码消息,绕过 publication 过滤
- 无需建表、无需改 table.include.list、无需改 publication
- 不产生额外 Kafka 消息,对消费端零影响
- 要求 PostgreSQL >= 14 (当前 16.11)
影响范围: 全部5个 outbox connector 配置文件
操作方式: PUT /connectors/{name}/config 热更新,触发 task 重启
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
parent
ab9212cefa
commit
54eb472faa
|
|
@ -35,6 +35,7 @@
|
|||
"transforms.route.replacement": "cdc.auth.outbox",
|
||||
|
||||
"heartbeat.interval.ms": "10000",
|
||||
"heartbeat.action.query": "SELECT pg_logical_emit_message(false, 'heartbeat', now()::varchar)",
|
||||
|
||||
"snapshot.mode": "initial",
|
||||
|
||||
|
|
|
|||
|
|
@ -35,6 +35,7 @@
|
|||
"transforms.route.replacement": "cdc.contribution.outbox",
|
||||
|
||||
"heartbeat.interval.ms": "10000",
|
||||
"heartbeat.action.query": "SELECT pg_logical_emit_message(false, 'heartbeat', now()::varchar)",
|
||||
|
||||
"snapshot.mode": "initial",
|
||||
|
||||
|
|
|
|||
|
|
@ -35,6 +35,7 @@
|
|||
"transforms.route.replacement": "cdc.mining.outbox",
|
||||
|
||||
"heartbeat.interval.ms": "10000",
|
||||
"heartbeat.action.query": "SELECT pg_logical_emit_message(false, 'heartbeat', now()::varchar)",
|
||||
|
||||
"snapshot.mode": "initial",
|
||||
|
||||
|
|
|
|||
|
|
@ -35,6 +35,7 @@
|
|||
"transforms.route.replacement": "cdc.mining-wallet.outbox",
|
||||
|
||||
"heartbeat.interval.ms": "10000",
|
||||
"heartbeat.action.query": "SELECT pg_logical_emit_message(false, 'heartbeat', now()::varchar)",
|
||||
|
||||
"snapshot.mode": "initial",
|
||||
|
||||
|
|
|
|||
|
|
@ -35,6 +35,7 @@
|
|||
"transforms.route.replacement": "cdc.trading.outbox",
|
||||
|
||||
"heartbeat.interval.ms": "10000",
|
||||
"heartbeat.action.query": "SELECT pg_logical_emit_message(false, 'heartbeat', now()::varchar)",
|
||||
|
||||
"snapshot.mode": "initial",
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue