DCF模式Primary节点无法正常降级为Standby节点

【标题描述】:DCF模式中，Primary节点收到Standby节点的Append Log Ack消息时，当Append Log Ack消息中的follower_term大于当前Primary节点的Term值时，Primary节点触发降级，但没有最终没有成功降级为Standby，而是整个进程退出。经查看源码发现在opengauss中处理降级请求是直接让整个进程退出，想知道原因是什么？
【测试类型：SQL功能/存储功能/接口功能/工具功能/性能/并发/压力长稳/故障注入/安全/资料/编码规范】【测试版本：x.x.x】 问题描述

【操作系统和硬件信息】(查询命令: cat /etc/system-release, uname -a):
openEuler release 22.03 LTS
Linux 3adf3c2c795d 4.15.0-194-generic #205-Ubuntu SMP Fri Sep 16 19:49:27 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
【测试环境】(单机/1主x备x级联备):
1主2备
【被测功能】:
DCF
【测试类型】:

【数据库版本】(查询命令: gaussdb -V):
gaussdb (openGauss 5.0.0 build ) compiled at 2023-04-06 15:35:14 commit 0 last mr  debug
【预置条件】:

【操作步骤】(请填写详细的操作步骤):
插桩DCF源码rep_leader.c，修改其中的ack.follower_term
// leader process follower's ack message
static status_t rep_appendlog_ack_proc(mec_message_t *pack)
{
    uint32 stream_id = pack->head->stream_id;
    uint32 node_id = pack->head->src_inst;
    rep_apendlog_ack_t ack;

if (rep_decode_appendlog_ack(pack, &ack) != CM_SUCCESS) {
        LOG_DEBUG_ERR("[REP]rep_decode_appendlog_ack failed.");
        return CM_ERROR;
    }
    // 在此修改ack.follower_term
    ....
}

【预期输出】:
Primary节点降级，触发新一轮选举

【实际输出】:
Primary节点退出
【原因分析】:
1. 这个问题的根因
opengauss源码处理降级请求逻辑
/* * process demote request from standby */
static void ProcessDemoteRequest(void)
{
    DemoteMode mode;

/* The temperary solution is to exit Gauss when demoting happened in DCF mode */
    if (g_instance.attr.attr_storage.dcf_attr.enable_dcf) {
        /* Don't free share memory */
        ereport(LOG, (errmsg("Exit postmaster when demoting.")));
        HandleChildCrash(t_thrd.proc_cxt.MyProcPid, 1, t_thrd.proc_cxt.MyProgName);
    }
    ....
}
2. 问题推断过程
3. 还有哪些原因可能造成类似现象
4. 该问题是否有临时规避措施
5. 问题解决方案
6. 预计修复问题时间

【日志信息】(请附上日志文件、截图、coredump信息):
opengauss/dcf_log/debug/dcf.dlog:
UTC+8 2023-11-16 16:34:14.262|DCF|34616|ERROR>[ELC]LEADER receive one's term larger than itself, demote to FOLLOWER
---------------------
opengauss/data/pg_log/postgresql-2023-11-16_160014.log:
2023-11-16 16:34:14.276 [unknown] [unknown] localhost 140377063351872 0[0:0#0]  0 [BACKEND] LOG:  Enter DemoteCallbackFunc, isDcfShmemInited = 1
2023-11-16 16:34:14.276 [unknown] [unknown] localhost 140377063351872 0[0:0#0]  0 [BACKEND] LOG:  Exit DemoteCallbackFunc, isDcfShmemInited = 1
2023-11-16 16:34:14.276 [unknown] [unknown] localhost 140382452831680 0[0:0#0]  0 [BACKEND] LOG:  Exit postmaster when demoting.
2023-11-16 16:34:14.277  [postmaster][reaper][140382452831680] LOG: gaussmaster (ThreadId 140382452831680) exited with exit code 0Failed process was running: (null)
2023-11-16 16:34:14.277  [postmaster][reaper][140382452831680] LOG: terminating any other active server processes
2023-11-16 16:34:14.277  [postmaster][reaper][140382452831680] LOG: gaussmaster (ThreadId 140382452831680) exited with exit code 0
2023-11-16 16:34:14.277  [postmaster][reaper][140382452831680] LOG: the server process exits
【测试代码】:

openGauss/openGauss-server

内容风险标识

评论 (3)

openGauss/openGauss-server .gitee-modal { width: 500px !important; }

内容风险标识