【测试类型：功能测试】【测试版本：6.0.0】【资源池化】debug版本，开启按需回放+实时构建，switchover过程产生SSWaitStartupExit相关core

【标题描述】:debug版本，开启按需回放+实时构建，switchover过程产生SSWaitStartupExit相关core

【操作系统和硬件信息】(查询命令: cat /etc/system-release, uname -a): ARM+openEuler

【测试环境】(单机/1主x备x级联备): 一主两备

【被测功能】: switchover

【测试类型】: 功能测试

【数据库版本】(查询命令: gaussdb -V):
![输入图片说明](https://foruda.gitee.com/images/1715690120159297267/886dac8e_7782923.png "屏幕截图")

【预置条件】: 一主两备资源池化集群

【操作步骤】(请填写详细的操作步骤):
1. 主节点执行读写操作；
2. failover后拉起被踢出节点，执行switchover操作恢复集群初始状态 or 执行switchover切换节点

【预期输出】: switchover操作成功，节点状态正常

【实际输出】: switchover操作超时，节点coredump

【原因分析】:
1. 这个问题的根因
2. 问题推断过程
3. 还有哪些原因可能造成类似现象
4. 该问题是否有临时规避措施
5. 问题解决方案
6. 预计修复问题时间

【日志信息】(请附上日志文件、截图、coredump信息):
![输入图片说明](https://foruda.gitee.com/images/1715690089697544280/73d51b39_7782923.png "屏幕截图")

![输入图片说明](https://foruda.gitee.com/images/1715690303563321580/ff9fe694_7782923.png "屏幕截图")

相关日志保存至如下路径：(与I9PAZC同环境问题，日志保存一份)
![输入图片说明](https://foruda.gitee.com/images/1715690398424057514/3237540f_7782923.png "屏幕截图")

【测试代码】: 不涉及

抓取相关堆栈，故障原因是SSProcessForceExit主动退出
Thread 200 (LWP 3560880):
#0  SSProcessForceExit () at ss_reform_common.cpp:583
#1  0x0000aaaaf23a5e1c in SSWaitStartupExit (send_signal=<optimized out>) at ss_reform_common.cpp:634
#2  0x0000aaaaf239b690 in CBReformStartNotify (db_handle=<optimized out>, rs_cxt=0xff943b9ca700) at ss_dms_callback.cpp:1932
#3  0x0000ff9f308966b4 in dms_reform_start () at /usr1/build/workspace/openGauss/DMS/src/rc/dms_reform_proc.c:126
#4  0x0000ff9f30899ce8 in dms_reform_proc_inner () at /usr1/build/workspace/openGauss/DMS/src/rc/dms_reform_proc.c:1855
#5  dms_reform_inner () at /usr1/build/workspace/openGauss/DMS/src/rc/dms_reform_proc.c:1879
#6  dms_reform_proc_thread (thread=0xff9f30c25808 <g_dms+21864>) at /usr1/build/workspace/openGauss/DMS/src/rc/dms_reform_proc.c:1919
#7  0x0000ff9f308f2070 in cm_thread_run (arg=<optimized out>) at /usr1/build/workspace/openGauss/CBB/src/cm_concurrency/cm_thread.c:196
#8  0x0000ffff9b4988cc in ?? () from /lib64/libpthread.so.0
#9  0x0000ffff9b3d954c in ?? () from /lib64/libc.so.6

而SSProcessForceExit主动退出是因为OndemandXLogParseBufferDestory尚未执行完毕，导致StopRecoveryWorkers未退出。
Thread 223 (LWP 3561825):
#0  0x0000ffff9b38b078 in ?? () from /lib64/libc.so.6
#1  0x0000aaaaf1f8acbc in OndemandXLogParseBufferDestory (parsemanager=parsemanager@entry=0xff94190ad158) at redo_utils.cpp:48
#2  0x0000aaaaf209aeb0 in XLogParseBufferDestory (parsemanager=parsemanager@entry=0xff94190ad158) at redo_xlogutils.cpp:1163
#3  0x0000aaaaf1f7dc44 in ondemand_extreme_rto::DestroyPageRedoWorker (worker=0xff94190ad050) at page_redo.cpp:305
#4  0x0000aaaaf1f7ac80 in ondemand_extreme_rto::SSDestroyRecoveryWorkers () at dispatcher.cpp:737
#5  ondemand_extreme_rto::StopRecoveryWorkers (code=<optimized out>, arg=<optimized out>) at dispatcher.cpp:725
#6  0x0000aaaaf2245fc0 in shmem_exit (code=code@entry=0) at ipc.cpp:508
#7  0x0000aaaaf22460a4 in proc_exit_prepare (code=code@entry=0) at ipc.cpp:380
#8  0x0000aaaaf2246874 in proc_exit (code=code@entry=0) at ipc.cpp:284
#9  0x0000aaaaf1f78404 in ondemand_extreme_rto::HandleStartupInterruptsForExtremeRto () at dispatcher.cpp:427
#10 0x0000aaaaf1f87378 in ondemand_extreme_rto::SPSCBlockingQueueTake (queue=queue@entry=0xff9414c71a90) at spsc_blocking_queue.cpp:143
#11 0x0000aaaaf1f886f0 in ondemand_extreme_rto::ReadNextRecordFromQueue (emode=15) at xlog_read.cpp:52
#12 ondemand_extreme_rto::ReadNextXLogRecord (xlogreaderptr=0xff941ccb94e8, emode=15) at xlog_read.cpp:131
#13 0x0000aaaaf1ffa0b8 in StartupXLOG () at xlog.cpp:10544
#14 0x0000aaaaf1b8c738 in StartupProcessMain () at startup.cpp:374
#15 0x0000aaaaf1b7e22c in GaussDbAuxiliaryThreadMain<(knl_thread_role)24> (arg=0xff9fdb657968) at postmaster.cpp:13746
#16 GaussDbThreadMain<(knl_thread_role)24> (arg=0xff9fdb657968) at postmaster.cpp:14227
#17 0x0000aaaaf1b560a0 in InternalThreadFunc (args=<optimized out>) at postmaster.cpp:14819
#18 0x0000ffff9b4988cc in ?? () from /lib64/libpthread.so.0
#19 0x0000ffff9b3d954c in ?? () from /lib64/libc.so.6

通过走读代码，该函数执行耗时较长是因为在stopRecoveryWorkers过程中对HashMap的内存进行了写0操作（本次故障场景是50GB），执行时间较久。修改方案是不在stopRecoveryWorkers过程进行HashMap写0，调整至startRecoveryWorkers进行。

GVP openGauss/openGauss-server

内容风险标识

评论 (6)

GVPopenGauss/openGauss-server

内容风险标识