【测试类型：故障注入】【测试版本：7.0.0】【可靠性】主机dn注入权限故障后，备机failover超时

【标题描述】: 备机failover超时
【测试类型：可靠性】【测试版本：7.0.0】 主机注入权限故障后，备机failover超时
【操作系统和硬件信息】(查询命令: cat /etc/system-release, uname -a):
CentOS Linux release 7.6.1908 (Core)

【测试环境】(单机/1主x备x级联备):
一主两备

【被测功能】:
备机failover

【测试类型】:
可靠性

【数据库版本】(查询命令: gaussdb -V):
gaussdb (openGauss 7.0.0-RC1 build f8ba23e6) compiled at 2024-12-19 00:08:34 commit 0 last mr

【预置条件】:

【操作步骤】(请填写详细的操作步骤):
1. 修改主机数据库实例目录权限
chmod 000 /data/relia1468/cluster/dn1
2. 备机1进行failover
gs_ctl failover -D /data/relia1468/cluster/dn1

【预期输出】:
1. 修改主机数据库实例目录权限，成功
2. 备机1进行failover，成功

【实际输出】:
1. 修改主机数据库实例目录权限，成功
2. 备机1进行failover，超时失败

【原因分析】:
1. 这个问题的根因
2. 问题推断过程
3. 还有哪些原因可能造成类似现象
4. 该问题是否有临时规避措施
5. 问题解决方案
6. 预计修复问题时间

【日志信息】(请附上日志文件、截图、coredump信息):

![输入图片说明](https://foruda.gitee.com/images/1735088836177532249/6b57adb9_8568116.png "屏幕截图")
详细PG日志；

```
2024-12-20 06:34:24.802 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:34:24.824 67649f47.1102 [unknown] 140298319558400 dn_6002 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  received failover notification
2024-12-20 06:34:24.824 67649f47.1102 [unknown] 140298319558400 dn_6002 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  startup shut down walreceiver.
2024-12-20 06:34:25.845 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:34:26.888 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:34:27.947 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:34:29.007 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:34:30.043 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:34:31.098 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:34:32.151 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:34:33.190 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:34:34.221 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:34:35.257 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:34:36.301 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:34:37.336 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:34:38.388 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:34:39.427 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:34:40.465 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:34:41.521 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:34:42.556 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:34:43.588 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:34:44.624 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:34:45.670 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:34:46.695 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:34:47.729 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:34:48.780 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:34:49.808 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:34:50.826 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:34:51.841 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:34:52.878 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:34:53.927 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:34:54.979 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:34:55.993 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:34:57.048 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:34:58.073 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:34:59.120 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:35:00.164 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:35:01.217 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:35:02.283 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:35:03.336 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:35:04.369 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:35:05.429 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:35:06.484 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:35:07.499 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:35:08.537 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:35:09.576 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:35:10.608 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:35:11.662 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:35:12.710 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:35:13.758 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:35:14.797 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:35:15.842 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:35:16.891 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:35:17.946 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:35:18.984 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:35:20.050 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:35:21.083 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:35:22.129 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:35:23.149 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:35:24.187 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:35:25.227 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:35:26.287 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:35:27.332 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Instance to do failover.
2024-12-20 06:35:53.598 67649f4a.1114 [unknown] 140298055317248 dn_6002 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  connection closed by peer, disconnect.
2024-12-20 06:35:53.598 67649f4a.1114 [unknown] 140298055317248 dn_6002 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Disconnect the heartbeat client.
2024-12-20 06:35:54.607 67649f4a.1114 [unknown] 140298055317248 dn_6002 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Connect failed, total failed times is 1.
2024-12-20 06:35:54.607 67649f4a.1114 [unknown] 140298055317248 dn_6002 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Failed to connect to heartbeat primary.
2024-12-20 06:35:55.633 67649f4a.1110 [unknown] 140297837733632 dn_6002 0 dn_6001_6002_6003 42809  0 [BACKEND] FATAL:  terminating walreceiver due to heartbeat timeout,now time(2024-12-20 06:35:55.606178+08) last heartbeat time(2024-12-20 06:35:52.598185+08) calculateTime(2024-12-20 06:35:55.598185+08)
2024-12-20 06:35:55.633 67649f4a.1110 [unknown] 140297837733632 dn_6002 0 dn_6001_6002_6003 42809  0 [BACKEND] BACKTRACELOG:  tid[31770]'s backtrace:
	/data/relia1468/cluster/app/bin/gaussdb(+0xe3928b) [0x55846e7f728b]
	/data/relia1468/cluster/app/bin/gaussdb(_Z9errfinishiz+0x59b) [0x55846e7eea3b]
	/data/relia1468/cluster/app/bin/gaussdb(+0x1aa00a3) [0x55846f45e0a3]
	/data/relia1468/cluster/app/bin/gaussdb(_Z18WalRcvrProcessDataPlPb+0x49e) [0x55846f467fae]
	/data/relia1468/cluster/app/bin/gaussdb(_Z15WalReceiverMainv+0x76b) [0x55846f46969b]
	/data/relia1468/cluster/app/bin/gaussdb(_Z17GaussDbThreadMainIL15knl_thread_role41EEiP14knl_thread_arg+0x3ca) [0x55846edafa6a]
	/data/relia1468/cluster/app/bin/gaussdb(+0x13c5585) [0x55846ed83585]
	/lib64/libpthread.so.0(+0x7e65) [0x7f9af6db1e65]
	/lib64/libc.so.6(clone+0x6d) [0x7f9af6ada88d]
	Use addr2line to get pretty function name and line
	
2024-12-20 06:35:55.647 67649f4a.1110 [unknown] 140297837733632 dn_6002 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  waiting walrcvwriter: 140297854514944 terminate
2024-12-20 06:35:55.647 67649f4a.1114 [unknown] 140298055317248 dn_6002 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Failed to connect to heartbeat primary.
2024-12-20 06:35:55.648 67649f4a.1113 [unknown] 140297854514944 dn_6002 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  walrcvwriter thread shut down
2024-12-20 06:35:55.657 67649f4a.1110 [unknown] 140297837733632 dn_6002 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  set local_redo_finish_status to false in WalRcvDie
2024-12-20 06:35:55.682 67649f4a.1110 [unknown] 140297837733632 dn_6002 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  walreceiver thread shut down
2024-12-20 06:35:55.735 67649f47.1102 [unknown] 140298319558400 dn_6002 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  startup shut down datareceiver.
2024-12-20 06:35:55.748 67649f47.1102 [unknown] 140298319558400 dn_6002 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  StartPageRedoWorker successfully create page-redo-worker id: 0, threadId:140297837733632.
2024-12-20 06:35:55.748 67649f47.1102 [unknown] 140298319558400 dn_6002 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  StartPageRedoWorker successfully create page-redo-worker id: 1, threadId:140297854514944.
2024-12-20 06:35:55.748 67649f47.1102 [unknown] 140298319558400 dn_6002 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  StartPageRedoWorker successfully create page-redo-worker id: 2, threadId:140297977722624.
2024-12-20 06:35:55.748 67649f47.1102 [unknown] 140298319558400 dn_6002 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  StartPageRedoWorker successfully create page-redo-worker id: 3, threadId:140297445046016.
2024-12-20 06:35:55.748 67649f47.1102 [unknown] 140298319558400 dn_6002 0 dn_6001_6002_6003 DB010  0 [REDO] LOG:  [PR]: max=4, thrd=4, workers=0
2024-12-20 06:35:55.802 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  gaussdb: fsync file "/data/relia1468/cluster/dn1/gaussdb.state.temp" success
2024-12-20 06:35:55.846 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  set gaussdb state file: db state(PROMOTING_STATE), server mode(Standby)
2024-12-20 06:35:55.982 67649fcb.1113 [unknown] 140297854514944 dn_6002 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Page-redo-worker thread 1 started.
2024-12-20 06:35:55.991 67649fcb.1115 [unknown] 140297837733632 dn_6002 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Page-redo-worker thread 0 started.
2024-12-20 06:35:55.994 67649fcb.1110 [unknown] 140297445046016 dn_6002 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Page-redo-worker thread 3 started.
2024-12-20 06:35:55.994 67649fcb.1116 [unknown] 140297977722624 dn_6002 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Page-redo-worker thread 2 started.
2024-12-20 06:35:55.996 67649f47.1102 [unknown] 140298319558400 dn_6002 0 dn_6001_6002_6003 DB010  0 [REDO] LOG:  WaitWorkerReady total worker count:4, readyWorkerCnt:4
2024-12-20 06:35:55.996 67649f47.1102 [unknown] 140298319558400 dn_6002 0 dn_6001_6002_6003 DB010  0 [REDO] LOG:  WaitWorkerReady total worker count:4, readyWorkerCnt:4
2024-12-20 06:35:55.996 67649f47.1102 [unknown] 140298319558400 dn_6002 0 dn_6001_6002_6003 DB010  0 [REDO] LOG:  RearrangeWorkers, rearrange ready workers originWorkerId :0, threadId:140297837733632, newWorkerId:0
2024-12-20 06:35:55.996 67649f47.1102 [unknown] 140298319558400 dn_6002 0 dn_6001_6002_6003 DB010  0 [REDO] LOG:  RearrangeWorkers, rearrange ready workers originWorkerId :1, threadId:140297854514944, newWorkerId:1
2024-12-20 06:35:55.996 67649f47.1102 [unknown] 140298319558400 dn_6002 0 dn_6001_6002_6003 DB010  0 [REDO] LOG:  RearrangeWorkers, rearrange ready workers originWorkerId :2, threadId:140297977722624, newWorkerId:2
2024-12-20 06:35:55.996 67649f47.1102 [unknown] 140298319558400 dn_6002 0 dn_6001_6002_6003 DB010  0 [REDO] LOG:  RearrangeWorkers, rearrange ready workers originWorkerId :3, threadId:140297445046016, newWorkerId:3
2024-12-20 06:35:55.996 67649f47.1102 [unknown] 140298319558400 dn_6002 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  redo starts at D/8FF29688
2024-12-20 06:35:55.996 67649f47.1102 [unknown] 140298319558400 dn_6002 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  set knl_g_set_redo_finish_status to false when starting redo
2024-12-20 06:35:57.015 67649f47.1102 [unknown] 140298319558400 dn_6002 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  read record failed when promoting, current lsn (D/8FF29868), received lsn(D/8FF29848),sources[1], failedSources[3], readSource[2], readFile[-1], readId[0],readSeg[3471], readOff[15892480], readLen[8192]
2024-12-20 06:35:57.015 67649f47.1102 [unknown] 140298319558400 dn_6002 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  read xlog page failed at D/8FF29848
2024-12-20 06:35:57.015 67649f47.1102 [unknown] 140298319558400 dn_6002 0 dn_6001_6002_6003 DB010  0 [REDO] LOG:  [REDO_LOG_TRACE]SendRecoveryEndMarkToWorkersAndWaitForFinish, ready to stop redo workers, code: 0
2024-12-20 06:35:57.017 67649fcb.1113 [unknown] 140297854514944 dn_6002 0 dn_6001_6002_6003 DB010  0 [REDO] LOG:  worker[1]: multipage cnt = 2, wait reach elapsed 0 us, wait replay elapsed 0 us, total elapsed = 1034900
2024-12-20 06:35:57.017 67649fcb.1113 [unknown] 140297854514944 dn_6002 0 dn_6001_6002_6003 DB010  0 [REDO] LOG:  btree_xlog_cleanup is over, it takes time:0 microseconds
2024-12-20 06:35:57.017 67649fcb.1113 [unknown] 140297854514944 dn_6002 0 dn_6001_6002_6003 DB010  0 [REDO] LOG:  UBTreeXlogCleanup is over, it takes time:1 microseconds
2024-12-20 06:35:57.017 67649fcb.1113 [unknown] 140297854514944 dn_6002 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Page-redo-worker thread 1 terminated, retcode 0.
2024-12-20 06:35:57.018 67649fcb.1116 [unknown] 140297977722624 dn_6002 0 dn_6001_6002_6003 DB010  0 [REDO] LOG:  worker[2]: multipage cnt = 2, wait reach elapsed 0 us, wait replay elapsed 0 us, total elapsed = 1023365
2024-12-20 06:35:57.018 67649fcb.1116 [unknown] 140297977722624 dn_6002 0 dn_6001_6002_6003 DB010  0 [REDO] LOG:  btree_xlog_cleanup is over, it takes time:0 microseconds
2024-12-20 06:35:57.018 67649fcb.1116 [unknown] 140297977722624 dn_6002 0 dn_6001_6002_6003 DB010  0 [REDO] LOG:  UBTreeXlogCleanup is over, it takes time:0 microseconds
2024-12-20 06:35:57.018 67649fcb.1116 [unknown] 140297977722624 dn_6002 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Page-redo-worker thread 2 terminated, retcode 0.
2024-12-20 06:35:57.019 67649fcb.1110 [unknown] 140297445046016 dn_6002 0 dn_6001_6002_6003 DB010  0 [REDO] LOG:  worker[3]: multipage cnt = 2, wait reach elapsed 0 us, wait replay elapsed 0 us, total elapsed = 1024937
2024-12-20 06:35:57.019 67649fcb.1110 [unknown] 140297445046016 dn_6002 0 dn_6001_6002_6003 DB010  0 [REDO] LOG:  btree_xlog_cleanup is over, it takes time:0 microseconds
2024-12-20 06:35:57.019 67649fcb.1110 [unknown] 140297445046016 dn_6002 0 dn_6001_6002_6003 DB010  0 [REDO] LOG:  UBTreeXlogCleanup is over, it takes time:0 microseconds
2024-12-20 06:35:57.019 67649fcb.1110 [unknown] 140297445046016 dn_6002 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Page-redo-worker thread 3 terminated, retcode 0.
2024-12-20 06:35:57.020 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 DB010  0 [REDO] LOG:  page worker thread 140297854514944 exit, state 3
2024-12-20 06:35:57.020 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 DB010  0 [REDO] LOG:  page worker thread 140297977722624 exit, state 3
2024-12-20 06:35:57.020 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 DB010  0 [REDO] LOG:  page worker thread 140297445046016 exit, state 3
2024-12-20 06:35:57.022 67649fcb.1115 [unknown] 140297837733632 dn_6002 0 dn_6001_6002_6003 DB010  0 [REDO] LOG:  worker[0]: multipage cnt = 2, wait reach elapsed 0 us, wait replay elapsed 0 us, total elapsed = 1030911
2024-12-20 06:35:57.022 67649fcb.1115 [unknown] 140297837733632 dn_6002 0 dn_6001_6002_6003 DB010  0 [REDO] LOG:  btree_xlog_cleanup is over, it takes time:1 microseconds
2024-12-20 06:35:57.022 67649fcb.1115 [unknown] 140297837733632 dn_6002 0 dn_6001_6002_6003 DB010  0 [REDO] LOG:  UBTreeXlogCleanup is over, it takes time:0 microseconds
2024-12-20 06:35:57.022 67649fcb.1115 [unknown] 140297837733632 dn_6002 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Page-redo-worker thread 0 terminated, retcode 0.
2024-12-20 06:35:57.024 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 DB010  0 [REDO] LOG:  page worker thread 140297837733632 exit, state 3
2024-12-20 06:35:57.026 67649f47.1102 [unknown] 140298319558400 dn_6002 0 dn_6001_6002_6003 DB010  0 [REDO] LOG:  [REDO_LOG_TRACE]SendRecoveryEndMarkToWorkersAndWaitForFinish, disptach total elapsed: 0, txn elapsed: 0, process pending record elapsed: 0 code: 0
2024-12-20 06:35:57.026 67649f47.1102 [unknown] 140298319558400 dn_6002 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  set knl_g_set_redo_finish_status to true when redo done
2024-12-20 06:35:57.026 67649f47.1102 [unknown] 140298319558400 dn_6002 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  redo done at D/8FF297A8, end at D/8FF29848
2024-12-20 06:35:57.026 67649f47.1102 [unknown] 140298319558400 dn_6002 0 dn_6001_6002_6003 DB010  0 [REDO] LOG:  [PR]: Recoverying elapsed: 1029934 us, redoTotalBytes:448,EndRecPtr:58249615432, redoStartPtr:58249614984,speed:0 MB/s, totalTime:1029934
2024-12-20 06:35:57.026 67649f47.1102 [unknown] 140298319558400 dn_6002 0 dn_6001_6002_6003 DB010  0 [REDO] LOG:  [REDO_STATS]redo_unlink_stats_file: unlink redo.state sucessfully!, ret:-1
2024-12-20 06:35:57.026 67649f47.1102 [unknown] 140298319558400 dn_6002 0 dn_6001_6002_6003 DB010  0 [REDO] LOG:  [REDO_STATS]redo_dump_all_stats: the basic statistic during redo are as follows : redo_start_ptr:0000000D/8FF29688, redo_start_time:787962823801679, min_recovery_point:0000000D/8FF295E8, read_ptr:0000000D/8FF297A8, last_replayed_read_Ptr:0000000D/8FF29848, speed:0 KB/s
2024-12-20 06:35:57.026 67649f47.1102 [unknown] 140298319558400 dn_6002 0 dn_6001_6002_6003 DB010  0 [REDO] LOG:  [REDO_STATS]redo_dump_all_stats WAIT_READ_XLOG: the event io statistic during redo are as follows : total_duration:1012, counter:22
2024-12-20 06:35:57.026 67649f47.1102 [unknown] 140298319558400 dn_6002 0 dn_6001_6002_6003 DB010  0 [REDO] LOG:  [REDO_STATS]redo_dump_all_stats WAIT_READ_DATA: the event io statistic during redo are as follows : total_duration:0, counter:0
2024-12-20 06:35:57.026 67649f47.1102 [unknown] 140298319558400 dn_6002 0 dn_6001_6002_6003 DB010  0 [REDO] LOG:  [REDO_STATS]redo_dump_all_stats WAIT_WRITE_DATA: the event io statistic during redo are as follows : total_duration:0, counter:0
2024-12-20 06:35:57.026 67649f47.1102 [unknown] 140298319558400 dn_6002 0 dn_6001_6002_6003 DB010  0 [REDO] LOG:  [REDO_STATS]redo_dump_all_stats WAIT_PROCESS_PENDING: the event io statistic during redo are as follows : total_duration:16, counter:16
2024-12-20 06:35:57.026 67649f47.1102 [unknown] 140298319558400 dn_6002 0 dn_6001_6002_6003 DB010  0 [REDO] LOG:  [REDO_STATS]redo_dump_all_stats WAIT_APPLY: the event io statistic during redo are as follows : total_duration:16, counter:7
2024-12-20 06:35:57.026 67649f47.1102 [unknown] 140298319558400 dn_6002 0 dn_6001_6002_6003 DB010  0 [REDO] LOG:  [REDO_STATS]redo_dump_all_stats: the redo worker queue statistic during redo are as follows : worker info id:0, originId:0, tid:140297837733632, queue_usage:0, queue_max_usage:1, redo_rec_count:3
2024-12-20 06:35:57.026 67649f47.1102 [unknown] 140298319558400 dn_6002 0 dn_6001_6002_6003 DB010  0 [REDO] LOG:  [REDO_STATS]redo_dump_all_stats: the redo worker queue statistic during redo are as follows : worker info id:1, originId:1, tid:140297854514944, queue_usage:0, queue_max_usage:1, redo_rec_count:3
2024-12-20 06:35:57.026 67649f47.1102 [unknown] 140298319558400 dn_6002 0 dn_6001_6002_6003 DB010  0 [REDO] LOG:  [REDO_STATS]redo_dump_all_stats: the redo worker queue statistic during redo are as follows : worker info id:2, originId:2, tid:140297977722624, queue_usage:0, queue_max_usage:1, redo_rec_count:3
2024-12-20 06:35:57.026 67649f47.1102 [unknown] 140298319558400 dn_6002 0 dn_6001_6002_6003 DB010  0 [REDO] LOG:  [REDO_STATS]redo_dump_all_stats: the redo worker queue statistic during redo are as follows : worker info id:3, originId:3, tid:140297445046016, queue_usage:0, queue_max_usage:1, redo_rec_count:3
2024-12-20 06:35:57.026 67649f47.1102 [unknown] 140298319558400 dn_6002 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  startup shut down datareceiver.
2024-12-20 06:35:57.026 67649f47.1102 [unknown] 140298319558400 dn_6002 0 dn_6001_6002_6003 DB010  0 [REDO] LOG:  CollectInvalidPagesStates: nstates:4
2024-12-20 06:35:57.111 67649f47.1102 [unknown] 140298319558400 dn_6002 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  archive recovery complete
2024-12-20 06:35:57.113 67649f47.1102 [unknown] 140298319558400 dn_6002 0 dn_6001_6002_6003 DB010  0 [REDO] LOG:  btree_xlog_cleanup is over, it takes time:0 microseconds
2024-12-20 06:35:57.113 67649f47.1102 [unknown] 140298319558400 dn_6002 0 dn_6001_6002_6003 DB010  0 [REDO] LOG:  UBTreeXlogCleanup is over, it takes time:0 microseconds
2024-12-20 06:35:57.114 67649f49.1105 [unknown] 140298267129600 dn_6002 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  CreateRestartPoint PrintCkpXctlControlFile: [checkPoint] oldCkpLoc:D/8FF295E8, oldRedo:D/8FF295E8, newCkpLoc:D/8FF297A8, newRedo:D/8FF297A8, preCkpLoc:D/8FF295E8
2024-12-20 06:35:57.114 67649f49.1105 [unknown] 140298267129600 dn_6002 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  CreateRestartPoint, need flush 0 pages.
2024-12-20 06:35:57.464 67649f49.1105 [unknown] 140298267129600 dn_6002 0 dn_6001_6002_6003 00000  0 [DBL_WRT] LOG:  [batch flush] DW truncate end: file_head[dwn 1, start 31905], total_pages 0
2024-12-20 06:35:57.582 67649f49.1105 [unknown] 140298267129600 dn_6002 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  slotname: dn_6003, dummy: 0, restartlsn: D/8FF29548
2024-12-20 06:35:57.582 67649f49.1105 [unknown] 140298267129600 dn_6002 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  slotname: dn_6001, dummy: 0, restartlsn: D/8FF29548
2024-12-20 06:35:57.582 67649f49.1105 [unknown] 140298267129600 dn_6002 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  attempting to remove WAL segments older than log file 000000010000000D0000007D
2024-12-20 06:35:57.660 67649f47.1102 [unknown] 140298319558400 dn_6002 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  redo done, nextXid: 3439621, startupMaxXid: 3439621, recentLocalXmin: 3438530, recentGlobalXmin: 3438530, PendingPreparedXacts: 0, NextCommitSeqNo: 2294352, cutoff_csn_min: 0.
2024-12-20 06:35:57.660 [MOT] <TID:31754/00003> <SID:00003/00003> [INFO]     <Recovery>           RecoverDbEnd: Recovery finished successfully (CSN: 0, TxnId: 0)
2024-12-20 06:35:57.671 67649f47.1102 [unknown] 140298319558400 dn_6002 0 dn_6001_6002_6003 DB010  0 [REDO] LOG:  parallel redo workers are going to stop, code:0, arg:0
2024-12-20 06:35:57.671 67649f47.1102 [unknown] 140298319558400 dn_6002 0 dn_6001_6002_6003 DB010  0 [REDO] LOG:  parallel redo(startup) thread exit
2024-12-20 06:35:57.671 [MOT] <TID:31754/-----> <SID:-----/-----> [WARNING]  <FDW>                Ignoring request to delete session context: already deleted
2024-12-20 06:35:57.671 [MOT] <TID:31754/-----> <SID:-----/-----> [WARNING]  <System>             Ignoring attempt to cleanup current thread with invalid thread identifier: 65535
2024-12-20 06:35:57.671  [postmaster][reaper][140303910787520] LOG: startupprocess exit
2024-12-20 06:35:57.671 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Archive slot change to active when DBstatus Promoting
2024-12-20 06:35:57.671 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Archive slot change to active when DBstatus Promoting
2024-12-20 06:35:57.671 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Archive slot change to active when DBstatus Promoting
2024-12-20 06:35:57.671 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Archive slot change to active when DBstatus Promoting
2024-12-20 06:35:57.671 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Archive slot change to active when DBstatus Promoting
2024-12-20 06:35:57.671 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Archive slot change to active when DBstatus Promoting
2024-12-20 06:35:57.671 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Archive slot change to active when DBstatus Promoting
2024-12-20 06:35:57.671 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Archive slot change to active when DBstatus Promoting
2024-12-20 06:35:57.683 67649f4a.1114 [unknown] 140298055317248 dn_6002 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Stop the heartbeat server.
2024-12-20 06:35:57.683 67649f4a.1114 [unknown] 140298055317248 dn_6002 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  heartbeat thread shut down
2024-12-20 06:35:57.904 67649f45.1 [unknown] 140303910787520 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  gaussdb: fsync file "/data/relia1468/cluster/dn1/gaussdb.state.temp" success
2024-12-20 06:35:57.926  [postmaster][reaper][140303910787520] LOG: update gaussdb state file: db state(NORMAL_STATE), server mode(Primary)
2024-12-20 06:35:57.926  [postmaster][reaper][140303910787520] LOG: database system is ready to accept connections
2024-12-20 06:35:57.997 67649fcd.867 [unknown] 140297837733632 IMCStoreVacuumWorker 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  imcstore vacuum worker started
```

【测试代码】:

openGauss/openGauss-server

内容风险标识

评论 (1)

openGauss/openGauss-server .gitee-modal { width: 500px !important; }

内容风险标识