【测试类型：功能测试】【测试版本：6.0.0】【资源池化】开启按需回放实时构建，主机stop后，拉起主机后，前端返回成功，集群状态还是starting

【标题描述】:开启按需回放实时构建，主机stop后，拉起主机后，前端返回成功，集群状态还是starting
【测试类型：SQL功能/存储功能/接口功能/工具功能/性能/并发/压力长稳/故障注入/安全/资料/编码规范】【测试版本：6.0.0】 问题描述
开启按需回放实时构建，主机stop后，拉起主机后，前端返回成功，集群状态还是starting
【操作系统和硬件信息】(查询命令: cat /etc/system-release, uname -a):
Linux openGauss127 3.10.0-1160.108.1.el7.x86_64 #1 SMP Thu Jan 25 16:17:31 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
【测试环境】(单机/1主x备x级联备):
一主两备
【被测功能】:
cm选主
【测试类型】:
工具功能
【数据库版本】(查询命令: gaussdb -V):
gaussdb (openGauss 6.0.0 build ada68ffa) compiled at 2024-06-26 00:10:57 commit 0 last mr 
【预置条件】:
安装池化一主两备环境
三个节点开启按需回放实时构建
【操作步骤】(请填写详细的操作步骤):
1. cm_ctl stop -n 1
2. cm_ctl start -n 1

【预期输出】:
集群状态和前端返回一致
【实际输出】:
![输入图片说明](https://foruda.gitee.com/images/1719629454002710447/4c0abf88_12053974.png "屏幕截图")
![输入图片说明](https://foruda.gitee.com/images/1719629471007469589/d2ddc51c_12053974.png "屏幕截图")

![输入图片说明](https://foruda.gitee.com/images/1719998724139203986/ff68ce79_12053974.png "屏幕截图")

【原因分析】:
1. 这个问题的根因
2. 问题推断过程
3. 还有哪些原因可能造成类似现象
4. 该问题是否有临时规避措施
5. 问题解决方案
6. 预计修复问题时间

【日志信息】(请附上日志文件、截图、coredump信息):
日志备份信息：127、125、129
/data/backup/dss1_20240702/20240703
【测试代码】:

1. 根据pg_log日志分析：
125节点
2024-07-03 17:13:09.771 668508b7.1 [unknown] 139641357531840 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  DMS uninit worker threads, DRC, errdesc and DL
2024-07-03 17:24:31.794 [unknown] [unknown] localhost 140437427320576 0[0:0#0]  0 [DMS] LOG:  [SS reform] reform start enter: pmState=4, SSClusterState=0, demotion=0-0, rec=0
127节点
2024-07-03 17:24:28.973 668518cc.10000 [unknown] 140213110175488 dn_6001 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  [Alarm Module]alarm checker started.
2024-07-03 17:24:28.974 668518cc.10000 [unknown] 140213080160000 dn_6001 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  reaper backend started.
2024-07-03 17:24:29.201 [MOT] <TID:04021/-----> <SID:-----/-----> [INFO]     <System>             Startup: Loading configuration from /data/dss1_20240702/omm/openGauss/dn1/mot.con
129节点
2024-07-03 17:24:31.543 [unknown] [unknown] localhost 139947220137728 0[0:0#0]  0 [DMS] LOG:  [SS reform] reform start enter: pmState=4, SSClusterState=0, demotion=0-0, rec=0

在17:24:28，127节点才被拉起来，之后在17:24:31组织了DMS reform，用CM查询节点状态都是starting，DB侧日志和pg_log日志打印符合预期。

2. 根据127节点DBstart日志分析
2024-07-03 17:24:23.558 668518c5.1 [unknown] 140223647931072 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  ConfigRecoveryParallelism, parse workers:4, redo workers per parse worker:4, total workernums is 29
2024-07-03 17:24:23.590 668518c5.1 [unknown] 140223647931072 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  [Alarm Module]can not read GAUSS_WARNING_TYPE env.
2024-07-03 17:24:23.590 668518c5.1 [unknown] 140223647931072 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  [Alarm Module]Host Name: openGauss127
2024-07-03 17:24:23.590 668518c5.1 [unknown] 140223647931072 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  [Alarm Module]Host IP: openGauss127. Copy hostname directly in case of taking 10s to use 'gethostbyname' when /etc/hosts does not contain <HOST IP>
2024-07-03 17:24:23.590 668518c5.1 [unknown] 140223647931072 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  [Alarm Module]Cluster Name: openGauss
2024-07-03 17:24:23.590 668518c5.1 [unknown] 140223647931072 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  [Alarm Module]Invalid data in AlarmItem file! Read alarm English name failed! line: 58
2024-07-03 17:24:23.618 668518c5.1 [unknown] 140223647931072 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  loaded library "security_plugin"
2024-07-03 17:24:23.641 668518c5.1 [unknown] 140223647931072 [unknown] 0 dn_6001_6002_6003 01000  0 [BACKEND] WARNING:  could not create any HA TCP/IP sockets
2024-07-03 17:24:23.692 668518c5.1 [unknown] 140223647931072 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  gstrace initializes with failure. errno = -1.
2024-07-03 17:24:23.692 668518c5.1 [unknown] 140223647931072 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  InitNuma numaNodeNum: 1 numa_distribute_mode: none inheritThreadPool: 0.
2024-07-03 17:24:23.692 668518c5.1 [unknown] 140223647931072 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  reserved memory for backend threads is: 4420 MB
2024-07-03 17:24:23.692 668518c5.1 [unknown] 140223647931072 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  reserved memory for WAL buffers is: 320 MB
2024-07-03 17:24:23.692 668518c5.1 [unknown] 140223647931072 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  Set max backend reserve memory is: 4740 MB, max dynamic memory is: 242646 MB
2024-07-03 17:24:23.692 668518c5.1 [unknown] 140223647931072 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  shared memory 8613 Mbytes, memory context 247386 Mbytes, max process memory 257024 Mbytes
2024-07-03 17:24:23.692 668518c5.1 [unknown] 140223647931072 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  shared memory that key is 35301001 is owned by pid 24349
2024-07-03 17:24:24.736 668518c5.1 [unknown] 140223647931072 [unknown] 0 dn_6001_6002_6003 00000  0 [CACHE] LOG:  set data cache  size(805306368)
2024-07-03 17:24:25.392 668518c5.1 [unknown] 140223647931072 [unknown] 0 dn_6001_6002_6003 00000  0 [SEGMENT_PAGE] LOG:  Segment-page constants: DF_MAP_SIZE: 8156, DF_MAP_BIT_CNT: 65248, DF_MAP_GROUP_EXTENTS: 4175872, IPBLOCK_SIZE: 8168, EXTENTS_PER_IPBLOCK: 1021, IPBLOCK_GROUP_SIZE: 4090, BMT_HEADER_LEVEL0_TOTAL_PAGES: 8323072, BktMapEntryNumberPerBlock: 2038, BktMapBlockNumber: 25, BktBitMaxMapCnt: 512
2024-07-03 17:24:28.809 668518c5.1 [unknown] 140223647931072 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  gaussdb: fsync file "/data/dss1_20240702/omm/openGauss/dn1/gaussdb.state.temp" success
2024-07-03 17:24:28.809 668518c5.1 [unknown] 140223647931072 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  create gaussdb state file success: db state(STARTING_STATE), server mode(Normal), connection index(1)
2024-07-03 17:24:28.913 668518c5.1 [unknown] 140223647931072 [unknown] 0 dn_6001_6002_6003 00000  0 [BACKEND] LOG:  max_safe_fds = 965, usable_fds = 1000, already_open = 25
bbox_dump_path is set to /home/core/

数据库在17:24:23被拉起，17:24:28 gs_ctl侧处理完成，符合上面127节点的pg_log日志。

DSS在17:24:20被拉起，17:24:23完成集群加入，符合预期

4. 查看127节点cm_ctl日志
![输入图片说明](https://foruda.gitee.com/images/1723283949205966342/00d4beb3_7613729.png "屏幕截图")
可能的问题是，数据库实际未启动成功，但是cm返回成功，待CM分析

openGauss/CM

内容风险标识

评论 (8)

openGauss/CM .gitee-modal { width: 500px !important; }

内容风险标识