【测试类型：工具功能】【测试版本：7.0.0rc1】pg数据库增量迁移至openGauss数据库，枚举、数组类型迁移失败

【标题描述】:pg数据库增量迁移至openGauss数据库，枚举、数组类型迁移失败
【测试类型：工具功能】【测试版本：7.0.0rc1】 问题描述：pg数据库增量迁移至openGauss数据库，枚举、数组类型迁移失败

【操作系统和硬件信息】(查询命令: cat /etc/system-release, uname -a):
openEuler release 20.03 (LTS)
Linux openGauss85 4.19.90-2003.4.0.0036.oe1.aarch64 #1 SMP Mon Mar 23 19:06:43 UTC 2020 aarch64 aarch64 aarch64 GNU/Linux

【测试环境】(单机/1主x备x级联备):
单机

【被测功能】:
pg增量迁移

【测试类型】:
功能测试

【数据库版本】(查询命令: gaussdb -V):
gaussdb (openGauss 7.0.0-RC1 build 24e89e20) compiled at 2025-02-12 18:26:05 commit 0 last mr

【预置条件】:
增量迁移环境搭建成功

【操作步骤】(请填写详细的操作步骤):
1. 在pg与og分别预置表

```
CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
CREATE TABLE person (
    name text,
    current_mood mood
);
CREATE TYPE happiness AS ENUM ('happy', 'very happy', 'ecstatic');
CREATE TABLE holidays (
    num_weeks integer,
    happiness happiness
);
CREATE TABLE array_test (
    id SERIAL,
    t1 INT[],          
    t2 TEXT[],         
    t3 VARCHAR(10)[],  
    t4 JSONB[],        
    t5 INT[][],         
    t6 integer[],
    t7 text[][],
    t8 integer[3][3]
);

```

2. 在pg侧写入数据
```
INSERT INTO holidays(num_weeks,happiness) VALUES (4, 'happy');
INSERT INTO holidays(num_weeks,happiness) VALUES (6, 'very happy');
INSERT INTO holidays(num_weeks,happiness) VALUES (8, 'ecstatic');
INSERT INTO holidays(num_weeks,happiness) VALUES (2, 'sad');
INSERT INTO person VALUES ('Moe', 'happy');
INSERT INTO person VALUES ('Larry', 'sad');
INSERT INTO person VALUES ('Curly', 'ok');
INSERT INTO array_test (t1, t2, t3, t4, t5, t6, t7, t8) VALUES 
(
  ARRAY[-2147483648, 2147483647], 
  ARRAY['', 'max_length_string'],  
  ARRAY['1234567890', ''],        
  ARRAY['null'::jsonb, '{"key":[]}'], 
  ARRAY[[1],[2]],               
  '{}',                           
  ARRAY[['']],                    
  '{{1,2,3},{4,5,6},{7,8,9}}'  
);

INSERT INTO array_test (t1, t2, t6) VALUES 
(
  '{1, 2, 3}',
  ARRAY['特殊"字符', 'line\nbreak'],
  ARRAY[1, 2, 3]::integer[]
);

INSERT INTO array_test (t4) VALUES 
(
  ARRAY[
    '{"id":1, "tags":["A","B"]}'::jsonb,
    '[true, {"score":95.5}]'::jsonb 
  ]
);

INSERT INTO array_test (t8) VALUES 
(
  '{{1,2,3}, {4,5,6}, {7,8,9}}'
),
(
  ARRAY[[1,2,3], [4,5,6], [7,8,9]]
);

INSERT INTO array_test (t8) VALUES 
(ARRAY[1, 2, 3]), 
(ARRAY[ARRAY[NULL::INTEGER, NULL]]),
(ARRAY[NULL, 3, 5]::INTEGER[]);  
```
3. 在openGauss端查看增量迁移成功

【预期输出】:
1. 预置表成功
2. 写入数据成功
3. 在openGauss端查看增量迁移成功

【实际输出】:
1. 预置表成功
2. 写入数据成功
3. 增量迁移失败

```
:706)
[2025-02-20 19:56:01,788] INFO create statement info success (io.debezium.sink.worker.ReplayWorkThread:82)
[2025-02-20 19:56:01,803] ERROR convert occurred exception, columnName: current_mood, columnType: USER-DEFINED, value: sad (io.debezium.connector.postgresql.sink.utils.DebeziumValueConverters:140)
org.apache.kafka.connect.errors.DataException: Field 'current_mood' is not of type BYTES
        at org.apache.kafka.connect.data.Struct.getCheckType(Struct.java:263)
        at org.apache.kafka.connect.data.Struct.getBytes(Struct.java:165)
        at io.debezium.connector.postgresql.sink.utils.DebeziumValueConverters.convertByte(DebeziumValueConverters.java:337)
        at io.debezium.connector.postgresql.sink.utils.DebeziumValueConverters.access$000(DebeziumValueConverters.java:64)
        at io.debezium.connector.postgresql.sink.utils.DebeziumValueConverters$1.lambda$new$30(DebeziumValueConverters.java:112)
        at io.debezium.connector.postgresql.sink.utils.DebeziumValueConverters.getValue(DebeziumValueConverters.java:128)
        at io.debezium.connector.postgresql.sink.utils.SqlTools.getValueList(SqlTools.java:332)
        at io.debezium.connector.postgresql.sink.utils.SqlTools.getInsertSql(SqlTools.java:259)
        at io.debezium.connector.postgresql.sink.worker.PostgresDataReplayWorkThread.constructSql(PostgresDataReplayWorkThread.java:213)
        at io.debezium.connector.postgresql.sink.worker.PostgresDataReplayWorkThread.run(PostgresDataReplayWorkThread.java:129)
[2025-02-20 19:56:01,805] ERROR DataException occurred because of invalid field, possible reason is tables of OpenGauss and Postgresql have same table name but different table structure. (io.debezium.connector.postgresql.sink.worker.PostgresDataReplayWorkThread:154)
org.apache.kafka.connect.errors.DataException: org.apache.kafka.connect.errors.DataException: Field 'current_mood' is not of type BYTES
        at io.debezium.connector.postgresql.sink.utils.DebeziumValueConverters.getValue(DebeziumValueConverters.java:142)
        at io.debezium.connector.postgresql.sink.utils.SqlTools.getValueList(SqlTools.java:332)
        at io.debezium.connector.postgresql.sink.utils.SqlTools.getInsertSql(SqlTools.java:259)
        at io.debezium.connector.postgresql.sink.worker.PostgresDataReplayWorkThread.constructSql(PostgresDataReplayWorkThread.java:213)
        at io.debezium.connector.postgresql.sink.worker.PostgresDataReplayWorkThread.run(PostgresDataReplayWorkThread.java:129)
Caused by: org.apache.kafka.connect.errors.DataException: Field 'current_mood' is not of type BYTES
        at org.apache.kafka.connect.data.Struct.getCheckType(Struct.java:263)
        at org.apache.kafka.connect.data.Struct.getBytes(Struct.java:165)
        at io.debezium.connector.postgresql.sink.utils.DebeziumValueConverters.convertByte(DebeziumValueConverters.java:337)
        at io.debezium.connector.postgresql.sink.utils.DebeziumValueConverters.access$000(DebeziumValueConverters.java:64)
        at io.debezium.connector.postgresql.sink.utils.DebeziumValueConverters$1.lambda$new$30(DebeziumValueConverters.java:112)
        at io.debezium.connector.postgresql.sink.utils.DebeziumValueConverters.getValue(DebeziumValueConverters.java:128)
        ... 4 more
[2025-02-20 19:56:02,705] INFO incremental migration have replayed 4 data, and current time is 2025-02-20 19:56:02.705, and current speed is 1 (io.debezium.co
```

【原因分析】:
1. 这个问题的根因
2. 问题推断过程
3. 还有哪些原因可能造成类似现象
4. 该问题是否有临时规避措施
5. 问题解决方案
6. 预计修复问题时间

【日志信息】(请附上日志文件、截图、coredump信息):

【测试代码】:

openGauss/debezium

内容风险标识

评论 (4)

openGauss/debezium .gitee-modal { width: 500px !important; }

内容风险标识