22 Star 33 Fork 51

openGauss / debezium

 / 详情

【测试类型:工具功能】【测试版本:5.1.0】增量迁移设置commit.process.while.running=true、 sink.process.file.path='' 时,不展示迁移进度日志没有明显提示信息

已验收
缺陷
创建于  
2023-06-05 15:03

【标题描述】:增量迁移设置commit.process.while.running=true、 sink.process.file.path='' 时,不展示迁移进度日志没有明显提示信息
【测试类型:工具功能】【测试版本:5.1.0】 增量迁移设置commit.process.while.running=true、 sink.process.file.path='' 时,不展示迁移进度日志没有明显提示信息
source.process.file.path设为相对路径时,也不展示迁移进度日志没有明显提示信息

【操作系统和硬件信息】(查询命令: cat /etc/system-release, uname -a):
openEuler release 20.03 (LTS-SP1)
Linux kwepwebenv15399 4.19.90-2012.5.0.0054.oe1.x86_64 #1 SMP Tue Dec 22 15:58:47 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

【测试环境】(单机/1主x备x级联备):单机

【被测功能】:增量迁移进度展示功能

【测试类型】:工具功能

【数据库版本】(查询命令: gaussdb -V):
gaussdb (openGauss 5.1.0 build 54ad2fd4) compiled at 2023-05-17 00:10:18 commit 0 last mr
mysql 5.7

【预置条件】:
迁移环境已部署完成

【操作步骤】(请填写详细的操作步骤):
1.source端和sink端分别
commit.process.while.running=true
source.process.file.path=''

设置commit.process.while.running=true
sink.process.file.path=''

2.source端和sink端分别设置
create.count.info.path=/data/houshaolong/speed_source/
create.count.info.path=''

【预期输出】:
1.数据迁移成功,有信息提示不展示迁移进度
2.数据迁移成功,sink端有提示信息报参数设置不合理

【实际输出】:
1.数据迁移成功,没有提示信息,不展示迁移进度

./bin/connect-standalone etc/schema-registry/connect-avro-standalone.properties etc/kafka/mysql-source.properties
[2023-06-05 15:10:13,508] WARN The configuration 'value.converter.schema.registry.url' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:355)
[2023-06-05 15:10:13,511] WARN The configuration 'internal.key.converter' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:355)
[2023-06-05 15:10:13,511] WARN The configuration 'offset.storage.file.filename' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:355)
[2023-06-05 15:10:13,511] WARN The configuration 'internal.value.converter.schemas.enable' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:355)
[2023-06-05 15:10:13,512] WARN The configuration 'internal.value.converter' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:355)
[2023-06-05 15:10:13,512] WARN The configuration 'plugin.path' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:355)
[2023-06-05 15:10:13,512] WARN The configuration 'value.converter' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:355)
[2023-06-05 15:10:13,512] WARN The configuration 'internal.key.converter.schemas.enable' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:355)
[2023-06-05 15:10:13,512] WARN The configuration 'key.converter' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:355)
[2023-06-05 15:10:13,512] WARN The configuration 'key.converter.schema.registry.url' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:355)
Jun 05, 2023 3:10:14 PM org.glassfish.jersey.internal.inject.Providers checkProviderRuntime
WARNING: A provider org.apache.kafka.connect.runtime.rest.resources.RootResource registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider org.apache.kafka.connect.runtime.rest.resources.RootResource will be ignored.
Jun 05, 2023 3:10:14 PM org.glassfish.jersey.internal.inject.Providers checkProviderRuntime
WARNING: A provider org.apache.kafka.connect.runtime.rest.resources.ConnectorPluginsResource registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider org.apache.kafka.connect.runtime.rest.resources.ConnectorPluginsResource will be ignored.
Jun 05, 2023 3:10:14 PM org.glassfish.jersey.internal.inject.Providers checkProviderRuntime
WARNING: A provider org.apache.kafka.connect.runtime.rest.resources.LoggingResource registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider org.apache.kafka.connect.runtime.rest.resources.LoggingResource will be ignored.
Jun 05, 2023 3:10:14 PM org.glassfish.jersey.internal.inject.Providers checkProviderRuntime
WARNING: A provider org.apache.kafka.connect.runtime.rest.resources.ConnectorsResource registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider org.apache.kafka.connect.runtime.rest.resources.ConnectorsResource will be ignored.
Jun 05, 2023 3:10:14 PM org.glassfish.jersey.internal.Errors logErrors
WARNING: The following warnings have been detected: WARNING: The (sub)resource method listLoggers in org.apache.kafka.connect.runtime.rest.resources.LoggingResource contains empty path annotation.
WARNING: The (sub)resource method createConnector in org.apache.kafka.connect.runtime.rest.resources.ConnectorsResource contains empty path annotation.
WARNING: The (sub)resource method listConnectors in org.apache.kafka.connect.runtime.rest.resources.ConnectorsResource contains empty path annotation.
WARNING: The (sub)resource method listConnectorPlugins in org.apache.kafka.connect.runtime.rest.resources.ConnectorPluginsResource contains empty path annotation.
WARNING: The (sub)resource method serverInfo in org.apache.kafka.connect.runtime.rest.resources.RootResource contains empty path annotation.

[2023-06-05 15:10:15,511] WARN The configuration 'value.serializer' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:355)
[2023-06-05 15:10:15,512] WARN The configuration 'batch.size' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:355)
[2023-06-05 15:10:15,512] WARN The configuration 'max.block.ms' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:355)
[2023-06-05 15:10:15,512] WARN The configuration 'acks' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:355)
[2023-06-05 15:10:15,512] WARN The configuration 'buffer.memory' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:355)
[2023-06-05 15:10:15,512] WARN The configuration 'key.serializer' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:355)
[2023-06-05 15:10:15,513] WARN The configuration 'linger.ms' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:355)
Jun 05, 2023 3:10:16 PM com.github.shyiko.mysql.binlog.BinaryLogClient connect
INFO: Connected to 10.29.163.42:11666 at 5737c725-b8c6-11ed-aa54-fa163ecae5e4:1-30181 (sid:1, cid:472)
Jun 05, 2023 3:10:17 PM com.github.shyiko.mysql.binlog.BinaryLogClient$8 run

2.数据迁移成功,sink端没有提示信息报参数设置不合理
【原因分析】:

  1. 这个问题的根因
  2. 问题推断过程
  3. 还有哪些原因可能造成类似现象
  4. 该问题是否有临时规避措施
  5. 问题解决方案
  6. 预计修复问题时间

【日志信息】(请附上日志文件、截图、coredump信息):

【测试代码】:

评论 (3)

houshaolong 创建了缺陷

Hey @houshaolong, Welcome to openGauss Community.
All of the projects in openGauss Community are maintained by @opengauss-bot.
That means the developers can comment below every pull request or issue to trigger Bot Commands.
Please follow instructions at Here to find the details.

opengauss-bot 添加了
 
sig/tools
标签

Hi @houshaolong,
if you want to get quick review about your issue, please contact the owner in first: @douxin ,
and then any of the maintainers: @熊小军
and then any of the committers: @周斌 , @pengjiong , @lizhenfeng , @贺承汉 , @buter , @douxin
if you have any question, please contact the SIG: Tools.

houshaolong 负责人设置为王正元
houshaolong 添加协作者周斌
houshaolong 优先级设置为次要
houshaolong 关联分支设置为master
houshaolong 修改了描述
houshaolong 修改了描述
王正元 任务状态待办的 修改为已确认
王正元 任务状态已确认 修改为修复中
houshaolong 修改了描述
王正元 通过opengauss/debezium Pull Request !109任务状态修复中 修改为已完成
houshaolong 任务状态已完成 修改为测试中

验收通过:
source端提示信息:
输入图片说明
sink端提示信息:
输入图片说明
source端迁移文件内容:
输入图片说明
sink端迁移文件内容:
输入图片说明

houshaolong 任务状态测试中 修改为已验收

登录 后才可以发表评论

状态
负责人
项目
里程碑
Pull Requests
关联的 Pull Requests 被合并后可能会关闭此 issue
分支
开始日期   -   截止日期
-
置顶选项
优先级
预计工期 (小时)
参与者(3)
5622128 opengauss bot 1581905080
1
https://gitee.com/opengauss/debezium.git
git@gitee.com:opengauss/debezium.git
opengauss
debezium
debezium

搜索帮助

53164aa7 5694891 3bd8fe86 5694891