dataCompare is a database comparison and profiling platform
(1)support Hive table data comparison, MySQL、Doris data comparison, realize automatic configuration for data comparison, avoid frequent SQL writing for processing
(2)support easy configuration for data profiling
data-compare
(1)Interface-level interactive data comparison task configuration, low code and small amount of configuration to quickly generate comparison tasks
(2)Magnitude comparison, consistency comparison, automatic difference case discovery
(3)JDBC databases such as MySQL, Apache Hive, and Apache Doris are currently supported
(4)Already supports the comparison results to automatically send email alarm reports
data-profiling
(1)Data detection can be completed with low code and a small amount of configuration
(2)Primary key, enumeration value, null value detection
Technology stack:
End:Spring boot + Mybatis
DataBase:MySQL
Parsing Engine:Antrl
Big Data:Hive、Spark
Home
data-compare:
DbConfig
mysql config
hive config
job config
Comparison results are displayed
data-profiling:
job config
profiling result
java jdk8
mysql 5.7.36
Runing config
(1)Run the SQL files in the SQL directory in the database, create database and tables
(2)download the release jar(https://github.com/zhugezifang/dataCompare/releases) or build jar using the source code of the project
(3)edit database config information of application.yml
(4)run java -jar -Dspring.config.location=application.yml dataCompare.jar (application.yml and jar must in the same directory)
(5)visit http://127.0.0.1/ (UserName:admin PassWord:admin123)
Environment installation configuration
(1)If you want to implement Hive data comparison configuration, you need to install the Hive environment first(The installation documentation refers to the docker quick installation of the Hive environment:https://blog.csdn.net/ifenggege/article/details/107860477)
(2)After installation, when creating a new data source connection, select Hive at the address jdbc:hive2://ip:10000
Thanks ruoyi Provides front-end services
此处可能存在不合适展示的内容,页面不予展示。您可通过相关编辑功能自查并修改。
如您确认内容无涉及 不当用语 / 纯广告导流 / 暴力 / 低俗色情 / 侵权 / 盗版 / 虚假 / 无价值内容或违法国家有关法律法规的内容,可点击提交进行申诉,我们将尽快为您处理。
Activity
Community
Health
Trend
Influence
:Code submit frequency
:React/respond to issue & PR etc.
:Well-balanced team members and collaboration
:Recent popularity of project
:Star counts, download counts etc.