2 Star 1 Fork 1

mirrors_mariadb-corporation/mariadb-columnstore-data-adapters

加入 Gitee
与超过 1200万 开发者一起发现、参与优秀开源项目,私有仓库也完全免费 :)
免费加入
文件
该仓库未声明开源许可证文件(LICENSE),使用请关注具体项目描述及其代码上游依赖。
克隆/下载
贡献代码
同步代码
取消
提示: 由于 Git 不支持空文件夾,创建文件夹后会生成空的 .keep 文件
Loading...
README

MaxScale Kakfa+CDC to ColumnStore Data Adapter

The MaxScale Kakfa+CDC to ColumnStore data adapter reads change events sent by MariaDB MaxScale to MariaDB ColumnStore via Kafka.

How to Build

The adapter requires the following libraries present on the system.

  • mcsapi (an DEB/RPM is provided by MariaDB, please install prior to the instructions below)
  • jansson
  • rdkafka++

Ubuntu Xenial (16.04)

sudo add-apt-repository ppa:opencontrail/ppa
sudo apt-get update
sudo apt-get install cmake git g++ libjansson-dev librdkafka-dev libssl-dev
git clone https://github.com/mariadb-corporation/mariadb-columnstore-data-adapters
mkdir build && cd build
cmake ../mariadb-columnstore-data-adapters -DCMAKE_INSTALL_PREFIX=/usr -DKAFKA=OFF -DKETTLE=OFF -DMAX_CDC=OFF -DMAX_KAFKA=ON
make
sudo make install

Debian Stretch (9)

sudo apt-get update
sudo apt-get install cmake git g++ libjansson-dev librdkafka-dev libssl-dev
git clone https://github.com/mariadb-corporation/mariadb-columnstore-data-adapters
mkdir build && cd build
cmake ../mariadb-columnstore-data-adapters -DCMAKE_INSTALL_PREFIX=/usr -DKAFKA=OFF -DKETTLE=OFF -DMAX_CDC=OFF -DMAX_KAFKA=ON
make
sudo make install

Debian Jessie (8)

Add the following to your /etc/apt/sources.list:

deb http://ftp.debian.org/debian jessie-backports main

Then:

sudo apt-get update
sudo apt-get install cmake git g++ libjansson-dev librdkafka-dev=0.9.3-1~bpo8+1 librdkafka1=0.9.3-1~bpo8+1 libssl-dev
git clone https://github.com/mariadb-corporation/mariadb-columnstore-data-adapters
mkdir build && cd build
cmake ../mariadb-columnstore-data-adapters -DCMAKE_INSTALL_PREFIX=/usr -DKAFKA=OFF -DKETTLE=OFF -DMAX_CDC=OFF -DMAX_KAFKA=ON
make
sudo make install

RHEL/CentOS 7

sudo yum install git cmake gcc-c++ jansson-devel librdkafka-devel openssl-devel
git clone https://github.com/mariadb-corporation/mariadb-columnstore-data-adapters
mkdir build && cd build
cmake ../mariadb-columnstore-data-adapters -DCMAKE_INSTALL_PREFIX=/usr -DKAFKA=OFF -DKETTLE=OFF -DMAX_CDC=OFF -DMAX_KAFKA=ON
make
sudo make install

Usage

Usage: mcskafka [OPTION...] BROKER TOPIC SCHEMA TABLE
mcskafka - A Kafka consumer to write to MariaDB ColumnStore

  -g, --group=GROUP_ID       The Kafka group ID (default 1)
  -?, --help                 Give this help list
      --usage                Give a short usage message
  -V, --version              Print program version
  • BROKER: The host/IP of the Kafka broker server
  • TOPIC: The Kafka topic to consume
  • SCHEMA: The target ColumnStore schema name
  • TABLE: The target ColumnStore table name

Quickstart

Setup MaxScale and Kafka as outlined in the MaxScale Kafka Blog Post.

Start mcskafka using the follow (assuming your Kakfa server is localhost):

mcskafka localhost CDC_DataStream test t1

You can exit safely at any time using Ctrl-C.

马建仓 AI 助手
尝试更多
代码解读
代码找茬
代码优化
1
https://gitee.com/mirrors_mariadb-corporation/mariadb-columnstore-data-adapters.git
git@gitee.com:mirrors_mariadb-corporation/mariadb-columnstore-data-adapters.git
mirrors_mariadb-corporation
mariadb-columnstore-data-adapters
mariadb-columnstore-data-adapters
master

搜索帮助