make shared_lib
which is in RocksDB's INSTALL.md
to compile rocksdb.Enter the GOPATH
directory, cd $GOPATH/src
mkdir -p github/vearch
cd github/vearch
Download the source code: git clone https://xxxxxx/vearch.git
($vearch denotes the absolute path of vearch code)
Compile gamma
cd $vearch/engine/gamma
mkdir build && cd build
export FAISS_HOME=the installed path of faiss
export ROCKSDB_HOME=the installed path of RocksDB
(optional)
cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=$vearch/ps/engine/gammacb/lib ..
make && make install
Compile vearch
cd $vearch
export FAISS_HOME=the installed path of faiss
export ROCKSDB_HOME=the installed path of RocksDB
(optional)
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$vearch/ps/engine/gammacb/lib/lib:$FAISS_HOME/lib
or export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$vearch/ps/engine/gammacb/lib/lib:$FAISS_HOME/lib:$ROCKSDB_HOME
if ROCKSDB_HOME is setgo build -o vearch
when vearch
file generated, it is ok../compile_env.sh
you will got a image named vearch_env
./compile.sh
you will compile Vearch in $vearch/build/bin
, $vearch/build/lib
./build.sh
you will got a image named vearch
good luckyou can use
docker run -it -v config.toml:/vearch/config.toml vearch all
to start vearch by local model the last param has four type[ps
,router
,master
,all
] all means tree type to start
./run_docker.sh
Before run vearch, you shuld set LD_LIBRARY_PATH
, Ensure that system can find faiss and gamma dynamic libraries (like $vearch/ps/engine/gammacb/lib/lib and $FAISS_HOME/lib directory files) .
[global]
# the name will validate join cluster by same name
name = "vearch"
# you data save to disk path ,If you are in a production environment, You'd better set absolute paths
data = ["datas/"]
# log path , If you are in a production environment, You'd better set absolute paths
log = "logs/"
# default log type for any model
level = "debug"
# master <-> ps <-> router will use this key to send or receive data
signkey = "vearch"
skip_auth = true
# if you are master you'd better set all config for router and ps and router and ps use default config it so cool
[[masters]]
# name machine name for cluster
name = "m1"
# ip or domain
address = "127.0.0.1"
# api port for http server
api_port = 8817
# port for etcd server
etcd_port = 2378
# listen_peer_urls List of comma separated URLs to listen on for peer traffic.
# advertise_peer_urls List of this member's peer URLs to advertise to the rest of the cluster. The URLs needed to be a comma-separated list.
etcd_peer_port = 2390
# List of this member's client URLs to advertise to the public.
# The URLs needed to be a comma-separated list.
# advertise_client_urls AND listen_client_urls
etcd_client_port = 2370
skip_auth = true
[router]
# port for server
port = 9001
# skip auth for client visit data
skip_auth = true
[ps]
# port for server
rpc_port = 8081
# raft config begin
raft_heartbeat_port = 8898
raft_replicate_port = 8899
heartbeat-interval = 200 #ms
raft_retain_logs = 10000
raft_replica_concurrency = 1
raft_snap_concurrency = 1
./vearch -conf conf.toml
vearch has three module:
ps
(PartitionServer) ,master
,router
, run./vearch -f conf.toml ps/router/master
start ps/router/master module
Now we have five machine, two master, two ps and one router
[global]
name = "vearch"
data = ["datas/"]
log = "logs/"
level = "debug"
signkey = "vearch"
skip_auth = true
# if you are master you'd better set all config for router and ps and router and ps use default config it so cool
[[masters]]
name = "m1"
address = "192.168.1.1"
api_port = 8817
etcd_port = 2378
etcd_peer_port = 2390
etcd_client_port = 2370
[[masters]]
name = "m2"
address = "192.168.1.2"
api_port = 8817
etcd_port = 2378
etcd_peer_port = 2390
etcd_client_port = 2370
[router]
port = 9001
skip_auth = true
[ps]
rpc_port = 8081
raft_heartbeat_port = 8898
raft_replicate_port = 8899
heartbeat-interval = 200 #ms
raft_retain_logs = 10000
raft_replica_concurrency = 1
raft_snap_concurrency = 1
./vearch -conf conf.toml master
./vearch -conf conf.toml ps
./vearch -conf conf.toml router
此处可能存在不合适展示的内容,页面不予展示。您可通过相关编辑功能自查并修改。
如您确认内容无涉及 不当用语 / 纯广告导流 / 暴力 / 低俗色情 / 侵权 / 盗版 / 虚假 / 无价值内容或违法国家有关法律法规的内容,可点击提交进行申诉,我们将尽快为您处理。