2 Star 2 Fork 1

atompi / elk-docker

加入 Gitee
与超过 1200万 开发者一起发现、参与优秀开源项目,私有仓库也完全免费 :)
免费加入
克隆/下载
贡献代码
同步代码
取消
提示: 由于 Git 不支持空文件夾,创建文件夹后会生成空的 .keep 文件
Loading...
README
MIT

ELK stack

ELK stack 是以 Elasticsearch、Logstash、Kibana 三个开源软件为主的数据处理工具链。

在实时数据检索和分析场合,三者通常是配合使用,而且又先后归于Elastic.io公司名下,故有Elastic Stack简称。


ELK stack部署 (elasticsearch 2.4.4、Logstash 2.4.1、Kibana 4.6.4)

分为Linux Server Host部署方式和Docker部署方式。

初始化Linux Server Host

修改时区:
> sudo cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
> sudo dpkg-reconfigure -f noninteractive tzdata

Clone本仓库
> cd ~/
> git clone https://git.oschina.net/atompi/elk-docker.git
> cd ~/elk-docker

修改apt源为阿里云:
> sudo cp sources.list /etc/apt/sources.list
> sudo apt-get update

elasticsearch部署 (单节点)

Linux Server Host部署方式 (Ubuntu 16.04)

安装openjdk-8

> sudo apt install -y openjdk-8-jdk

修改内核参数

> echo "vm.max_map_count=262144" | sudo tee -a /etc/sysctl.conf
> sudo sysctl -w vm.max_map_count=262144

下载elasticsearch deb包

> cd /usr/local/src
> sudo wget https://download.elastic.co/elasticsearch/release/org/elasticsearch/distribution/deb/elasticsearch/2.4.4/elasticsearch-2.4.4.deb

安装elasticsearch

> cd /usr/local/src
> sudo dpkg -i elasticsearch-2.4.4.deb

配置elasticsearch

echo "network.host: $ELS_HOST" >> /etc/elasticsearch/elasticsearch.yml      # $ELS_HOST为elasticsearch部署的节点host ip 

启动elasticsearch

sudo service elasticsearch start

查看elasticsearch日志

tail -f /var/log/elasticsearch/elasticsearch.log

Logstash部署

Linux Server Host部署方式 (Ubuntu 16.04)

安装redis-3.2.8

> cd /usr/local/src
> sudo wget http://download.redis.io/releases/redis-3.2.8.tar.gz /usr/local/src/
> cd /usr/local/src && sudo tar -zxf redis-3.2.8.tar.gz
> cd /usr/local/src/redis-3.2.8
> make && sudo make install
> cd utils && (echo;echo;echo;echo;echo;echo)|sudo ./install_server.sh      # 默认方式安装redis server 也可以去掉 '|' 前面的命令组 (echo;echo;echo;echo;echo;echo) ,部署redis server时手动设置参数
> sudo sed -i 's/127.0.0.1/0.0.0.0/g' /etc/redis/6379.conf       # 通过redis手机shipper日志,所以要配置远程可访问

安装Logstash

> cd /usr/local/src
> sudo wget https://download.elastic.co/logstash/logstash/packages/debian/logstash-2.4.1_all.deb
> sudo dpkg -i logstash-2.4.1_all.deb

配置Logstash indexer (收集nginx access log)

> sudo mkdir -p /opt/logstash/patterns
> sudo cp elk/logstash-2.4.1/nginx /opt/logstash/patterns/
> sudo cp elk/logstash-2.4.1/nginx_access_indexer.conf /etc/logstash/conf.d/

配置Logstash shipper (收集nginx access log)

>sudo cp elk/logstash-2.4.1/nginx_access_shipper.conf /etc/logstash/conf.d/

启动Logstash

> sudo service logstash start

nginx log format (源码位于elk/logstash-2.4.1/nginx_logformat.conf)

log_format access_log    '$remote_addr - $remote_user [$time_local] "$request" '
                         '$status $body_bytes_sent "$http_referer" '
                         '"$http_user_agent" $http_x_forwarded_for '
                         '$request_time $upstream_response_time ';

access_log  logs/access.log access_log;

Kibana部署

Linux Server Host部署方式 (Ubuntu 16.04)

安装nginx、apache2-utils

> sudo apt install -y nginx apache2-utils

配置nginx、htpasswd验证

> sudo cp kibana.conf /etc/nginx/conf.d/
> sudo htpasswd -cb /opt/kibana/htpasswd.users kibana $KIBANA_PWD        # $KIBANA_PWD kibana用户密码

安装Kibana

> cd /usr/local/src
> sudo wget https://download.elastic.co/kibana/kibana/kibana-4.6.4-linux-x86_64.tar.gz
> sudo tar -zxf kibana-4.6.4-linux-x86_64.tar.gz
> sudo mv kibana-4.6.4-linux-x86_64 /opt/kibana

配置Kibana

> sudo echo "server.host: \"localhost\"" >> /opt/kibana/conf/kibana.conf
> sudo echo "elasticsearch.url: \"http:\/\/$ELS_HOST:9200\"

启动Kibana

> sudo mkdir /var/log/kibana
> su
> nohup /opt/kibana/bin/kibana > /var/log/kibana/kibana.log &

启动nginx

> sudo nginx -t
> sudo nginx -s reload

查看Kibala日志

> /var/log/kibana/kibana.log

浏览器访问

修改host,添加条目:$KIBANA_HOST elk.oschina.cc
浏览器访问地址:http://elk.oschina.cc
用户名:kibana
密码:$KIBANA_PWD设置的密码
The MIT License (MIT) Copyright (c) 2017 atompi Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

简介

elk stack Dockerfile 展开 收起
Docker
MIT
取消

发行版

暂无发行版

贡献者

全部

近期动态

加载更多
不能加载更多了
Docker
1
https://gitee.com/atompi/elk-docker.git
git@gitee.com:atompi/elk-docker.git
atompi
elk-docker
elk-docker
master

搜索帮助