# sample-jaeger

**Repository Path**: BGBiao/sample-jaeger

## Basic Information

- **Project Name**: sample-jaeger
- **Description**: 链路追踪系统 Jaeger 系统的 Helm Chart。 
- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 2
- **Created**: 2021-05-29
- **Last Updated**: 2022-08-24

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

## 一分钟快速构建一个分布式高可用的分布式追踪系统-Jaeger

**前提条件**

- 一个可用的Kubernetes 集群
- 一个6.2.X 版本的 ElasticSearch 实例(需要认证)
- helm 工具安装

**下载Jaeger Helm Chart**


```
# git clone https://gitee.com/BGBiao/sample-jaeger.git

# 目录结构

sh-4.2# cd sample-jaeger/
sh-4.2# ls
Chart.lock  charts  Chart.yaml	es  files  readme.md  templates  values.yaml
sh-4.2# tree -L 2 .
.
|-- Chart.lock
|-- charts
|-- Chart.yaml
|-- es
|   `-- elasticsearch.yml
|-- readme.md
|-- templates
|   |-- agent-deploy.yaml
|   |-- agent-svc.yaml
|   |-- collector-deploy.yaml
|   |-- collector-svc.yaml
|   |-- configmap.yaml
|   |-- elasticsearch-secret.yaml
|   |-- _helpers.tpl
|   |-- hotrod-deploy.yaml
|   |-- hotrod-ingress.yaml
|   |-- hotrod-svc.yaml
|   |-- query-deploy.yaml
|   `-- query-svc.yaml
`-- values.yaml

```


**修改 Chart 配置并安装 Release**

```

# 修改helm chart 中的es地址 storage.elasticsearch.host/storage.elasticsearch.password
#  部署整个jaeger 服务
sh-4.2# ls
Chart.lock  charts  Chart.yaml	es  files  readme.md  templates  values.yaml
sh-4.2# helm install -n sample-jaeger jaeger .
Error: create: failed to create: namespaces "sample-jaeger" not found
sh-4.2#


sh-4.2# kubectl  create ns sample-jaeger
namespace/sample-jaeger created
sh-4.2# helm install -n sample-jaeger jaeger .
NAME: jaeger
LAST DEPLOYED: Sun May 23 23:42:58 2021
NAMESPACE: sample-jaeger
STATUS: deployed
REVISION: 1
TEST SUITE: None
sh-4.2# helm list -n sample-jaeger
NAME  	NAMESPACE    	REVISION	UPDATED                               	STATUS  	CHART              	APP VERSION
jaeger	sample-jaeger	1       	2021-05-23 23:42:58.75504002 +0800 CST	deployed	sample-jaeger-0.1.0	1.16.0


sh-4.2# kubectl  get all -n sample-jaeger
NAME                                                       READY   STATUS    RESTARTS   AGE
pod/jaeger-agent-579587cb85-rzd46                          1/1     Running   0          27s
pod/jaeger-collector-57c57bc788-682sn                      1/1     Running   0          27s
pod/jaeger-query-85d86d44d-dhj7g                           1/1     Running   0          27s
pod/jaeger-sample-jaeger-jaeger-example-7bf65bd75b-mlxhv   1/1     Running   0          27s

NAME                                          TYPE       CLUSTER-IP       EXTERNAL-IP   PORT(S)                                                          AGE
service/jaeger-agent                          NodePort   172.16.127.123   <none>        6831:31139/UDP,5775:31355/TCP,6832:30937/TCP,5778:32360/TCP      27s
service/jaeger-agent-backup                   NodePort   172.16.125.116   <none>        6831:30036/UDP,5775:31138/TCP,6832:31179/TCP,5778:30568/TCP      27s
service/jaeger-collector                      NodePort   172.16.127.93    <none>        14250:30708/TCP,14268:32354/TCP,9411:32312/TCP,14267:30986/TCP   27s
service/jaeger-collector-backup               NodePort   172.16.125.137   <none>        14250:30477/TCP,14268:32513/TCP,9411:30679/TCP,14267:31237/TCP   27s
service/jaeger-query                          NodePort   172.16.124.251   <none>        16686:30843/TCP,16687:30530/TCP                                  27s
service/jaeger-query-backup                   NodePort   172.16.125.214   <none>        16686:30273/TCP,16687:30858/TCP                                  27s
service/jaeger-sample-jaeger-jaeger-example   NodePort   172.16.125.35    <none>        80:31255/TCP                                                     27s

NAME                                                  READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/jaeger-agent                          1/1     1            1           27s
deployment.apps/jaeger-collector                      1/1     1            1           27s
deployment.apps/jaeger-query                          1/1     1            1           27s
deployment.apps/jaeger-sample-jaeger-jaeger-example   1/1     1            1           27s

NAME                                                             DESIRED   CURRENT   READY   AGE
replicaset.apps/jaeger-agent-579587cb85                          1         1         1       27s
replicaset.apps/jaeger-collector-57c57bc788                      1         1         1       27s
replicaset.apps/jaeger-query-85d86d44d                           1         1         1       27s
replicaset.apps/jaeger-sample-jaeger-jaeger-example-7bf65bd75b   1         1         1       27s


# 这里我们测试下jaeger-query 的服务
sh-4.2# curl 172.16.124.251:16686 -I
HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
Date: Sun, 23 May 2021 15:44:39 GMT

# 此时发现我们核心的几个服务都没有问题(agent,collector,query)
# 接下里就可以直接访问我们的jaeger 的服务页面了
# 因为我们的service 采用了nodePort 类型，可以直接访问主机的ip:port

```

**Jaeger 系统使用**

![jaeger-首页](https://tva1.sinaimg.cn/large/008i3skNly1gqssor4at5j316b0u0tj8.jpg)

注意: 默认我们没有应用的数据采集，因此看不到任何链路相关的数据，其实可以看到，我们上面也部署了一个测试程序`jaeger-example`，它是一个简单但却完整的程序，贯穿了前后端，数据库缓存等全流程的逻辑，可以让我们清楚的查看整个调用链路。


这个时候我们直接打开`jaeger-example` 对应的地址即可。 `node:31255`

![jaeger-horod](https://tva1.sinaimg.cn/large/008i3skNly1gqsstii7c0j321a0qk434.jpg)

可以看到整个服务提供了一个全局视角，从用户的客户端id进入请求开始，来模拟全流程的服务请求流转，接下来就可以随意点击几个服务模块，让服务主动上报一些数据给jaeger  

![jaeger-hotROD-serving](https://tva1.sinaimg.cn/large/008i3skNly1gqst0eer6tj321k0sgqba.jpg)

模拟一些用户的请求后，我们就可以查看到服务的一些指标了

![choose-some-filter-find-trace](https://tva1.sinaimg.cn/large/008i3skNly1gqst2f95flj31670u0gxc.jpg)

可以看到刚才的模拟请求中，总共产生了5个Traces，也就分别对应了我们在 `hotROD` 服务中的5次请求

然后，点进去任何一条链路，都可以查看到整个调用过程的耗时情况

![tracing-timeline](https://tva1.sinaimg.cn/large/008i3skNly1gqst58e03cj316c0u07he.jpg)

![tracing-troubshooting](https://tva1.sinaimg.cn/large/008i3skNly1gqst8iy9b1j315w0u0an4.jpg)

从调用链路的方法调用，也能很快查看到调用超时以及异常模块

然后右上角可以选择 trace 数据的方式，比如可以看到调用关系，这样就很方便我们来查看一个完整服务的各个调用关系

![trace-graph](https://tva1.sinaimg.cn/large/008i3skNly1gqstb2t7raj31jb0u0tn1.jpg)


到这里，我们就可以看到，我们使用 helm 快速部署了一个分布式的 jaeger 集群

当然啦，hotROD 服务仅仅是测试服务，验证我们的jaeger 监控可用，之后我们需要将它销毁以释放相关的资源

**修改Helm Chart更新服务**

`注意:` 由于我们的 服务都是通过在chart 中的yaml定义的，因此服务的管控只需要改yaml就行。


修改`values.yaml` 文件中的`jaegerExample.enabled: false` 后，执行如下命令

```
# 制定配置文件，对指定的release 进行更新
sh-4.2# helm  upgrade -n sample-jaeger -f ./values.yaml jaeger .
Release "jaeger" has been upgraded. Happy Helming!
NAME: jaeger
LAST DEPLOYED: Mon May 24 00:18:46 2021
NAMESPACE: sample-jaeger
STATUS: deployed
REVISION: 2
TEST SUITE: None

# 可以看到，REVISION 已经更新到2
## 并且我们的jaeger-example 服务已经下线
sh-4.2# helm  list -n sample-jaeger
NAME  	NAMESPACE    	REVISION	UPDATED                                	STATUS  	CHART              	APP VERSION
jaeger	sample-jaeger	2       	2021-05-24 00:18:46.865941544 +0800 CST	deployed	sample-jaeger-0.1.0	1.16.0

sh-4.2# kubectl get all -n sample-jaeger
NAME                                    READY   STATUS    RESTARTS   AGE
pod/jaeger-agent-579587cb85-rzd46       1/1     Running   0          38m
pod/jaeger-collector-57c57bc788-682sn   1/1     Running   0          38m
pod/jaeger-query-85d86d44d-dhj7g        1/1     Running   0          38m

NAME                              TYPE       CLUSTER-IP       EXTERNAL-IP   PORT(S)                                                          AGE
service/jaeger-agent              NodePort   172.16.127.123   <none>        6831:31139/UDP,5775:31355/TCP,6832:30937/TCP,5778:32360/TCP      38m
service/jaeger-agent-backup       NodePort   172.16.125.116   <none>        6831:30036/UDP,5775:31138/TCP,6832:31179/TCP,5778:30568/TCP      38m
service/jaeger-collector          NodePort   172.16.127.93    <none>        14250:30708/TCP,14268:32354/TCP,9411:32312/TCP,14267:30986/TCP   38m
service/jaeger-collector-backup   NodePort   172.16.125.137   <none>        14250:30477/TCP,14268:32513/TCP,9411:30679/TCP,14267:31237/TCP   38m
service/jaeger-query              NodePort   172.16.124.251   <none>        16686:30843/TCP,16687:30530/TCP                                  38m
service/jaeger-query-backup       NodePort   172.16.125.214   <none>        16686:30273/TCP,16687:30858/TCP                                  38m

NAME                               READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/jaeger-agent       1/1     1            1           38m
deployment.apps/jaeger-collector   1/1     1            1           38m
deployment.apps/jaeger-query       1/1     1            1           38m

NAME                                          DESIRED   CURRENT   READY   AGE
replicaset.apps/jaeger-agent-579587cb85       1         1         1       38m
replicaset.apps/jaeger-collector-57c57bc788   1         1         1       38m
replicaset.apps/jaeger-query-85d86d44d        1         1         1       38m



```

如此，我们就使用helm 来很优雅的管理了我们服务的部署以及实例的管理。

最后，我们有没有发现，直接使用helm 可能会有几个问题: 

- 1. 命名空间没办法声明，且不能自动创建
- 2. 不支持像kubectl apply命令，来直接更新对应资源的配置
- 3. 无法做到环境的区分以及chart版本控制

其实，在社区里已经有相关的工具来解决这些问题了，就是 [helmfile](https://github.com/roboll/helmfile) ，它可以通过`helmfile.yaml` 文件来帮助用户管理和维护众多 helm chart，提高部署的可观测性和可重复性，区分环境，免去各种 `--set` 参数的困扰，感兴趣的同学可以去试一试。




**备注**

如果没有 ElasticSearch 实例可以使用 Docker 方式快速创建一个单节点实例

```

# 使用docker 方式准备一个外置的es 实例
$ docker run --name jaeger-es  -itd -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" -v $PWD/es/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml elasticsearch:6.4.3

$ curl localhost:9200
{"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication token for REST request [/]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}}],"type":"security_exception","reason":"missing authentication token for REST request [/]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}},"status":401}

# 因为es 开启了认证，需要先设置密码访问
sh-4.2# docker exec -it jaeger-es-tmp bash
./bin/elasticsearch-setup-passwords
Sets the passwords for reserved users

Commands
--------
auto - Uses randomly generated passwords
interactive - Uses passwords entered by a user

Non-option arguments:
command

Option         Description
------         -----------
-h, --help     show help
-s, --silent   show minimal output
-v, --verbose  show verbose output
ERROR: Missing command

# 设置elasticstack 所有的密码
[root@b90822f2c26b elasticsearch]# ./bin/elasticsearch-setup-passwords interactive
Initiating the setup of passwords for reserved users elastic,kibana,logstash_system,beats_system.
You will be prompted to enter passwords as the process progresses.
Please confirm that you would like to continue [y/N]y


Enter password for [elastic]:
Reenter password for [elastic]:
Enter password for [kibana]:
Reenter password for [kibana]:
Enter password for [logstash_system]:
Reenter password for [logstash_system]:
Enter password for [beats_system]:
Reenter password for [beats_system]:
Changed password for user [kibana]
Changed password for user [logstash_system]
Changed password for user [beats_system]
Changed password for user [elastic]

# 使用密码访问es实例
$ curl -u 'elastic:xxxxxxx' http://localhost:9200
{
  "name" : "1xdrvFl",
  "cluster_name" : "docker-cluster",
  "cluster_uuid" : "CO76XmP0Qbexysev4Al2MQ",
  "version" : {
    "number" : "6.4.3",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "fe40335",
    "build_date" : "2018-10-30T23:17:19.084789Z",
    "build_snapshot" : false,
    "lucene_version" : "7.4.0",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  },
  "tagline" : "You Know, for Search"
}
```