# k8s **Repository Path**: Sphere_Docker/consul_on_k8s ## Basic Information - **Project Name**: k8s - **Description**: kubernetes 部署consul集群方式 - **Primary Language**: Docker - **License**: Not specified - **Default Branch**: master - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 1 - **Forks**: 0 - **Created**: 2020-04-02 - **Last Updated**: 2023-11-19 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README # 在kubernetes上部署consul集群 #### 介绍 本次consul 集群的三个节点部署方式使用； consul集群成员之间使用TLS进行安全通信。 #### 软件使用 | 名称 | 版本 | 备注 | | :--------------: | :----: | :----------: | | Kubernetes | 1.14.6 | 阿里云 | | consul | 1.5.2 | Dokcerhub | | cfssl\|cfssljson | | 用于证书认证 | #### 业务部署 1. 下载源码： https://gitee.com/sub9a758/consul_on_k8s.git 2. 进入目录： `cd consul_cluster` 3. 生成生成 TLS 证书： ```shell # 1.初始化CA证书 cfssl gencert -initca ca/ca-csr.json | cfssljson -bare ca # 2. 使用以下命令创建 TLS 证书 和 私有密钥: cfssl gencert \ -ca=ca.pem \ -ca-key=ca-key.pem \ -config=ca/ca-config.json \ -profile=default \ ca/consul-csr.json | cfssljson -bare consul # 3. 执行完成后会在当前目录下生成 ca-key.pem ca.pem consul-key.pem consul.pem # 4. 生成 Consul Gossip 加密密钥 GOSSIP_ENCRYPTION_KEY=$(consul keygen) ``` 4. 生成集群的`Secret` ```shell # 1. 将 gossip 加密密钥 和 TLS 证书 存储在kubernetes的 Secret中: kubectl create secret generic consul \ --from-literal="gossip-encryption-key=${GOSSIP_ENCRYPTION_KEY}" \ --from-file=ca.pem \ --from-file=consul.pem \ --from-file=consul-key.pem ``` 5. 生成集群的`Configmap` ```shell kubectl create configmap consul --from-file=configs/server.json ``` 6. 生成 pv/pvc 这里我们使用的是阿里的nas 直接在界面创建 我们拿个生成的配置文件来 ```shell apiVersion: v1 kind: PersistentVolume metadata: annotations: pv.kubernetes.io/bound-by-controller: 'yes' creationTimestamp: '2019-09-29T01:16:07Z' finalizers: - kubernetes.io/pv-protection labels: alicloud-pvname: pv-consul-0 name: pv-consul-0 resourceVersion: '13901010' selfLink: /api/v1/persistentvolumes/pv-consul-0 uid: b712b6a6-e256-11e9-9bee-0e66c1930d6c spec: accessModes: - ReadWriteMany capacity: storage: 1Gi claimRef: apiVersion: v1 kind: PersistentVolumeClaim name: woqu-consul-0 namespace: default resourceVersion: '13901008' uid: edab-e256-11e9-ba9e-42b7e870 flexVolume: driver: alicloud/nas options: mode: '755' path: /consul/data/0 server: 0ae5b-cxxx5.cn-hangzhou.nas.aliyuncs.com vers: '3' persistentVolumeReclaimPolicy: Retain storageClassName: nas volumeMode: Filesystem status: phase: Bound ``` ```shell apiVersion: v1 kind: PersistentVolumeClaim metadata: annotations: pv.kubernetes.io/bind-completed: 'yes' pv.kubernetes.io/bound-by-controller: 'yes' creationTimestamp: '2019-09-29T01:17:39Z' finalizers: - kubernetes.io/pvc-protection name: woqu-consul-0 namespace: default resourceVersion: '13901014' selfLink: /api/v1/namespaces/default/persistentvolumeclaims/woqu-consul-0 uid: edab-e256-11e9-ba9e-42b751f70 spec: accessModes: - ReadWriteMany resources: requests: storage: 1Gi selector: matchLabels: alicloud-pvname: pv-consul-0 storageClassName: nas volumeMode: Filesystem volumeName: pv-consul-0 status: accessModes: - ReadWriteMany capacity: storage: 1Gi phase: Bound ``` 7. 创建`service` ```shell kubectl apply -f services/consul_service.yml ``` 8. 创建`Service Account` ```shell kubectl apply -f serviceaccounts/consul.yaml kubectl apply -f clusterroles/consul.yaml ``` 9. 创建pod ```shell apiVersion: apps/v1 kind: StatefulSet metadata: name: consul spec: selector: matchLabels: app: consul component: server serviceName: consul replicas: 3 template: metadata: labels: app: consul component: server annotations: "consul.hashicorp.com/connect-inject": "false" spec: serviceAccountName: consul affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: - key: app operator: In values: - consul topologyKey: kubernetes.io/hostname terminationGracePeriodSeconds: 10 securityContext: fsGroup: 1000 containers: - name: consul image: consul:1.5.2 env: - name: POD_IP valueFrom: fieldRef: fieldPath: status.podIP - name: GOSSIP_ENCRYPTION_KEY valueFrom: secretKeyRef: name: consul key: gossip-encryption-key - name: NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace args: - "agent" - "-server" - "-bootstrap-expect=3" - "-ui" - "-data-dir=/consul/data" - "-bind=$(POD_IP)" - "-client=0.0.0.0" - "-advertise=$(POD_IP)" - "-retry-join=consul-0.consul.$(NAMESPACE).svc.cluster.local" - "-retry-join=consul-1.consul.$(NAMESPACE).svc.cluster.local" - "-retry-join=consul-2.consul.$(NAMESPACE).svc.cluster.local" - "-domain=cluster.local" - "-datacenter=dc1" - "-encrypt=$(GOSSIP_ENCRYPTION_KEY)" - "-disable-host-node-id" volumeMounts: - name: woqu mountPath: /consul/data - name: config mountPath: /etc/consul/config - name: tls mountPath: /etc/tls lifecycle: preStop: exec: command: - /bin/sh - -c - consul leave ports: - containerPort: 8500 name: ui-port - containerPort: 8400 name: alt-port - containerPort: 53 name: udp-port - containerPort: 8443 name: https-port - containerPort: 8080 name: http-port - containerPort: 8301 name: serflan - containerPort: 8302 name: serfwan - containerPort: 8600 name: consuldns - containerPort: 8300 name: server volumes: - name: config configMap: name: consul - name: tls secret: secretName: consul volumeClaimTemplates: - metadata: name: woqu spec: accessModes: - ReadWriteMany resources: requests: storage: 1Gi ``` #### 查看服务 1. `kubectl get pods` ```shell NAME READY STATUS RESTARTS AGE consul-0 1/1 Running 0 8h consul-1 1/1 Running 0 7h58m consul-2 1/1 Running 0 8h ``` 2. 查看日志 ```shell kubectl logs consul-0 ``` 3. 查看集群 ```shell # kubectl exec -it consul-0 sh / # consul members Node Address Status Type Build Protocol DC Segment consul-0 192.168.59.3:8301 alive server 1.5.2 2 dc1 consul-1 192.168.57.10:8301 alive server 1.5.2 2 dc1 consul-2 192.168.59.12:8301 alive server 1.5.2 2 dc1 ``` 到此集群我们已经搭建完毕，如果想要外网访问网页我们在配置下服务的`Ingress`。