Version 2.0.5, commit f84000843bf2ba98adea547721b235e3b77168e1
NAME="openEuler"
VERSION="20.09"
ID="openEuler"
VERSION_ID="20.09"
PRETTY_NAME="openEuler 20.09"
ANSI_COLOR="0;31"
k8s版本:v1.17.9
部署k8s后pod无法启动,查看isulad的日志报以下错误
{
"group": "isulad",
"default-runtime": "lcr",
"graph": "/var/lib/isulad",
"state": "/var/run/isulad",
"engine": "lcr",
"log-level": "ERROR",
"pidfile": "/var/run/isulad.pid",
"log-opts": {
"log-file-mode": "0600",
"log-path": "/var/lib/isulad",
"max-file": "1",
"max-size": "30KB"
},
"log-driver": "stdout",
"hook-spec": "/etc/default/isulad/hooks/default.json",
"start-timeout": "2m",
"storage-driver": "overlay2",
"storage-opts": [
"overlay2.override_kernel_check=true"
],
"registry-mirrors": [
"docker.io"
],
"insecure-registries": [
],
"pod-sandbox-image": "kubesphere/pause:3.2",
"native.umask": "secure",
"network-plugin": "cni",
"cni-bin-dir": "/opt/cni/bin",
"cni-conf-dir": "/etc/cni/net.d",
"image-layer-check": false,
"use-decrypted-key": true,
"insecure-skip-verify-enforce": false
}
Hey pixiake, Welcome to openEuler Community.
All of the projects in openEuler Community are maintained by @openeuler-ci-bot.
That means the developers can comment below every pull request or issue to trigger Bot Commands.
Please follow instructions at https://gitee.com/openeuler/community/blob/master/en/sig-infrastructure/command.md to find the details.
此处可能存在不合适展示的内容,页面不予展示。您可通过相关编辑功能自查并修改。
如您确认内容无涉及 不当用语 / 纯广告导流 / 暴力 / 低俗色情 / 侵权 / 盗版 / 虚假 / 无价值内容或违法国家有关法律法规的内容,可点击提交进行申诉,我们将尽快为您处理。
1.isulad 的配置文件/etc/isulad/daemon.json
参考官方文档,增加hosts 的设置项
{
"group": "isulad",
"default-runtime": "lcr",
"graph": "/var/lib/isulad",
"state": "/var/run/isulad",
"engine": "lcr",
"log-level": "ERROR",
"pidfile": "/var/run/isulad.pid",
"log-opts": {
"log-file-mode": "0600",
"log-path": "/var/lib/isulad",
"max-file": "1",
"max-size": "30KB"
},
"log-driver": "stdout",
"hook-spec": "/etc/default/isulad/hooks/default.json",
"start-timeout": "2m",
"storage-driver": "overlay2",
"storage-opts": [
"overlay2.override_kernel_check=true"
],
"registry-mirrors": [ # 配置镜像仓库地址
"https://72idtxd8.mirror.aliyuncs.com",
"https://reg-mirror.qiniu.com",
"http://hub-mirror.c.163.com"
],
"insecure-registries": [ # 不使用 TLS 校验的镜像仓库
"192.168.1.153:5000"
],
"pod-sandbox-image": "192.168.1.153:5000/kubesphere/pause:3.1", # pod 默认使用镜像
"native.umask": "secure",
"network-plugin": "cni", # 指定网络插件为 cni
"cni-bin-dir": "",
"cni-conf-dir": "",
"image-layer-check": false,
"use-decrypted-key": true,
"insecure-skip-verify-enforce": false,
"hosts" : [ # 指定通信方式
"unix:///var/run/isulad.sock"
]
}
2.使用命令kubeadm config print init-defaults > init.yml
可导出初始化配置文件,根据实际环境修改init.yml
apiVersion: kubeadm.k8s.io/v1beta2
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
token: abcdef.0123456789abcdef
ttl: 24h0m0s
usages:
- signing
- authentication
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.1.153 # 修改为本机IP
bindPort: 6443
nodeRegistration:
criSocket: /var/run/isulad.sock # 修改容器引擎为isulad
name: master # 修改为本机的主机名
taints:
- effect: NoSchedule
key: node-role.kubernetes.io/master
---
apiServer:
timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta2
certificatesDir: /etc/kubernetes/pki
clusterName: cluster.local # 修改k8s集群的名字
controlPlaneEndpoint: lb.kubesphere.local # 修改可以映射到 IP 地址的 DNS 名称
controllerManager: {}
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/etcd
imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers # 使用国内镜像仓库
kind: ClusterConfiguration
kubernetesVersion: v1.17.9 # 修改k8s版本号
networking:
dnsDomain: cluster.local
serviceSubnet: 10.233.0.0/18
podSubnet: 10.233.64.0/18
scheduler: {}
执行kubeadm init --config=init.yml
完成初始化集群
3.启用 CNI 插件
编辑 kubelet 的配置文件/usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf,传递 --network-plugin=cni 命令行选项来选择 CNI 插件,重启kubelet 服务
# Note: This dropin only works with kubeadm and kubelet v1.11+
[Service]
Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --cgroup-driver=cgroupfs --network-plugin=cni --pod-infra-container-image=192.168.1.153:5000/kubesphere/pause:3.1 --node-ip=192.168.1.153 --hostname-override=master"
Environment="KUBELET_CONFIG_ARGS=--config=/var/lib/kubelet/config.yaml"
# This is a file that "kubeadm init" and "kubeadm join" generates at runtime, populating the KUBELET_KUBEADM_ARGS variable dynamically
EnvironmentFile=-/var/lib/kubelet/kubeadm-flags.env
# This is a file that the user can use for overrides of the kubelet args as a last resort. Preferably, the user should use
# the .NodeRegistration.KubeletExtraArgs object in the configuration files instead. KUBELET_EXTRA_ARGS should be sourced from this file.
EnvironmentFile=-/etc/sysconfig/kubelet
ExecStart=
ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS
@pixiake 从你截图的日志来看,isulad收到了容器停止命令,发送15号信号给了容器,导致容器退出,你高亮的日志是容器退出的正常打印,需要你那边查看下k8s的日志,找到为什么会下发停止容器的命令。
[图片上传中…(image-TBkxy71MfCQFBsfqtEAA)]
[图片上传中…(image-pJh9u0lwryODB3i3Jlfo)]
问题找到了,部署v3.16+的calico会遇到集群挂掉的情况。
原因是v3.16+的calico的yaml中挂载了主机/sys/fs
这个目录,目前我把这部分注释掉之后calico可以正常运行了,会不会是这个挂载项跟isula有冲突?
# For eBPF mode, we need to be able to mount the BPF filesystem at /sys/fs/bpf so we mount in the parent directory.
- name: sysfs
mountPath: /sys/fs/
# Bidirectional means that, if we mount the BPF filesystem at /sys/fs/bpf it will propagate to the host.
# If the host is known to mount that filesystem already then Bidirectional can be omitted.
mountPropagation: Bidirectional
查看calico-node容器日志也只有容器启动错误的信息:
kubelet日志:
@pixiake 启动错误时的iSulad日志是否可以完整的贴下,你之前的截图来看,是iSulad启动容器成功,后面发送了stop命令停止了容器,和你后面贴的启动时错误对应不上
问题找到了,部署v3.16+的calico会遇到集群挂掉的情况。
原因是v3.16+的calico的yaml中挂载了主机/sys/fs
这个目录,目前我把这部分注释掉之后calico可以正常运行了,会不会是这个挂载项跟isula有冲突?# For eBPF mode, we need to be able to mount the BPF filesystem at /sys/fs/bpf so we mount in the parent directory. - name: sysfs mountPath: /sys/fs/ # Bidirectional means that, if we mount the BPF filesystem at /sys/fs/bpf it will propagate to the host. # If the host is known to mount that filesystem already then Bidirectional can be omitted. mountPropagation: Bidirectional
查看calico-node容器日志也只有容器启动错误的信息:
kubelet日志:
@pixiake calico的yaml中挂载了主机/sys/fs,你那边了解这个配置的作用是什么吗?k8s会怎么处理这个挂载项,是否会将主机的/sys/fs 重新挂载?如果重新挂载的话,启动容器时,会向/sys/fs/cgroup 目录写入配置,如果主机的/sys/fs被重新挂载,应该就找不到/sys/fs/cgroup目录,导致写入失败了
@pixiake 我们在本地已经复现了这个问题,正在进行修改
@pixiake 已经提交MR修改,PR, 问题原因为底层的容器runtime lxc不支持挂载sys/fs目录,已修正。另外提交了iSulad 的MR 修正了mount 配置顺序错误逻辑
!846:Mounts: only qsort the configed mounts
登录 后才可以发表评论