我的k8s随笔:k3s部署

k3s集群部署

资源:
k3s(待移动):
https://github.com/rancher/k3s
https://www.kubernetes.org.cn/5180.html
https://docs.kubeedge.io/en/latest/setup/installer_setup.html
http://www.sohu.com/a/333788182_764649
http://dockone.io/article/8662
https://yq.aliyun.com/articles/704089
https://blog.csdn.net/huangruifeng/article/details/95541041

环境:

2机器。ubuntu系统。内存至少1GB(注:k3s服务器启动后,占用400多MB内存)。
下载 https://github.com/rancher/k3s/blob/master/install.sh 到每台机器的~/k3s目录。此为工作目录。
如果用install.sh安装太慢,先下载k3s到/usr/local/bin目录(脚本肝自动判断,如已下载,则不会重新下载)。
1.0地址:
https://github.com/rancher/k3s/releases/download/v1.0.0/k3s

主:
sudo sh -c “curl -sfL https://get.k3s.io | sh -“
查看token:
sudo cat /var/lib/rancher/k3s/server/node-token
内容:
K10df6003a637f58cc1717e08a4276815a247adcd076aa3994d6a0e6a83e0c3628e::server:766d6bd4c264659b88cd64efbd2fc2f9

节点:
sudo wget https://github.com/rancher/k3s/releases/download/v1.0.0/k3s -O /usr/local/bin/k3s
sudo chmod +x /usr/local/bin/k3s

将节加入主:

k3s agent –server https://192.168.28.131:6443 –token K10df6003a637f58cc1717e08a4276815a247adcd076aa3994d6a0e6a83e0c3628e::server:766d6bd4c264659b88cd64efbd2fc2f9 –docker

INFO[2019-12-10T13:13:14.147525203+08:00] Starting k3s agent v1.0.0 (18bd921c)
INFO[2019-12-10T13:13:14.148281444+08:00] module overlay was already loaded
INFO[2019-12-10T13:13:14.148675126+08:00] module nf_conntrack was already loaded
INFO[2019-12-10T13:13:14.149277705+08:00] module br_netfilter was already loaded
INFO[2019-12-10T13:13:14.150247709+08:00] Running load balancer 127.0.0.1:35071 -> [192.168.28.131:6443]
ERRO[2019-12-10T13:13:14.344265519+08:00] failed to get CA certs at https://127.0.0.1:35071/cacerts: Get https://127.0.0.1:35071/cacerts: read tcp 127.0.0.1:37276->127.0.0.1:35071: read: bad address
ERRO[2019-12-10T13:13:16.381172067+08:00] token is not valid: Get https://127.0.0.1:35071/apis: read tcp 127.0.0.1:37292->127.0.0.1:35071: read: bad address
ERRO[2019-12-10T13:13:18.442983958+08:00] server https://127.0.0.1:35071/cacerts is not trusted: Get https://127.0.0.1:35071/cacerts: read tcp 127.0.0.1:37312->127.0.0.1:35071: read: bad address

主提示:
Dec 10 13:18:57 ubuntu k3s[24178]: I1210 13:18:57.772921 24178 log.go:172] http: TLS handshake error from 192.168.28.129:53010: remote error: tls: bad certificate
Dec 10 13:18:59 ubuntu k3s[24178]: I1210 13:18:59.810203 24178 log.go:172] http: TLS handshake error from 192.168.28.129:53026: remote error: tls: bad certificate
Dec 10 13:18:59 ubuntu k3s[24178]: I1210 13:18:59.821164 24178 log.go:172] http: TLS handshake error from 192.168.28.129:53030: EOF

失败

再次在节点上加入:
K3S_TOKEN=K10df6003a637f58cc1717e08a4276815a247adcd076aa3994d6a0e6a83e0c3628e::server:766d6bd4c264659b88cd64efbd2fc2f9 K3S_URL=https://192.168.28.131:6443 ./install.sh

启动失败,修改/etc/systemd/system/k3s-agent.service最后一行:
ExecStart=/usr/local/bin/k3s agent –docker

用 systemctl status k3s-agent.service 查看状态:
Dec 10 13:28:19 ubuntu k3s[2269]: time=”2019-12-10T13:28:19.582424149+08:00” level=error msg=”token is not valid: Get https://127.0.0.1:35157/apis: read tcp 127.0.0.1:
Dec 10 13:28:24 ubuntu k3s[2269]: time=”2019-12-10T13:28:24.605363075+08:00” level=error msg=”server https://127.0.0.1:35157/cacerts is not trusted: Get https://127.0.
Dec 10 13:28:29 ubuntu k3s[2269]: time=”2019-12-10T13:28:29.759107294+08:00” level=error msg=”Get https://127.0.0.1:35157/v1-k3s/serving-kubelet.crt: read tcp 127.0.0.
Dec 10 13:28:34 ubuntu k3s[2269]: time=”2019-12-10T13:28:34.780909274+08:00” level=error msg=”server https://127.0.0.1:35157/cacerts is not trusted: Get https://127.0.

失败。与前面结果一样。

安装ca
sudo apt-get install -y ca-certificates

/usr/local/bin/k3s-uninstall.sh

/usr/local/bin/k3s-agent-uninstall.sh

在主查看状态:
k3s kubectl get node -o wide

参考:https://qiita.com/msquare33/items/e472c913a8b401649c60

route add default gw 192.168.56.1

出错记录
INFO[2019-12-09T13:52:44.202956655Z] Waiting for containerd startup: rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection error: desc = “transport: Error while dialing dial unix /run/k3s/containerd/containerd.sock: connect: connection refused”

原因是没有启动containerd,如果使用docker的话,在agent后添加–docker参数。

k3s服务启动后,监听的端口:
netstat -ntp | grep k3s
tcp 0 0 192.168.28.131:6444 0.0.0.0: LISTEN 24178/k3s
tcp6 0 0 :::6443 :::
LISTEN 24178/k3s

服务:
/etc/systemd/system/k3s.service

[Unit]
Description=Lightweight Kubernetes
Documentation=https://k3s.io
Wants=network-online.target
[Install]
WantedBy=multi-user.target
[Service]
Type=notify
EnvironmentFile=/etc/systemd/system/k3s.service.env
KillMode=process
Delegate=yes
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
TasksMax=infinity
TimeoutStartSec=0
Restart=always
RestartSec=5s
ExecStartPre=-/sbin/modprobe br_netfilter
ExecStartPre=-/sbin/modprobe overlay
ExecStart=/usr/local/bin/k3s \
server \

改:
ExecStart=/usr/local/bin/k3s server –docker –no-deploy traefik

systemctl daemon-reload
systemctl restart k3s.service

install.sh分析:
下载压缩包,检查sha256sum。创建服务文件(/etc/systemd/system/k3s.service),启动服务
注:使用脚本生成的服务文件,最后多了 \,非法。提示
Failed to restart k3s.service: Unit k3s.service is not loaded properly: Invalid argument.

–docker –no-deploy traefik

armhf上测试

主:
sudo sh -c “curl -sfL https://get.k3s.io | sh -“
查看token:
sudo cat /var/lib/rancher/k3s/server/node-token

节点:
sudo wget https://github.com/rancher/k3s/releases/download/v1.0.0/k3s-armhf -O /usr/local/bin/k3s
sudo chmod +x /usr/local/bin/k3s