Evan：/* 2022 */

2022-07-07T08:00:02Z

2022

新页面

=2022=
<pre>
在集群初始化遇到问题，可以使用下面的命令进行清理后重新再初始化：

kubeadm reset
ifconfig cni0 down
ip link delete cni0
ifconfig flannel.1 down
ip link delete flannel.1
rm -rf /var/lib/cni/

root@debian11:~# apt-cache madison kubelet
kubelet | 1.24.2-00 | https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial/main amd64 Packages
kubelet | 1.24.1-00 | https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial/main amd64 Packages
kubelet | 1.24.0-00 | https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial/main amd64 Packages
kubelet | 1.23.8-00 | https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial/main amd64 Packages
kubelet | 1.23.7-00 | https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial/main amd64 Packages
kubelet | 1.23.6-00 | https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial/main amd64 Packages
kubelet | 1.23.5-00 | https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial/main amd64 Packages
kubelet | 1.23.4-00 | https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial/main amd64 Packages
kubelet | 1.23.3-00 | https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial/main amd64 Packages
kubelet | 1.23.2-00 | https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial/main amd64 Packages
kubelet | 1.23.1-00 | https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial/main amd64 Packages
kubelet | 1.23.0-00 | h
</pre>

=思路2021 =

2020年的
[[Use aliyun mirrors Install and Configure Kubernetes (k8s) on debian10]]

现在是直接用aliyun提供的就行了不用出外网
先在围墙外的机器 pull下来然后 push到自己的hub.docker 最后在内网的机器再pull 下来再tag一下

参考一下然后写成脚本吧 ubuntu 使用阿里云镜像源快速搭建kubernetes 1.15.2集群

初始化时指定aliyun mirrors 本来是指定 1。17。1版本的我改了新的
kubeadm init --apiserver-advertise-address=192.168.88.70 --image-repository registry.aliyuncs.com/google_containers --ignore-preflight-errors=all --kubernetes-version v1.17.3 --service-cidr=10.96.0.0/16 --pod-network-cidr=10.244.0.0/16

=国内及一些快速安装的办法=

https://gitee.com/imlzw/Kubernetes-1.12.3-all-auto-install

[https://www.oschina.net/p/k3os k3OS 由容器软件提供商 Rancher Labs发布，这是业界首个专为 Kubernetes 而生的极轻量操作系统]

[https://www.oschina.net/p/k8e k8e 是轻量级、可扩展的企业级 Kubernetes 发行版，允许用户统一管理、保护和获得适用于企业环境的开箱即用 Kubernetes 集群]

== nginx-ingress-controller_install.sh ==
<pre>
安装nginx-ingress-controller
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/mandatory.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/cloud-generic.yaml

</pre>

=[[Docker国内镜像的配置及使用]] pass=

=info=
k8s-master 192.168.88.70
k8-snode1 192.168.88.71
k8-snode2 192.168.88.72

<pre>
cat >>/etc/hosts <<EOF
192.168.88.70 k8s-master
192.168.88.71 k8s-node1
192.168.88.72 k8s-node2
EOF

每台机器最少2GB内存，2CPUs。
集群中所有机器之间网络连接正常。
打开相应的端口，详见： [ Check required ports https://kubernetes.io/docs/setup/independent/install-kubeadm/#check-required-ports]

Kubernetes要求集群中所有机器具有不同的Mac地址、产品uuid、Hostname。可以使用如下命令查看：

# UUID
cat /sys/class/dmi/id/product_uuid

# Mac地址
ip link

Set Hostname and update hosts file

sudo hostnamectl set-hostname "k8s-master"
sudo hostnamectl set-hostname k8s-node1
sudo hostnamectl set-hostname k8s-node2

#Add the following lines in /etc/hosts file on all three systems,
</pre>
==时间==
[[Linux时间同步的那些事儿#debian]]

=安装Docker=
https://blog.csdn.net/shykevin/article/details/98811021
=安装kubelet，kubeadm，kubectl=
==基础信息==
<pre>
https://mirrors.aliyun.com/kubernetes/apt/dists/kubernetes-jessie/

cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 10 (buster)"
NAME="Debian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"

下一代 Debian 正式发行版的代号为 "buster" — 发布时间尚未确定
Debian 9（"stretch"） — 当前的稳定版
Debian 8（"jessie"） — 被淘汰的稳定版

kubectl version
Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.5",

</pre>

==添加apt key以及源（所有主机）==
<pre>

cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
br_netfilter
EOF

cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
EOF

sudo sysctl -p /etc/sysctl.d/k8s.conf

apt update && apt install -y apt-transport-https curl

#听说 #kubernetes-xenial 不要改动

curl -s https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | apt-key add -
#curl -s https://mirrors.ustc.edu.cn/kubernetes/apt/dists/kubernetes-jessie/
#注意不是追加
#echo "deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-stretch main" >/etc/apt/sources.list.d/kubernetes.list

#echo "deb https://mirrors.ustc.edu.cn/kubernetes/apt/dists/kubernetes-xenial/ main" >/etc/apt/sources.list.d/kubernetes.list
echo "deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main" >/etc/apt/sources.list.d/kubernetes.list

官方居然是这个
echo "deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list

安装（所有主机）

apt-cache madison kubelet #查看版本

sudo apt update
apt install -y kubelet kubeadm kubectl
apt-mark hold kubelet kubeadm kubectl

#Setting up kubeadm (1.22.1-00)

#kubeadm is already the newest version (1.20.5-00).

apt search kubelet
Sorting... Done
Full Text Search... Done
kubelet/kubernetes-xenial 1.24.2-00 amd64
Kubernetes Node Agent

sudo apt install -y kubelet=1.15.2-00 kubeadm=1.15.2-00 kubectl=1.15.2-00
sudo apt-mark hold kubelet=1.15.2-00 kubeadm=1.15.2-00 kubectl=1.15.2-00</pre>

==安装kubernetes集群（仅master）==
<pre>
#2022 不指定版本了
kubeadm init --apiserver-advertise-address=192.168.88.70 --image-repository registry.aliyuncs.com/google_containers --ignore-preflight-errors=all --service-cidr=10.96.0.0/16 --pod-network-cidr=192.168.0.0/16 #calico

kubeadm init --apiserver-advertise-address=192.168.88.70 --image-repository registry.aliyuncs.com/google_containers --ignore-preflight-errors=all --kubernetes-version v1.20.2 --service-cidr=10.96.0.0/16 --pod-network-cidr=10.244.0.0/16 #flannel

—image-repository 指定镜像源，指定为阿里云的源，这样就会避免在拉取镜像超时，如果没问题，过几分钟就能看到成功的日志输入

sudo kubeadm init --image-repository registry.aliyuncs.com/google_containers --kubernetes-version v1.15.2 --pod-network-cidr=192.169.0.0/16

[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'

</pre>
=== init 过程===
<pre>

s-master:~# kubeadm init --apiserver-advertise-address=192.168.88.70 --image-repository registry.aliyuncs.com/google_containers --ignore-preflight-errors=all --kubernetes-version v1.20.2 --service-cidr=10.96.0.0/16 --pod-network-cidr=10.244.0.0/16
[init] Using Kubernetes version: v1.20.2
[preflight] Running pre-flight checks
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.5. Latest validated version: 19.03
[WARNING SystemVerification]: missing optional cgroups: hugetlb
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'

[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.88.70:6443 --token 8bemec.r64nlpkdmyijwrls \
--discovery-token-ca-cert-hash sha256:a21b898539b970e9efe7490037c0c55710d5541ce1c967ff5b9060be845b2d33

#2022 一定要指定版本安装的是 v1.20.6-00 而init 时为 v1.20.6

kubeadm init --apiserver-advertise-address="${ip}" --image-repository registry.aliyuncs.com/google_containers --kubernetes-version v1.20.6 --service-cidr=10.96.0.0/12 --pod-network-cidr=10.244.0.0/16

[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.10.148:6443 --token fvixji.55pd7qni1rmp1cjk \
--discovery-token-ca-cert-hash sha256:d65ef89fab0fdb6bcb32cd3bd4cafe6e0955454cf961bf22cce28ddb94ae44de
root@debian11:~#

</pre>

=== pod网络安装 flannel===
可以下载或者直接复制出来
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
https://docs.projectcalico.org/getting-started/kubernetes/flannel/flannel

https://docs.projectcalico.org/getting-started/kubernetes/quickstart

==node join==
<pre>
#在所有node上执行
kubeadm join 192.168.88.70:6443 --token 8bemec.r64nlpkdmyijwrls --discovery-token-ca-cert-hash sha256:a21b898539b970e9efe7490037c0c55710d5541ce1c967ff5b9060be845b2d33
</pre>

==常用命令==
<pre>

#等一下就全是Ready
kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master Ready control-plane,master 110m v1.20.5
k8s-node1 Ready <none> 93m v1.20.5
k8s-node2 Ready <none> 93m v1.20.5

root@k8s-master:~# kubectl get pod --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system coredns-7f89b7bc75-6fcdq 1/1 Running 0 4h23m
kube-system coredns-7f89b7bc75-9n56h 1/1 Running 0 4h23m
kube-system etcd-k8s-master 1/1 Running 0 4h23m
kube-system kube-apiserver-k8s-master 1/1 Running 0 4h23m
kube-system kube-controller-manager-k8s-master 1/1 Running 0 4h23m
kube-system kube-flannel-ds-9kcwd 1/1 Running 0 4h7m
kube-system kube-flannel-ds-ql2hn 1/1 Running 0 4h9m
kube-system kube-flannel-ds-vk2qb 1/1 Running 0 4h7m
kube-system kube-proxy-cgkhf 1/1 Running 0 4h23m
kube-system kube-proxy-gcbsk 1/1 Running 0 4h7m
kube-system kube-proxy-qnc27 1/1 Running 0 4h7m
kube-system kube-scheduler-k8s-master 1/1 Running 0 4h23m
root@k8s-master:~#

</pre>
=[[Kubernetes 1.20.x部署dashboard]]=

=集群测试万年hello=
<pre>
root@k8s-master:~# cat hello.yml
apiVersion: apps/v1
kind: Deployment
metadata:
name: hello-world
spec:
selector:
matchLabels:
run: load-balancer-example
replicas: 2
template:
metadata:
labels:
run: load-balancer-example
spec:
containers:
- name: hello-world
image: registry.cn-hangzhou.aliyuncs.com/aliyun_google/google-sample-node-hello:1.0
ports:
- containerPort: 8080
protocol: TCP

root@k8s-master:~# kubectl apply -f hello.yml
deployment.apps/hello-world created
root@k8s-master:~# kubectl get deployment
NAME READY UP-TO-DATE AVAILABLE AGE
hello-world 0/2 2 0 13s
my-httpd 2/2 2 2 39h
my-nginx 2/2 2 2 45h
nginx 1/1 1 1 3d13h
root@k8s-master:~# kubectl expose deployment hello-world --type=NodePort --name=node-service
service/node-service exposed
root@k8s-master:~# kubectl get service node-service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
node-service NodePort 10.96.231.107 <none> 8080:32355/TCP 48s
root@k8s-master:~#

root@k8s-master:~# kubectl describe service node-service
Name: node-service
Namespace: default
Labels: <none>
Annotations: <none>
Selector: run=load-balancer-example
Type: NodePort
IP Families: <none>
IP: 10.96.231.107
IPs: 10.96.231.107
Port: <unset> 8080/TCP
TargetPort: 8080/TCP
NodePort: <unset> 32355/TCP
Endpoints: 10.244.2.18:8080,10.244.2.19:8080
Session Affinity: None
External Traffic Policy: Cluster
Events: <none>

可以看到，服务通过30585 端口映射到容器的8080端口，这样我们就可以通过 32355端口访问容器内的web应用了，如下响应正常
</pre>

http://192.168.88.70:32355/
Hello Kubernetes!

=部署应用=
[[K8s使用deploy部署nginx]]

=站内资源=
[[Install and Configure Kubernetes (k8s) on ubuntu]]

[[使用kubeadm离线部署kubernetesv1.9.0 on centos7]]

[[Proxmox 笔记#proxmox and k8s]]

=用 k3s 部署 k8s=
<pre>
用 k3s 部署 k8s 不过还是原生的感觉好如果不只是学习
k3s 的好处就是配置十分简单：https://rancher.com/docs/k3s/latest/en/quick-start/。不需要装 docker，也不需要装 kubeadm。

在第一个 node 上跑：curl -sfL https://get.k3s.io | sh -
在第一个 node 上获取 token：cat /var/lib/rancher/k3s/server/node-token
在其他 node 上跑：curl -sfL https://get.k3s.io | K3S_URL=https://myserver:6443 K3S_TOKEN=mynodetoken sh -

然后就搞定了。从第一个 node 的 /etc/rancher/k3s/k3s.yaml获取 kubectl 配置。
</pre>

=troubleshooting=
<pre>
不然init 有错

kubeadm init shows kubelet isn't running or healthy

#2022 一定要指定版本安装的是 v1.20.6-00 而init 时为 v1.20.6
</pre>

=see also=

[https://blog.csdn.net/shykevin/article/details/98811021 ubuntu 使用阿里云镜像源快速搭建kubernetes 1.15.2集群]

[https://blog.csdn.net/telundusiji/article/details/114033799 kubernetes安装（国内网络+阿里云ecs）]

https://blog.csdn.net/u014636124/article/details/105145674

[https://cloud.tencent.com/developer/article/1652598 比Minikube更快，使用Kind快速创建K8S学习环境]

[http://mknight.cn/kubernetes%E6%9C%80%E6%96%B0%E9%83%A8%E7%BD%B2%E6%96%87%E6%A1%A3.html kubernetes最新部署文档 ]

[https://www.jianshu.com/p/651f4d12204f 构建kubernetes基础系统镜像-基于kubeadm安装kubernetes环境]

[https://blog.csdn.net/gedingbaod/article/details/107144648 使用阿里云ecs快速搭建Kubernetes(K8S)1.18]

[https://developer.aliyun.com/article/672675 15分钟在笔记本上搭建 Kubernetes + Istio开发环境]

[[category:k8s]] [[category:devops]] [[category: container]]

使用阿里云镜像源快速搭建kubernetes(k8s) on debian10 - 版本历史

Evan：​/* 2022 */

Evan：/* 2022 */