查看“︁使用阿里云镜像源快速搭建kubernetes(k8s) on debian10”︁的源代码

=2022=
<pre>
在集群初始化遇到问题，可以使用下面的命令进行清理后重新再初始化：

kubeadm reset
ifconfig cni0 down
ip link delete cni0
ifconfig flannel.1 down
ip link delete flannel.1
rm -rf /var/lib/cni/


root@debian11:~# apt-cache madison kubelet 
   kubelet |  1.24.2-00 | https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial/main amd64 Packages
   kubelet |  1.24.1-00 | https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial/main amd64 Packages
   kubelet |  1.24.0-00 | https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial/main amd64 Packages
   kubelet |  1.23.8-00 | https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial/main amd64 Packages
   kubelet |  1.23.7-00 | https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial/main amd64 Packages
   kubelet |  1.23.6-00 | https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial/main amd64 Packages
   kubelet |  1.23.5-00 | https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial/main amd64 Packages
   kubelet |  1.23.4-00 | https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial/main amd64 Packages
   kubelet |  1.23.3-00 | https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial/main amd64 Packages
   kubelet |  1.23.2-00 | https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial/main amd64 Packages
   kubelet |  1.23.1-00 | https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial/main amd64 Packages
   kubelet |  1.23.0-00 | h
</pre>

=思路2021 =

2020年的
[[Use aliyun mirrors Install and Configure Kubernetes (k8s) on debian10]]

现在是直接用aliyun提供的就行了 不用出外网 
先在围墙外的机器 pull下来 然后 push到自己的hub.docker 最后在内网的机器再pull 下来 再tag一下

参考一下 然后写成脚本吧 ubuntu 使用阿里云镜像源快速搭建kubernetes 1.15.2集群

初始化时 指定aliyun  mirrors  本来是指定 1。17。1版本的 我改了新的
   kubeadm init --apiserver-advertise-address=192.168.88.70 --image-repository registry.aliyuncs.com/google_containers --ignore-preflight-errors=all  --kubernetes-version v1.17.3 --service-cidr=10.96.0.0/16 --pod-network-cidr=10.244.0.0/16

=国内及一些快速安装的办法=

https://gitee.com/imlzw/Kubernetes-1.12.3-all-auto-install

[https://www.oschina.net/p/k3os k3OS 由容器软件提供商 Rancher Labs发布，这是业界首个专为 Kubernetes 而生的极轻量操作系统]

[https://www.oschina.net/p/k8e k8e 是轻量级、可扩展的企业级 Kubernetes 发行版，允许用户统一管理、保护和获得适用于企业环境的开箱即用 Kubernetes 集群]

== nginx-ingress-controller_install.sh ==
<pre>
 安装nginx-ingress-controller
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/mandatory.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/cloud-generic.yaml

</pre>

=[[Docker国内镜像的配置及使用]] pass=

=info=
 k8s-master 192.168.88.70
 k8-snode1 192.168.88.71
 k8-snode2 192.168.88.72
 
<pre>
cat >>/etc/hosts <<EOF
192.168.88.70  k8s-master
192.168.88.71  k8s-node1
192.168.88.72  k8s-node2
EOF


每台机器最少2GB内存，2CPUs。
集群中所有机器之间网络连接正常。
打开相应的端口，详见： [ Check required ports https://kubernetes.io/docs/setup/independent/install-kubeadm/#check-required-ports]


Kubernetes要求集群中所有机器具有不同的Mac地址、产品uuid、Hostname。可以使用如下命令查看：

# UUID
 cat /sys/class/dmi/id/product_uuid

# Mac地址
 ip link

Set Hostname and update hosts file

sudo hostnamectl set-hostname "k8s-master"
sudo hostnamectl set-hostname k8s-node1
sudo hostnamectl set-hostname k8s-node2

#Add the following lines in /etc/hosts file on all three systems,
</pre>
==时间==
[[Linux时间同步的那些事儿#debian]]

=安装Docker=
https://blog.csdn.net/shykevin/article/details/98811021
=安装kubelet，kubeadm，kubectl=
==基础信息==
<pre>
https://mirrors.aliyun.com/kubernetes/apt/dists/kubernetes-jessie/

 cat /etc/os-release 
PRETTY_NAME="Debian GNU/Linux 10 (buster)"
NAME="Debian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"


下一代 Debian 正式发行版的代号为 "buster" — 发布时间尚未确定
Debian 9（"stretch"） — 当前的稳定版
Debian 8（"jessie"） — 被淘汰的稳定版

kubectl version
Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.5",

</pre>

==添加apt key以及源（所有主机）==
 <pre>

cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
br_netfilter
EOF

cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
EOF

sudo sysctl -p /etc/sysctl.d/k8s.conf



apt update &&  apt install -y apt-transport-https curl

#听说 #kubernetes-xenial 不要改动
 

curl -s https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | apt-key add -
#curl -s https://mirrors.ustc.edu.cn/kubernetes/apt/dists/kubernetes-jessie/
#注意  不是追加
#echo "deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-stretch main" >/etc/apt/sources.list.d/kubernetes.list

#echo "deb https://mirrors.ustc.edu.cn/kubernetes/apt/dists/kubernetes-xenial/ main" >/etc/apt/sources.list.d/kubernetes.list
echo "deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main" >/etc/apt/sources.list.d/kubernetes.list


官方居然是这个 
echo "deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list

安装（所有主机）

apt-cache madison kubelet #查看版本

sudo apt update
apt install -y kubelet kubeadm kubectl
apt-mark hold kubelet kubeadm kubectl

#Setting up kubeadm (1.22.1-00)

#kubeadm is already the newest version (1.20.5-00).

 apt search  kubelet 
Sorting... Done
Full Text Search... Done
kubelet/kubernetes-xenial 1.24.2-00 amd64
  Kubernetes Node Agent



sudo apt install -y kubelet=1.15.2-00 kubeadm=1.15.2-00 kubectl=1.15.2-00
sudo apt-mark hold kubelet=1.15.2-00 kubeadm=1.15.2-00 kubectl=1.15.2-00</pre>

==安装kubernetes集群（仅master）==
<pre>
#2022 不指定版本了 
  kubeadm init --apiserver-advertise-address=192.168.88.70 --image-repository registry.aliyuncs.com/google_containers --ignore-preflight-errors=all  --service-cidr=10.96.0.0/16 --pod-network-cidr=192.168.0.0/16 #calico

  kubeadm init --apiserver-advertise-address=192.168.88.70 --image-repository registry.aliyuncs.com/google_containers --ignore-preflight-errors=all  --kubernetes-version v1.20.2 --service-cidr=10.96.0.0/16 --pod-network-cidr=10.244.0.0/16 #flannel


—image-repository 指定镜像源，指定为阿里云的源，这样就会避免在拉取镜像超时，如果没问题，过几分钟就能看到成功的日志输入

sudo kubeadm init --image-repository registry.aliyuncs.com/google_containers --kubernetes-version v1.15.2 --pod-network-cidr=192.169.0.0/16

[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'




</pre>
=== init 过程===
<pre>

s-master:~#  kubeadm init --apiserver-advertise-address=192.168.88.70 --image-repository registry.aliyuncs.com/google_containers --ignore-preflight-errors=all  --kubernetes-version v1.20.2 --service-cidr=10.96.0.0/16 --pod-network-cidr=10.244.0.0/16
[init] Using Kubernetes version: v1.20.2
[preflight] Running pre-flight checks
	[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
	[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.5. Latest validated version: 19.03
	[WARNING SystemVerification]: missing optional cgroups: hugetlb
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'



[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.88.70:6443 --token 8bemec.r64nlpkdmyijwrls \
    --discovery-token-ca-cert-hash sha256:a21b898539b970e9efe7490037c0c55710d5541ce1c967ff5b9060be845b2d33 

#2022  一定要指定版本 安装的是 v1.20.6-00 而init 时为 v1.20.6   

kubeadm init --apiserver-advertise-address="${ip}" --image-repository registry.aliyuncs.com/google_containers --kubernetes-version v1.20.6 --service-cidr=10.96.0.0/12 --pod-network-cidr=10.244.0.0/16


[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.10.148:6443 --token fvixji.55pd7qni1rmp1cjk \
    --discovery-token-ca-cert-hash sha256:d65ef89fab0fdb6bcb32cd3bd4cafe6e0955454cf961bf22cce28ddb94ae44de 
root@debian11:~# 



</pre>

=== pod网络安装 flannel===
可以下载或者直接复制出来 
 kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
https://docs.projectcalico.org/getting-started/kubernetes/flannel/flannel

https://docs.projectcalico.org/getting-started/kubernetes/quickstart

==node join==
<pre>
#在所有node上执行
kubeadm join 192.168.88.70:6443 --token 8bemec.r64nlpkdmyijwrls     --discovery-token-ca-cert-hash sha256:a21b898539b970e9efe7490037c0c55710d5541ce1c967ff5b9060be845b2d33 
</pre>

==常用命令==
<pre>

#等一下就全是Ready
 kubectl get nodes
NAME         STATUS   ROLES                  AGE    VERSION
k8s-master   Ready    control-plane,master   110m   v1.20.5
k8s-node1    Ready    <none>                 93m    v1.20.5
k8s-node2    Ready    <none>                 93m    v1.20.5



root@k8s-master:~# kubectl get pod --all-namespaces
NAMESPACE     NAME                                 READY   STATUS    RESTARTS   AGE
kube-system   coredns-7f89b7bc75-6fcdq             1/1     Running   0          4h23m
kube-system   coredns-7f89b7bc75-9n56h             1/1     Running   0          4h23m
kube-system   etcd-k8s-master                      1/1     Running   0          4h23m
kube-system   kube-apiserver-k8s-master            1/1     Running   0          4h23m
kube-system   kube-controller-manager-k8s-master   1/1     Running   0          4h23m
kube-system   kube-flannel-ds-9kcwd                1/1     Running   0          4h7m
kube-system   kube-flannel-ds-ql2hn                1/1     Running   0          4h9m
kube-system   kube-flannel-ds-vk2qb                1/1     Running   0          4h7m
kube-system   kube-proxy-cgkhf                     1/1     Running   0          4h23m
kube-system   kube-proxy-gcbsk                     1/1     Running   0          4h7m
kube-system   kube-proxy-qnc27                     1/1     Running   0          4h7m
kube-system   kube-scheduler-k8s-master            1/1     Running   0          4h23m
root@k8s-master:~# 

</pre>
=[[Kubernetes 1.20.x部署dashboard]]=

=集群测试 万年hello=
<pre>
root@k8s-master:~# cat hello.yml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: hello-world
spec:
  selector:
    matchLabels:
      run: load-balancer-example
  replicas: 2
  template:
    metadata:
      labels:
        run: load-balancer-example
    spec:
      containers:
        - name: hello-world
          image: registry.cn-hangzhou.aliyuncs.com/aliyun_google/google-sample-node-hello:1.0
          ports:
            - containerPort: 8080
              protocol: TCP


root@k8s-master:~# kubectl  apply  -f hello.yml 
deployment.apps/hello-world created
root@k8s-master:~# kubectl  get deployment 
NAME          READY   UP-TO-DATE   AVAILABLE   AGE
hello-world   0/2     2            0           13s
my-httpd      2/2     2            2           39h
my-nginx      2/2     2            2           45h
nginx         1/1     1            1           3d13h
root@k8s-master:~# kubectl expose deployment hello-world --type=NodePort --name=node-service
service/node-service exposed
root@k8s-master:~# kubectl  get service node-service 
NAME           TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
node-service   NodePort   10.96.231.107   <none>        8080:32355/TCP   48s
root@k8s-master:~# 



root@k8s-master:~# kubectl  describe service node-service 
Name:                     node-service
Namespace:                default
Labels:                   <none>
Annotations:              <none>
Selector:                 run=load-balancer-example
Type:                     NodePort
IP Families:              <none>
IP:                       10.96.231.107
IPs:                      10.96.231.107
Port:                     <unset>  8080/TCP
TargetPort:               8080/TCP
NodePort:                 <unset>  32355/TCP
Endpoints:                10.244.2.18:8080,10.244.2.19:8080
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   <none>


可以看到，服务通过30585 端口映射到容器的8080端口，这样我们就可以通过 32355端口访问容器内的web应用了，如下响应正常
</pre>

http://192.168.88.70:32355/
Hello Kubernetes!

=部署应用=
[[K8s使用deploy部署nginx]]

=站内资源=
[[Install and Configure Kubernetes (k8s) on ubuntu]] 

[[使用kubeadm离线部署kubernetesv1.9.0 on centos7]]

[[Proxmox 笔记#proxmox and k8s]]

=用 k3s 部署 k8s=
<pre>
用 k3s 部署 k8s 不过还是原生的感觉好 如果不只是学习
k3s 的好处就是配置十分简单：https://rancher.com/docs/k3s/latest/en/quick-start/。不需要装 docker，也不需要装 kubeadm。

    在第一个 node 上跑：curl -sfL https://get.k3s.io | sh -
    在第一个 node 上获取 token：cat /var/lib/rancher/k3s/server/node-token
    在其他 node 上跑：curl -sfL https://get.k3s.io | K3S_URL=https://myserver:6443 K3S_TOKEN=mynodetoken sh -

然后就搞定了。从第一个 node 的 /etc/rancher/k3s/k3s.yaml获取 kubectl 配置。
</pre>

=troubleshooting=
<pre>
不然init 有错 

kubeadm init shows kubelet isn't running or healthy

#2022  一定要指定版本 安装的是 v1.20.6-00 而init 时为 v1.20.6   
</pre>

=see also=

[https://blog.csdn.net/shykevin/article/details/98811021  ubuntu 使用阿里云镜像源快速搭建kubernetes 1.15.2集群]

[https://blog.csdn.net/telundusiji/article/details/114033799 kubernetes安装（国内网络+阿里云ecs）]

https://blog.csdn.net/u014636124/article/details/105145674

[https://cloud.tencent.com/developer/article/1652598 比Minikube更快，使用Kind快速创建K8S学习环境]

[http://mknight.cn/kubernetes%E6%9C%80%E6%96%B0%E9%83%A8%E7%BD%B2%E6%96%87%E6%A1%A3.html kubernetes最新部署文档 ]

[https://www.jianshu.com/p/651f4d12204f 构建kubernetes基础系统镜像-基于kubeadm安装kubernetes环境]

[https://blog.csdn.net/gedingbaod/article/details/107144648  使用阿里云ecs快速搭建Kubernetes(K8S)1.18]


[https://developer.aliyun.com/article/672675 15分钟在笔记本上搭建 Kubernetes + Istio开发环境]

[[category:k8s]] [[category:devops]] [[category: container]]