在线午夜精品自拍小视频_无码av无码专区线_亚洲无码精品人妻_人人澡欧美一区

關(guān)于K8s中Pod調(diào)度[選擇器,指定節(jié)點(diǎn),主機(jī)親和性]方式和節(jié)點(diǎn)[coedon,drain,taint]標(biāo)記的Demo

寫在前面
嗯，整理K8s中pod調(diào)度相關(guān)筆記，這里分享給小伙伴
博文內(nèi)容涉及：
kube-scheduler組件的簡(jiǎn)述
Pod的調(diào)度(選擇器、指定節(jié)點(diǎn)、主機(jī)親和性)方式
節(jié)點(diǎn)的coedon與drain標(biāo)記
節(jié)點(diǎn)的taint(污點(diǎn))標(biāo)記及pod的容忍污點(diǎn)(tolerations)定義
食用方式：
需要了解K8s基礎(chǔ)知識(shí)
熟悉資源對(duì)象pod，deploy的創(chuàng)建，了解資源對(duì)象定義yaml文件
了解kubectl常用命令
理解不足小伙伴幫忙指正
「 所謂成功就是用自己的方式度過人生。--------《明朝那些事》」

Scheduler 調(diào)度組件簡(jiǎn)述
Kubernetes Scheduler是什么
眾多周知，Kubernetes Scheduler 是 Kubernetes中負(fù)責(zé)Pod調(diào)度的重要功能模塊，運(yùn)行在k8s 集群中的master節(jié)點(diǎn)。

「作用」 : Kubernetes Scheduler的作用是將待調(diào)度的Pod (API新創(chuàng)建的Pod, Controller Manager為補(bǔ)足副本而創(chuàng)建的Pod等)按照特定的調(diào)度算法和調(diào)度策略綁定(Binding)到集群中某個(gè)合適的Node上,并將綁定信息寫入etcd中。

在整個(gè)調(diào)度過程中涉及三個(gè)對(duì)象,分別是

待調(diào)度Pod列表
可用Node列表
以及調(diào)度算法和策略
「整體流程」 :通過調(diào)度算法調(diào)度,為待調(diào)度Pod列表中的每個(gè)Pod從Node列表中選擇一個(gè)最適合的Node隨后, 目標(biāo)node節(jié)點(diǎn)上的kubelet通過APIServer監(jiān)聽到Kubernetes Scheduler產(chǎn)生的Pod綁定事件,然后獲取對(duì)應(yīng)的Pod清單,下載Image鏡像并啟動(dòng)容器。



kubelet進(jìn)程通過與API Server的交互，每隔一個(gè)時(shí)間周期,就會(huì)調(diào)用一次API Server的REST接口報(bào)告自身狀態(tài), API Server接收到這些信息后,將節(jié)點(diǎn)狀態(tài)信息更新到etcd中。同時(shí)kubelet也通過API Server的Watch接口監(jiān)聽Pod信息,

如果監(jiān)聽到新的Pod副本被調(diào)度綁定到本節(jié)點(diǎn),則執(zhí)行Pod對(duì)應(yīng)的容器的創(chuàng)建和啟動(dòng)邏輯;
如果監(jiān)聽到Pod對(duì)象被刪除,則刪除本節(jié)點(diǎn)上的相應(yīng)的Pod容器;
如果監(jiān)聽到修改Pod信息,則kubelet監(jiān)聽到變化后,會(huì)相應(yīng)地修改本節(jié)點(diǎn)的Pod容器。
所以說，kubernetes Schedule 在整個(gè)系統(tǒng)中承擔(dān)了承上啟下的重要功能，對(duì)上負(fù)責(zé)接收聲明式API或者控制器創(chuàng)建新pod的消息，并且為其安排一個(gè)合適的Node，對(duì)下，選擇好node之后，把工作交接給node上的kubelet，由kubectl負(fù)責(zé)pod的剩余生命周期。

Kubernetes Scheduler調(diào)度流程
Kubernetes Scheduler當(dāng)前提供的默認(rèn)調(diào)度流程分為以下兩步。這部分隨版本一直變化，小伙伴以官網(wǎng)為主

流程    描述
「預(yù)選調(diào)度過程」    即遍歷所有目標(biāo)Node,篩選出符合要求的候選節(jié)點(diǎn)。為此, Kubernetes內(nèi)置了多種預(yù)選策略(xxx Predicates)供用戶選擇
「確定最優(yōu)節(jié)點(diǎn)」    在第1步的基礎(chǔ)上,采用優(yōu)選策略(xxxPriority)計(jì)算出每個(gè)候選節(jié)點(diǎn)的積分,積分最高者勝出
Kubernetes Scheduler的調(diào)度流程是通過插件方式加載的"調(diào)度算法提供者” (AlgorithmProvider)具體實(shí)現(xiàn)的。一個(gè)AlgorithmProvider其實(shí)就是包括了一組預(yù)選策略與一組優(yōu)先選擇策略的結(jié)構(gòu)體.

Scheduler中可用的預(yù)選策略包含：NoDiskConflict、PodFitsResources、PodSelectorMatches、PodFitsHost、CheckNodeLabelPresence、CheckServiceAffinity 和PodFitsPorts策略等。

其默認(rèn)的AlgorithmProvider加載的預(yù)選策略返回布爾值包括：

PodFitsPorts（PodFitsPorts）：判斷端口是否沖突
PodFitsResources（PodFitsResources）：判斷備選節(jié)點(diǎn)的資源是否滿足備選Pod的需求
NoDiskConflict（NoDiskConflict）：判斷備選節(jié)點(diǎn)和已有節(jié)點(diǎn)是否磁盤沖突
MatchNodeSelector（PodSelectorMatches）：判斷備選節(jié)點(diǎn)是否包含備選Pod的標(biāo)簽選擇器指定的標(biāo)簽。
HostName（PodFitsHost）:判斷備選Pod的spec.nodeName域所指定的節(jié)點(diǎn)名稱和備選節(jié)點(diǎn)的名稱是否一致
即每個(gè)節(jié)點(diǎn)只有通過前面提及的5個(gè)默認(rèn)預(yù)選策略后，才能初步被選中，進(jìn)入到確認(rèn)最優(yōu)節(jié)點(diǎn)(優(yōu)選策略)流程。

Scheduler中的優(yōu)選策略包含(不限于下面3個(gè))：

LeastRequestedPriority:從備選節(jié)點(diǎn)列表中選出資源消耗最小的節(jié)點(diǎn),對(duì)各個(gè)節(jié)點(diǎn)公式打分
CalculateNodeLabelPriority：判斷策略列出的標(biāo)簽在備選節(jié)點(diǎn)中存在時(shí)，是否選擇該備選節(jié),這不太懂，打分
BalancedResourceAllocation：從備選節(jié)點(diǎn)列表中選出各項(xiàng)資源使用率最均衡的節(jié)點(diǎn)。對(duì)各個(gè)節(jié)點(diǎn)公式打分
每個(gè)節(jié)點(diǎn)通過優(yōu)先選擇策略時(shí)都會(huì)算出一個(gè)得分，計(jì)算各項(xiàng)得分，最終選出得分值最大的節(jié)點(diǎn)作為優(yōu)選的結(jié)果（也是調(diào)度算法的結(jié)果）。

Pod的調(diào)度
手動(dòng)指定pod的運(yùn)行位置
:
通過給node節(jié)點(diǎn)設(shè)置指定的標(biāo)簽，然后我們可以在創(chuàng)建pod里指定通過選擇器選擇node標(biāo)簽，類似前端里面DOM操作元素定位，或者直接指定節(jié)點(diǎn)名

節(jié)點(diǎn)標(biāo)簽常用命令

標(biāo)簽設(shè)置    --
查看    kubectl get nodes --show-labels
設(shè)置    kubectl label node node2 disktype=ssd
取消    kubectl label node node2 disktype
所有節(jié)點(diǎn)設(shè)置    kubectl  label node all key=vale
給節(jié)點(diǎn)設(shè)置標(biāo)簽

┌──[root@vms81.liruilongs.github.io]-[~/ansible]
└─$kubectl label node vms82.liruilongs.github.io disktype=node1
node/vms82.liruilongs.github.io labeled
┌──[root@vms81.liruilongs.github.io]-[~/ansible]
└─$kubectl label node vms83.liruilongs.github.io disktype=node2
node/vms83.liruilongs.github.io labeled
查看節(jié)點(diǎn)標(biāo)簽

┌──[root@vms81.liruilongs.github.io]-[~/ansible]
└─$kubectl get node --show-labels
NAME                         STATUS   ROLES                  AGE   VERSION   LABELS
vms81.liruilongs.github.io   Ready    control-plane,master   45d   v1.22.2   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=vms81.liruilongs.github.io,kubernetes.io/os=linux,node-role.kubernetes.io/control-plane=,node-role.kubernetes.io/master=,node.kubernetes.io/exclude-from-external-load-balancers=
vms82.liruilongs.github.io   Ready    <none>                 45d   v1.22.2   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,disktype=node1,kubernetes.io/arch=amd64,kubernetes.io/hostname=vms82.liruilongs.github.io,kubernetes.io/os=linux
vms83.liruilongs.github.io   Ready    <none>                 45d   v1.22.2   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,disktype=node2,kubernetes.io/arch=amd64,kubernetes.io/hostname=vms83.liruilongs.github.io,kubernetes.io/os=linux
┌──[root@vms81.liruilongs.github.io]-[~/ansible]
└─$
「特殊的內(nèi)置標(biāo)簽node-role.kubernetes.io/control-plane=,node-role.kubernetes.io/master=,用于設(shè)置角色列roles」

┌──[root@vms81.liruilongs.github.io]-[~/ansible]
└─$kubectl get node
NAME                         STATUS   ROLES                  AGE   VERSION
vms81.liruilongs.github.io   Ready    control-plane,master   45d   v1.22.2
vms82.liruilongs.github.io   Ready    <none>                 45d   v1.22.2
vms83.liruilongs.github.io   Ready    <none>                 45d   v1.22.2
「我們也可以做worker節(jié)點(diǎn)上設(shè)置標(biāo)簽」

┌──[root@vms81.liruilongs.github.io]-[~/ansible]
└─$kubectl label nodes vms82.liruilongs.github.io node-role.kubernetes.io/worker1=
node/vms82.liruilongs.github.io labeled
┌──[root@vms81.liruilongs.github.io]-[~/ansible]
└─$kubectl label nodes vms83.liruilongs.github.io node-role.kubernetes.io/worker2=
node/vms83.liruilongs.github.io labeled
查看標(biāo)簽

┌──[root@vms81.liruilongs.github.io]-[~/ansible]
└─$kubectl get node
NAME                         STATUS   ROLES                  AGE   VERSION
vms81.liruilongs.github.io   Ready    control-plane,master   45d   v1.22.2
vms82.liruilongs.github.io   Ready    worker1                45d   v1.22.2
vms83.liruilongs.github.io   Ready    worker2                45d   v1.22.2
┌──[root@vms81.liruilongs.github.io]-[~/ansible]
└─$
選擇器(nodeSelector)方式
「在特定節(jié)點(diǎn)上運(yùn)行pod,給vms83.liruilongs.github.io節(jié)點(diǎn)打disktype=node2的標(biāo)簽，yaml文件選擇器指定對(duì)應(yīng)的標(biāo)簽」

┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl get nodes -l disktype=node2
NAME                         STATUS   ROLES     AGE   VERSION
vms83.liruilongs.github.io   Ready    worker2   45d   v1.22.2
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$vim pod-node2.yaml
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl apply  -f pod-node2.yaml
pod/podnode2 created
「pod-node2.yaml」

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: podnode2
  name: podnode2
spec:
  nodeSelector:
    disktype: node2
  containers:
  - image: nginx
    imagePullPolicy: IfNotPresent
    name: podnode2
    resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}
pod被調(diào)度到了vms83.liruilongs.github.io

┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl get pods -owide
NAME       READY   STATUS    RESTARTS   AGE   IP               NODE                         NOMINATED NODE   READINESS GATES
podnode2   1/1     Running   0          13m   10.244.70.60     vms83.liruilongs.github.io   <none>           <none>
指定節(jié)點(diǎn)名稱(nodeName)的方式
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$vim pod-node1.yaml
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl apply -f pod-node1.yaml
pod/podnode1 created
「pod-node1.yaml」

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: podnode1
  name: podnode1
spec:
  nodeName: vms82.liruilongs.github.io
  containers:
  - image: nginx
    imagePullPolicy: IfNotPresent
    name: podnode1
    resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}
需要注意 「當(dāng)pod資源文件指定的節(jié)點(diǎn)標(biāo)簽,或者節(jié)點(diǎn)名不存在時(shí)，這個(gè)pod資源是無法創(chuàng)建成功的」

┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl get pods -owide
NAME       READY   STATUS    RESTARTS   AGE   IP               NODE                         NOMINATED NODE   READINESS GATES
podnode1   1/1     Running   0          36s   10.244.171.165   vms82.liruilongs.github.io   <none>           <none>
podnode2   1/1     Running   0          13m   10.244.70.60     vms83.liruilongs.github.io   <none>           <none>

主機(jī)親和性
所謂主機(jī)親和性，即在滿足指定條件的節(jié)點(diǎn)上運(yùn)行。分為硬策略(必須滿足)，軟策略(最好滿足)

硬策略(requiredDuringSchedulingIgnoredDuringExecution)
所調(diào)度節(jié)點(diǎn)的標(biāo)簽必須為其中的一個(gè)，這個(gè)標(biāo)簽是一個(gè)默認(rèn)標(biāo)簽，會(huì)自動(dòng)添加

  affinity:
    nodeAffinity: #主機(jī)親和性
      requiredDuringSchedulingIgnoredDuringExecution: #硬策略
        nodeSelectorTerms:
        - matchExpressions:
          - key: kubernetes.io/hostname
            operator: In
            values:
            - vms85.liruilongs.github.io
            - vms84.liruilongs.github.io
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl apply -f pod-node-a.yaml
pod/podnodea created
「pod-node-a.yaml」

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: podnodea
  name: podnodea
spec:
  containers:
  - image: nginx
    imagePullPolicy: IfNotPresent
    name: podnodea
    resources: {}
  affinity:
    nodeAffinity: #主機(jī)親和性
      requiredDuringSchedulingIgnoredDuringExecution: #硬策略
        nodeSelectorTerms:
        - matchExpressions:
          - key: kubernetes.io/hostname
            operator: In
            values:
            - vms85.liruilongs.github.io
            - vms84.liruilongs.github.io
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}
「條件不滿足，所以 Pending」

┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl get pods
NAME       READY   STATUS    RESTARTS   AGE
podnodea   0/1     Pending   0          8s
「我修改一下，修改為存在的標(biāo)簽」

┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$sed -i  's/vms84.liruilongs.github.io/vms83.liruilongs.github.io/' pod-node-a.yaml
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl apply -f pod-node-a.yaml
pod/podnodea created
「可以發(fā)現(xiàn)pod調(diào)度成功」

┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl get pods -owide
NAME       READY   STATUS    RESTARTS   AGE   IP             NODE                         NOMINATED NODE   READINESS GATES
podnodea   1/1     Running   0          13s   10.244.70.61   vms83.liruilongs.github.io   <none>           <none>
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$
軟策略(preferredDuringSchedulingIgnoredDuringExecution)
所調(diào)度節(jié)點(diǎn)盡量為其中一個(gè)

  affinity:
    nodeAffinity: #主機(jī)親和性
      preferredDuringSchedulingIgnoredDuringExecution: # 軟策略
      - weight: 2
        preference:
          matchExpressions:
          - key: kubernetes.io/hostname
            operator: In
            values:
            - vms85.liruilongs.github.io
            - vms84.liruilongs.github.io
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$vim pod-node-a-r.yaml
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl apply -f  pod-node-a-r.yaml
pod/podnodea created
「pod-node-a-r.yaml」

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: podnodea
  name: podnodea
spec:
  containers:
  - image: nginx
    imagePullPolicy: IfNotPresent
    name: podnodea
    resources: {}
  affinity:
    nodeAffinity: #主機(jī)親和性
      preferredDuringSchedulingIgnoredDuringExecution: # 軟策略
      - weight: 2
        preference:
          matchExpressions:
          - key: kubernetes.io/hostname
            operator: In
            values:
            - vms85.liruilongs.github.io
            - vms84.liruilongs.github.io
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}
「檢查一下,調(diào)度OK」

┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl get pods -owide
NAME       READY   STATUS    RESTARTS   AGE   IP             NODE                         NOMINATED NODE   READINESS GATES
podnodea   1/1     Running   0          28s   10.244.70.62   vms83.liruilongs.github.io   <none>           <none>

常見的標(biāo)簽運(yùn)算符

運(yùn)算符    描述
In    包含自, 比如上面的硬親和就包含env_role=dev、env_role=test兩種標(biāo)簽
NotIn    和上面相反，凡是包含該標(biāo)簽的節(jié)點(diǎn)都不會(huì)匹配到
Exists    存在里面和In比較類似，凡是有某個(gè)標(biāo)簽的機(jī)器都會(huì)被選擇出來。使用Exists的operator的話，values里面就不能寫東西了。
Gt    greater than的意思，表示凡是某個(gè)value大于設(shè)定的值的機(jī)器則會(huì)被選擇出來。
Lt    less than的意思，表示凡是某個(gè)value小于設(shè)定的值的機(jī)器則會(huì)被選擇出來。
DoesNotExists    不存在該標(biāo)簽的節(jié)點(diǎn)
節(jié)點(diǎn)的coedon與drain
「如果想把某個(gè)節(jié)點(diǎn)設(shè)置為不可用的話，可以對(duì)節(jié)點(diǎn)實(shí)施cordon或者drain」

如果一個(gè)node被標(biāo)記為cordon，新創(chuàng)建的pod不會(huì)被調(diào)度到此node上，已經(jīng)調(diào)度上去的不會(huì)被移走，coedon用于節(jié)點(diǎn)的維護(hù)，當(dāng)不希望再節(jié)點(diǎn)分配pod，那么可以使用coedon把節(jié)點(diǎn)標(biāo)記為不可調(diào)度。

「這里我們?yōu)榱朔奖?，?chuàng)建一個(gè)Deployment控制器用去用于演示，deploy擁有3個(gè)pod副本」

┌──[root@vms81.liruilongs.github.io]-[~/ansible]
└─$kubectl create deployment nginx  --image=nginx  --dry-run=client -o yaml >nginx-dep.yaml
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$vim nginx-dep.yaml
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl apply  -f nginx-dep.yaml
deployment.apps/nginx created
「nginx-dep.yaml」

apiVersion: apps/v1
kind: Deployment
metadata:
  creationTimestamp: null
  labels:
    app: nginx
  name: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  strategy: {}
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: nginx
    spec:
      containers:
      - image: nginx
        name: nginx
        imagePullPolicy: IfNotPresent
        resources: {}
status: {}
可以看到pod調(diào)度到了兩個(gè)工作節(jié)點(diǎn)

┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl get pods -owide
NAME                     READY   STATUS    RESTARTS   AGE     IP               NODE
      NOMINATED NODE   READINESS GATES
nginx-7cf7d6dbc8-hx96s   1/1     Running   0          2m16s   10.244.171.167   vms82.liruilongs.github.io   <none>           <none>
nginx-7cf7d6dbc8-wshxp   1/1     Running   0          2m16s   10.244.70.1      vms83.liruilongs.github.io   <none>           <none>
nginx-7cf7d6dbc8-x78x4   1/1     Running   0          2m16s   10.244.70.63     vms83.liruilongs.github.io   <none>           <none>
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$
節(jié)點(diǎn)的coedon
基本命令

kubectl cordon vms83.liruilongs.github.io  #標(biāo)記不可用
kubectl uncordon vms83.liruilongs.github.io #取消標(biāo)記
「通過cordon把vms83.liruilongs.github.io標(biāo)記為不可調(diào)度」

┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl cordon vms83.liruilongs.github.io  #通過cordon把83標(biāo)記為不可調(diào)度
node/vms83.liruilongs.github.io cordoned
「查看節(jié)點(diǎn)狀態(tài)，vms83.liruilongs.github.io變成SchedulingDisabled」

┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl get nodes
NAME                         STATUS                     ROLES                  AGE   VERSION
vms81.liruilongs.github.io   Ready                      control-plane,master   48d   v1.22.2
vms82.liruilongs.github.io   Ready                      worker1                48d   v1.22.2
vms83.liruilongs.github.io   Ready,SchedulingDisabled   worker2                48d   v1.22.2
「修改deployment副本數(shù)量 --replicas=6 」

┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl scale deployment nginx  --replicas=6
deployment.apps/nginx scaled
「新增的pod都調(diào)度到了vms82.liruilongs.github.io 節(jié)點(diǎn)」

┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl get pods -o wide
NAME                     READY   STATUS    RESTARTS   AGE     IP               NODE                         NOMINATED NODE   READINESS GATES
nginx-7cf7d6dbc8-2nmsj   1/1     Running   0          64s     10.244.171.170   vms82.liruilongs.github.io   <none>           <none>
nginx-7cf7d6dbc8-chsrn   1/1     Running   0          63s     10.244.171.168   vms82.liruilongs.github.io   <none>           <none>
nginx-7cf7d6dbc8-hx96s   1/1     Running   0          7m30s   10.244.171.167   vms82.liruilongs.github.io   <none>           <none>
nginx-7cf7d6dbc8-lppbp   1/1     Running   0          63s     10.244.171.169   vms82.liruilongs.github.io   <none>           <none>
nginx-7cf7d6dbc8-wshxp   1/1     Running   0          7m30s   10.244.70.1      vms83.liruilongs.github.io   <none>           <none>
nginx-7cf7d6dbc8-x78x4   1/1     Running   0          7m30s   10.244.70.63     vms83.liruilongs.github.io   <none>           <none>
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$
「把vms83.liruilongs.github.io節(jié)點(diǎn)上的pod都干掉，會(huì)發(fā)現(xiàn)新增pod都調(diào)度到了vms82.liruilongs.github.io」

┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl delete pod nginx-7cf7d6dbc8-wshxp
pod "nginx-7cf7d6dbc8-wshxp" deleted
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl get pods -o wide
NAME                     READY   STATUS    RESTARTS   AGE     IP               NODE                         NOMINATED NODE   READINESS GATES
nginx-7cf7d6dbc8-2nmsj   1/1     Running   0          2m42s   10.244.171.170   vms82.liruilongs.github.io   <none>           <none>
nginx-7cf7d6dbc8-5hnc7   1/1     Running   0          10s     10.244.171.171   vms82.liruilongs.github.io   <none>           <none>
nginx-7cf7d6dbc8-chsrn   1/1     Running   0          2m41s   10.244.171.168   vms82.liruilongs.github.io   <none>           <none>
nginx-7cf7d6dbc8-hx96s   1/1     Running   0          9m8s    10.244.171.167   vms82.liruilongs.github.io   <none>           <none>
nginx-7cf7d6dbc8-lppbp   1/1     Running   0          2m41s   10.244.171.169   vms82.liruilongs.github.io   <none>           <none>
nginx-7cf7d6dbc8-x78x4   1/1     Running   0          9m8s    10.244.70.63     vms83.liruilongs.github.io   <none>           <none>
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl delete pod nginx-7cf7d6dbc8-x78x4
pod "nginx-7cf7d6dbc8-x78x4" deleted
pod都位于正常的節(jié)點(diǎn)

┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl get pods -o wide
NAME                     READY   STATUS    RESTARTS   AGE     IP               NODE                         NOMINATED NODE   READINESS GATES
nginx-7cf7d6dbc8-2nmsj   1/1     Running   0          3m31s   10.244.171.170   vms82.liruilongs.github.io   <none>           <none>
nginx-7cf7d6dbc8-5hnc7   1/1     Running   0          59s     10.244.171.171   vms82.liruilongs.github.io   <none>           <none>
nginx-7cf7d6dbc8-chsrn   1/1     Running   0          3m30s   10.244.171.168   vms82.liruilongs.github.io   <none>           <none>
nginx-7cf7d6dbc8-hx96s   1/1     Running   0          9m57s   10.244.171.167   vms82.liruilongs.github.io   <none>           <none>
nginx-7cf7d6dbc8-lppbp   1/1     Running   0          3m30s   10.244.171.169   vms82.liruilongs.github.io   <none>           <none>
nginx-7cf7d6dbc8-m8ltr   1/1     Running   0          30s     10.244.171.172   vms82.liruilongs.github.io   <none>           <none>
「通過 uncordon恢復(fù)節(jié)點(diǎn)vms83.liruilongs.github.io狀態(tài)」

┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl uncordon vms83.liruilongs.github.io #恢復(fù)節(jié)點(diǎn)狀態(tài)
node/vms83.liruilongs.github.io uncordoned
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl get nodes
NAME                         STATUS   ROLES                  AGE   VERSION
vms81.liruilongs.github.io   Ready    control-plane,master   48d   v1.22.2
vms82.liruilongs.github.io   Ready    worker1                48d   v1.22.2
vms83.liruilongs.github.io   Ready    worker2                48d   v1.22.2
「刪除所有的pod」

┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl scale deployment nginx --replicas=0
deployment.apps/nginx scaled
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl get pods -o wide
No resources found in liruilong-pod-create namespace.
節(jié)點(diǎn)的drain
「如果一個(gè)節(jié)點(diǎn)被設(shè)置為drain，則此節(jié)點(diǎn)不再被調(diào)度pod，且此節(jié)點(diǎn)上已經(jīng)運(yùn)行的pod會(huì)被驅(qū)逐(evicted)到其他節(jié)點(diǎn)」

「drain包含兩種狀態(tài)：cordon不可被調(diào)度，evicted驅(qū)逐當(dāng)前節(jié)點(diǎn)所以pod」

kubectl drain vms83.liruilongs.github.io   --ignore-daemonsets
kubectl uncordon vms83.liruilongs.github.io  
「通過deployment添加4個(gè)nginx副本--replicas=4」

┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl scale deployment nginx --replicas=4
deployment.apps/nginx scaled
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl get pods -o wide --one-output
NAME                     READY   STATUS    RESTARTS   AGE   IP               NODE                         NOMINATED NODE   READINESS GATES
nginx-7cf7d6dbc8-2clnb   1/1     Running   0          22s   10.244.171.174   vms82.liruilongs.github.io   <none>           <none>
nginx-7cf7d6dbc8-9p6g2   1/1     Running   0          22s   10.244.70.2      vms83.liruilongs.github.io   <none>           <none>
nginx-7cf7d6dbc8-ptqxm   1/1     Running   0          22s   10.244.171.173   vms82.liruilongs.github.io   <none>           <none>
nginx-7cf7d6dbc8-zmdqm   1/1     Running   0          22s   10.244.70.4      vms83.liruilongs.github.io   <none>           <none>
「添加一下drain 將節(jié)點(diǎn)vms82.liruilongs.github.io設(shè)置為drain」

┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl drain vms82.liruilongs.github.io --ignore-daemonsets --delete-emptydir-data
node/vms82.liruilongs.github.io cordoned
WARNING: ignoring DaemonSet-managed Pods: kube-system/calico-node-ntm7v, kube-system/kube-proxy-nzm24
evicting pod liruilong-pod-create/nginx-7cf7d6dbc8-ptqxm
evicting pod kube-system/metrics-server-bcfb98c76-wxv5l
evicting pod liruilong-pod-create/nginx-7cf7d6dbc8-2clnb
pod/nginx-7cf7d6dbc8-2clnb evicted
pod/nginx-7cf7d6dbc8-ptqxm evicted
pod/metrics-server-bcfb98c76-wxv5l evicted
node/vms82.liruilongs.github.io evicted
查看節(jié)點(diǎn)，狀態(tài)為不可被調(diào)度

┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl get nodes
NAME                         STATUS                     ROLES                  AGE   VERSION
vms81.liruilongs.github.io   Ready                      control-plane,master   48d   v1.22.2
vms82.liruilongs.github.io   Ready,SchedulingDisabled   worker1                48d   v1.22.2
vms83.liruilongs.github.io   Ready                      worker2                48d   v1.22.2
「等一會(huì)，查看節(jié)點(diǎn)調(diào)度，所有pod調(diào)度到了vms83.liruilongs.github.io這臺(tái)機(jī)器」

┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl get pods -o wide --one-output
NAME                     READY   STATUS    RESTARTS   AGE     IP            NODE                         NOMINATED NODE   READINESS GATES
nginx-7cf7d6dbc8-9p6g2   1/1     Running   0          4m20s   10.244.70.2   vms83.liruilongs.github.io   <none>           <none>
nginx-7cf7d6dbc8-hkflr   1/1     Running   0          25s     10.244.70.5   vms83.liruilongs.github.io   <none>           <none>
nginx-7cf7d6dbc8-qt48k   1/1     Running   0          26s     10.244.70.7   vms83.liruilongs.github.io   <none>           <none>
nginx-7cf7d6dbc8-zmdqm   1/1     Running   0          4m20s   10.244.70.4   vms83.liruilongs.github.io   <none>           <none>
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$
「取消drain：kubectl uncordon vms82.liruilongs.github.io」

┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl uncordon vms82.liruilongs.github.io
node/vms82.liruilongs.github.io uncordoned
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$
「有時(shí)候會(huì)報(bào)錯(cuò)」

「將節(jié)點(diǎn)vms82.liruilongs.github.io設(shè)置為drain」

┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl drain vms82.liruilongs.github.io
node/vms82.liruilongs.github.io cordoned
DEPRECATED WARNING: Aborting the drain command in a list of nodes will be deprecated in v1.23.
The new behavior will make the drain command go through all nodes even if one or more nodes failed during the drain.
For now, users can try such experience via: --ignore-errors
error: unable to drain node "vms82.liruilongs.github.io", aborting command...

There are pending nodes to be drained:
 vms82.liruilongs.github.io
cannot delete DaemonSet-managed Pods (use --ignore-daemonsets to ignore): kube-system/calico-node-ntm7v, kube-system/kube-proxy-nzm24
cannot delete Pods with local storage (use --delete-emptydir-data to override): kube-system/metrics-server-bcfb98c76-wxv5l
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl get nodes
NAME                         STATUS                     ROLES                  AGE   VERSION
vms81.liruilongs.github.io   Ready                      control-plane,master   48d   v1.22.2
vms82.liruilongs.github.io   Ready,SchedulingDisabled   worker1                48d   v1.22.2
vms83.liruilongs.github.io   Ready                      worker2                48d   v1.22.2
「uncordon掉剛才的節(jié)點(diǎn)」

┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl uncordon vms82.liruilongs.github.io
node/vms82.liruilongs.github.io uncordoned
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl get nodes
NAME                         STATUS   ROLES                  AGE   VERSION
vms81.liruilongs.github.io   Ready    control-plane,master   48d   v1.22.2
vms82.liruilongs.github.io   Ready    worker1                48d   v1.22.2
vms83.liruilongs.github.io   Ready    worker2                48d   v1.22.2
節(jié)點(diǎn)taint及pod的tolerations
默認(rèn)情況下，pod是不會(huì)調(diào)度到有污點(diǎn)的節(jié)點(diǎn)，master節(jié)點(diǎn)從來沒有調(diào)度到pod，因?yàn)閙aster節(jié)點(diǎn)設(shè)置了污點(diǎn)，如果想要在某個(gè)被設(shè)置了污點(diǎn)的機(jī)器調(diào)度pod，那么pod需要設(shè)置tolerations(容忍污點(diǎn))才能夠被運(yùn)行。

master節(jié)點(diǎn)的污點(diǎn)

┌──[root@vms81.liruilongs.github.io]-[~/ansible]
└─$ansible master -m shell -a "kubectl describe  nodes vms81.liruilongs.github.io | grep -E '(Roles|Taints)'"
192.168.26.81 | CHANGED | rc=0 >>
Roles:              control-plane,master
Taints:             node-role.kubernetes.io/master:NoSchedule
taint（污點(diǎn)）的設(shè)置和查看
查看節(jié)點(diǎn)角色，和是否設(shè)置污點(diǎn)

┌──[root@vms81.liruilongs.github.io]-[~/ansible]
└─$kubectl describe  nodes vms82.liruilongs.github.io | grep -E '(Roles|Taints)'
Roles:              worker1
Taints:             <none>
給 vms83.liruilongs.github.io節(jié)點(diǎn)設(shè)置污點(diǎn)，指定key為key83

┌──[root@vms81.liruilongs.github.io]-[~/ansible]
└─$kubectl describe  nodes vms83.liruilongs.github.io | grep -E '(Roles|Taints)'
Roles:              worker2
Taints:             <none>
┌──[root@vms81.liruilongs.github.io]-[~/ansible]
└─$kubectl taint node vms83.liruilongs.github.io key83=:NoSchedule
node/vms83.liruilongs.github.io tainted
┌──[root@vms81.liruilongs.github.io]-[~/ansible]
└─$kubectl describe  nodes vms83.liruilongs.github.io | grep -E '(Roles|Taints)' # 從新查看污點(diǎn)信息
Roles:              worker2
Taints:             key83:NoSchedule
┌──[root@vms81.liruilongs.github.io]-[~/ansible]
└─$
「重新通過deployment 創(chuàng)建pod，會(huì)發(fā)現(xiàn)pod都調(diào)度到82上面，因?yàn)?3設(shè)置了污點(diǎn)」

┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl scale deployment nginx --replicas=0
deployment.apps/nginx scaled
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl scale deployment nginx --replicas=4
deployment.apps/nginx scaled
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl get pods -o wide --one-output
NAME                     READY   STATUS              RESTARTS   AGE   IP       NODE                         NOMINATED NODE   READINESS GATES
nginx-7cf7d6dbc8-dhst5   0/1     ContainerCreating   0          12s   <none>   vms82.liruilongs.github.io   <none>           <none>
nginx-7cf7d6dbc8-j6g25   0/1     ContainerCreating   0          12s   <none>   vms82.liruilongs.github.io   <none>           <none>
nginx-7cf7d6dbc8-wpnhr   0/1     ContainerCreating   0          12s   <none>   vms82.liruilongs.github.io   <none>           <none>
nginx-7cf7d6dbc8-zkww8   0/1     ContainerCreating   0          11s   <none>   vms82.liruilongs.github.io   <none>           <none>
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl delete deployment nginx
deployment.apps "nginx" deleted
「取消污點(diǎn)設(shè)置」

┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl taint node vms83.liruilongs.github.io key83-
node/vms83.liruilongs.github.io untainted
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl describe  nodes vms83.liruilongs.github.io | grep -E '(Roles|Taints)'
Roles:              worker2
Taints:             <none>
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$
設(shè)置operator的值為Equal
「如果需要在有污點(diǎn)的節(jié)點(diǎn)上運(yùn)行pod，那么需要在定義pod的時(shí)候指定toleration屬性」

給 vms82.liruilongs.github.io節(jié)點(diǎn)設(shè)置污點(diǎn)，指定key為key82,value為val82

┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl taint nodes  vms82.liruilongs.github.io key82=val82:NoSchedule
node/vms82.liruilongs.github.io tainted
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl describe  nodes vms82.liruilongs.github.io | grep -E '(Roles|Taints)'
Roles:              worker1
Taints:             key82=val82:NoSchedule
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$
「在設(shè)置節(jié)點(diǎn)taint的時(shí)候，如果value的值不為空，在pod里的tolerations字段只能寫Equal，不能寫Exists,」

「修改yaml文件 pod-taint3.yaml」

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: pod1
  name: pod1
spec:
  nodeSelector:
    disktype: node2
  tolerations:
  - key: "key82"
    operator: "Equal"
    value: "val82"
    effect: "NoSchedule"
  containers:
  - image: nginx
    name: pod1
    resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}
創(chuàng)建pod后，執(zhí)行成功，容忍污點(diǎn)

┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl apply -f pod-taint3.yaml
pod/pod1 created
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl get pods -o wide
NAME   READY   STATUS    RESTARTS   AGE   IP               NODE                         NOMINATED NODE   READINESS GATES
pod1   1/1     Running   0          11s   10.244.171.180   vms82.liruilongs.github.io   <none>           <none>
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$
設(shè)置operator的值為Exists
「如果使用Exists的話，那么pod中不能寫value」

「設(shè)置 vms83.liruilongs.github.io 節(jié)點(diǎn)污點(diǎn)標(biāo)記,value為空」

┌──[root@vms81.liruilongs.github.io]-[~/ansible]
└─$kubectl taint node vms83.liruilongs.github.io key83=:NoSchedule
node/vms83.liruilongs.github.io tainted
┌──[root@vms81.liruilongs.github.io]-[~/ansible]
└─$kubectl describe  nodes vms83.liruilongs.github.io | grep -E '(Roles|Taints)'
Roles:              worker2
Taints:             key83:NoSchedule
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl apply -f pod-taint.yaml
pod/pod1 created
「pod-taint.yaml」

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: pod1
  name: pod1
spec:
  nodeSelector:
    disktype: node2
  tolerations:
  - key: "key83"
    operator: "Exists"
    effect: "NoSchedule"
  containers:
  - image: nginx
    name: pod1
    resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}
「會(huì)發(fā)現(xiàn)節(jié)點(diǎn)調(diào)度到了有污點(diǎn)的vms83.liruilongs.github.io節(jié)點(diǎn)」

┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl get pods -o wide
NAME   READY   STATUS    RESTARTS   AGE    IP            NODE                         NOMINATED NODE   READINESS GATES
pod1   1/1     Running   0          3m4s   10.244.70.8   vms83.liruilongs.github.io   <none>           <none>
「當(dāng)然，value沒有值也可以這樣使用Equal」

┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$cp pod-taint.yaml pod-taint2.yaml
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$vim pod-taint2.yaml
「pod-taint2.yaml」

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: pod1
  name: pod1
spec:
  nodeSelector:
    disktype: node2
  tolerations:
  - key: "key83"
    operator: "Equal"
    value: ""
    effect: "NoSchedule"
  containers:
  - image: nginx
    name: pod1
    resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}
「會(huì)發(fā)現(xiàn)節(jié)點(diǎn)還是調(diào)度到了有污點(diǎn)的vms83.liruilongs.github.io節(jié)點(diǎn)」

┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl  delete -f pod-taint.yaml
pod "pod1" deleted
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl apply -f pod-taint2.yaml
pod/pod1 created
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl get pods -o wide
NAME   READY   STATUS              RESTARTS   AGE   IP       NODE                         NOMINATED NODE   READINESS GATES
pod1   0/1     ContainerCreating   0          8s    <none>   vms83.liruilongs.github.io   <none>           <none>
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$kubectl taint nodes vms83.liruilongs.github.io key83-
node/vms83.liruilongs.github.io untainted
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-pod-create]
└─$
關(guān)于pod調(diào)度和小伙伴們分享到這里.生活加油 ^_^

作者：山河已無恙

歡迎關(guān)注微信公眾號(hào) ：山河已無恙