k8s创建应用一直卡在 ContainerCreating的解决办法

今天使用k8s创建应用的时候一直卡在ContainerCreating状态

查看日志报如下信息

Error from server: Get https://worker03:10250/containerLogs/default/web1-7f87dfbd56-fjq9p/nginx: dial tcp: lookup worker03 on 114.114.114.114:53: no such host

排查worker03的kubelet服务也正常,也可正常pull镜像

继续排查信息如下

[root@master01 ~]# kubectl describe pod nginx
Name:           nginx-test-c675f77c4-47hw8
Namespace:      default
Priority:       0
Node:           worker03/172.18.1.226
Start Time:     Thu, 26 Nov 2020 14:35:09 +0800
Labels:         app=nginx-test
                pod-template-hash=c675f77c4
Annotations:    <none>
Status:         Pending
IP:             
IPs:            <none>
Controlled By:  ReplicaSet/nginx-test-c675f77c4
Containers:
  nginx:
    Container ID:   
    Image:          nginx
    Image ID:       
    Port:           <none>
    Host Port:      <none>
    State:          Waiting
      Reason:       ContainerCreating
    Ready:          False
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-wvncq (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:
  default-token-wvncq:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-wvncq
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 360s
                 node.kubernetes.io/unreachable:NoExecute for 360s
Events:
  Type     Reason                  Age                From               Message
  ----     ------                  ----               ----               -------
  Normal   Scheduled               2m11s              default-scheduler  Successfully assigned default/nginx-test-c675f77c4-47hw8 to worker03
  Warning  FailedCreatePodSandBox  2s (x4 over 109s)  kubelet, worker03  Failed to create pod sandbox: rpc error: code = Unknown desc = failed pulling image "k8s.gcr.io/pause-amd64:3.2": Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

从上面日志信息可以看出是缺少k8s.gcr.io/pause-amd64:3.2镜像导致

这里是因为会使用地址 k8s.gcr.io/pause-amd64:3.2 进行拉取镜像,但是这个地址被墙了,所以不通

解决办法:

在worker03节点上操作

[root@worker03 ~]# docker pull docker.io/kubernetes/pause
Using default tag: latest
latest: Pulling from kubernetes/pause
4f4fb700ef54: Pull complete 
b9c8ec465f6b: Pull complete 
Digest: sha256:b31bfb4d0213f254d361e0079deaaebefa4f82ba7aa76ef82e90b4935ad5b105
Status: Downloaded newer image for kubernetes/pause:latest
docker.io/kubernetes/pause:latest
[root@worker03 ~]# ^Cs.gcr.io/pause-amd64:3.2
[root@worker03 ~]# docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
nginx               latest              bc9a0695f571        30 hours ago        133MB
kubernetes/pause    latest              f9d5de079539        6 years ago         240kB
[root@worker03 ~]# docker tag kubernetes/pause:latest k8s.gcr.io/pause-amd64:3.2
[root@worker03 ~]# docker images
REPOSITORY               TAG                 IMAGE ID            CREATED             SIZE
nginx                    latest              bc9a0695f571        30 hours ago        133MB
kubernetes/pause         latest              f9d5de079539        6 years ago         240kB
k8s.gcr.io/pause-amd64   3.2                 f9d5de079539        6 years ago         240kB

删除之前创建的pod重新在创建

[root@master01 ~]#  kubectl create deployment nginx-test --image=nginx
deployment.apps/nginx-test created

再次查看问题解决

发表评论

后才能评论