构建 Kubernetes PV

    参考 https://kubernetes.feisky.xyz/concepts/objects/persistent-volume#volume-sheng-ming-zhou-qi

    参考 https://blog.csdn.net/liumiaocn/article/details/103388607

    环境准备

    一个搭建好的 nfs 系统, 并确定能使用

    test

    root@l2:~# df -h | grep mnt
    10.0.4.44:/z6dajk13   10G   32M   10G   1% /mnt/nfs
    root@l2:~# cd /mnt/nfs
    root@l2:/mnt/nfs# ls -l
    total 8
    -rw-r--r-- 1 root root    4 May 29 17:16 a.txt
    drwxr-xr-x 2 root root    6 May 29 17:30 k8s
    drwxr-xr-x 5 root root 4096 May 29 18:53 kubernates
    root@l2:/mnt/nfs# rm -rf k8s
    root@l2:/mnt/nfs# echo 'bb' >> b.txt
    root@l2:/mnt/nfs# cat b.txt 
    bb
    root@l2:/mnt/nfs# 
    

    构建 nfs client

    安装 nfs client.

    StorageClass for Persistent Volumeclassmy-nfsclient

    helm install my-nfsclient azure/nfs-client-provisioner --set nfs.server=10.0.2.123 --set nfs.path=/vrcuz9bj/k8s

    检测状态 kubectl get pods

    root@l2:/mnt/nfs# kubectl get pods
    NAME                                                   READY   STATUS             RESTARTS   AGE
    my-nfsclient-nfs-client-provisioner-7b7fbc4c58-pvm6z   1/1     Running            0          25m
    

    查看 pod 详情 kubectl describe pods my-nfsclient-nfs-client-provisioner-7b7fbc4c58-pvm6z

    
    root@l2:/mnt/nfs# kubectl describe pods my-nfsclient-nfs-client-provisioner-7b7fbc4c58-pvm6z
    Name:         my-nfsclient-nfs-client-provisioner-7b7fbc4c58-pvm6z
    Namespace:    default
    Priority:     0
    Node:         l4/10.0.4.12
    Start Time:   Sun, 29 May 2022 18:35:31 +0800
    Labels:       app=nfs-client-provisioner
                  pod-template-hash=7b7fbc4c58
                  release=my-nfsclient
    Annotations:  <none>
    Status:       Running
    IP:           10.244.2.9
    IPs:
      IP:           10.244.2.9
    Controlled By:  ReplicaSet/my-nfsclient-nfs-client-provisioner-7b7fbc4c58
    Containers:
      nfs-client-provisioner:
        Container ID:   docker://6e77341dcef10b16c9d6b23bee30ea4e45204f7b9791b8d807d75cb10f3cf449
        Image:          quay.io/external_storage/nfs-client-provisioner:v3.1.0-k8s1.11
        Image ID:       docker-pullable://quay.io/external_storage/nfs-client-provisioner@sha256:cdbccbf53d100b36eae744c1cb07be3d0d22a8e64bb038b7a3808dd29c174661
        Port:           <none>
        Host Port:      <none>
        State:          Running
          Started:      Sun, 29 May 2022 18:37:27 +0800
        Ready:          True
        Restart Count:  0
        Environment:
          PROVISIONER_NAME:  cluster.local/my-nfsclient-nfs-client-provisioner
          NFS_SERVER:        10.0.4.44
          NFS_PATH:          /z6dajk13/kubernates
        Mounts:
          /persistentvolumes from nfs-client-root (rw)
          /var/run/secrets/kubernetes.io/serviceaccount from my-nfsclient-nfs-client-provisioner-token-69ccz (ro)
    Conditions:
      Type              Status
      Initialized       True 
      Ready             True 
      ContainersReady   True 
      PodScheduled      True 
    Volumes:
      nfs-client-root:
        Type:      NFS (an NFS mount that lasts the lifetime of a pod)
        Server:    10.0.4.44
        Path:      /z6dajk13/kubernates
        ReadOnly:  false
      my-nfsclient-nfs-client-provisioner-token-69ccz:
        Type:        Secret (a volume populated by a Secret)
        SecretName:  my-nfsclient-nfs-client-provisioner-token-69ccz
        Optional:    false
    QoS Class:       BestEffort
    Node-Selectors:  <none>
    Tolerations:     node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                     node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
    Events:
      Type    Reason     Age   From               Message
      ----    ------     ----  ----               -------
      Normal  Scheduled  27m   default-scheduler  Successfully assigned default/my-nfsclient-nfs-client-provisioner-7b7fbc4c58-pvm6z to l4
      Normal  Pulling    27m   kubelet            Pulling image "quay.io/external_storage/nfs-client-provisioner:v3.1.0-k8s1.11"
      Normal  Pulled     25m   kubelet            Successfully pulled image "quay.io/external_storage/nfs-client-provisioner:v3.1.0-k8s1.11" in 1m50.942114237s
      Normal  Created    25m   kubelet            Created container nfs-client-provisioner
      Normal  Started    25m   kubelet            Started container nfs-client-provisioner
    

    安装 redis 进行测试

    检测 redis repo: https://artifacthub.io/packages/helm/bitnami/redis

    检查 pv class 配置项

    安装 helm install my-redis bitnami/redis --set global.storageClass=nfs-client

    root@l2:~# helm install my-redis bitnami/redis --set global.storageClass=nfs-client
    NAME: my-redis
    LAST DEPLOYED: Sun May 29 18:53:22 2022
    NAMESPACE: default
    STATUS: deployed
    REVISION: 1
    TEST SUITE: None
    NOTES:
    CHART NAME: redis
    CHART VERSION: 16.10.0
    APP VERSION: 6.2.7
    
    ** Please be patient while the chart is being deployed **
    
    Redis&trade; can be accessed on the following DNS names from within your cluster:
    
        my-redis-master.default.svc.cluster.local for read/write operations (port 6379)
        my-redis-replicas.default.svc.cluster.local for read-only operations (port 6379)
    
    
    
    To get your password run:
    
        export REDIS_PASSWORD=$(kubectl get secret --namespace default my-redis -o jsonpath="{.data.redis-password}" | base64 --decode)
    
    To connect to your Redis&trade; server:
    
    1. Run a Redis&trade; pod that you can use as a client:
    
       kubectl run --namespace default redis-client --restart='Never'  --env REDIS_PASSWORD=$REDIS_PASSWORD  --image docker.io/bitnami/redis:6.2.7-debian-10-r23 --command -- sleep infinity
    
       Use the following command to attach to the pod:
    
       kubectl exec --tty -i redis-client \
       --namespace default -- bash
    
    2. Connect using the Redis&trade; CLI:
       REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli -h my-redis-master
       REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli -h my-redis-replicas
    
    To connect to your database from outside the cluster execute the following commands:
    
        kubectl port-forward --namespace default svc/my-redis-master 6379:6379 &
        REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli -h 127.0.0.1 -p 6379
    

    检测状态 kubectl get pods 处于 running

    root@l2:~# kubectl get pods
    NAME                                                   READY   STATUS             RESTARTS   AGE
    my-nfsclient-nfs-client-provisioner-7b7fbc4c58-pvm6z   1/1     Running            0          57m
    my-redis-master-0                                      1/1     Running            0          39m
    my-redis-replicas-0                                    0/1     CrashLoopBackOff   13         39m
    

    检测详情 kubectl decribe pods my-redis-master-0

    root@l2:~# kubectl describe pods my-redis-master-0
    Name:         my-redis-master-0
    Namespace:    default
    Priority:     0
    Node:         l4/10.0.4.12
    Start Time:   Sun, 29 May 2022 18:53:26 +0800
    Labels:       app.kubernetes.io/component=master
                  app.kubernetes.io/instance=my-redis
                  app.kubernetes.io/managed-by=Helm
                  app.kubernetes.io/name=redis
                  controller-revision-hash=my-redis-master-564575987f
                  helm.sh/chart=redis-16.10.0
                  statefulset.kubernetes.io/pod-name=my-redis-master-0
    Annotations:  checksum/configmap: a746719e48ee91bc2f3077dba980d86d03278b30d1961852890e42a78771690f
                  checksum/health: c1626ed9079e04fb9c301d80021e964758fd212c0c66611f0a7f028e39778389
                  checksum/scripts: ac900030eba0d31a0a2ef7ed079ef662aa81d1d8f0fae534b22ecd40742cb714
                  checksum/secret: c1e0ecb80e7fee67b7520d43b3fe6266d166e0da4b0fb9055de5c1141532d56f
    Status:       Running
    IP:           10.244.2.10
    IPs:
      IP:           10.244.2.10
    Controlled By:  StatefulSet/my-redis-master
    Containers:
      redis:
        Container ID:  docker://da621ab4b95117c2eaeb41a99adba34d7f5f820dd8a4f4fbe50915197bc87783
        Image:         docker.io/bitnami/redis:6.2.7-debian-10-r23
        Image ID:      docker-pullable://bitnami/redis@sha256:a61e5bed84aa7b0434eb7003fcad586800a24518209efa5c5769806ec5d6e4d5
        Port:          6379/TCP
        Host Port:     0/TCP
        Command:
          /bin/bash
        Args:
          -c
          /opt/bitnami/scripts/start-scripts/start-master.sh
        State:          Running
          Started:      Sun, 29 May 2022 18:53:58 +0800
        Ready:          True
        Restart Count:  0
        Liveness:       exec [sh -c /health/ping_liveness_local.sh 5] delay=20s timeout=6s period=5s #success=1 #failure=5
        Readiness:      exec [sh -c /health/ping_readiness_local.sh 1] delay=20s timeout=2s period=5s #success=1 #failure=5
        Environment:
          BITNAMI_DEBUG:           false
          REDIS_REPLICATION_MODE:  master
          ALLOW_EMPTY_PASSWORD:    no
          REDIS_PASSWORD:          <set to the key 'redis-password' in secret 'my-redis'>  Optional: false
          REDIS_TLS_ENABLED:       no
          REDIS_PORT:              6379
        Mounts:
          /data from redis-data (rw)
          /health from health (rw)
          /opt/bitnami/redis/etc/ from redis-tmp-conf (rw)
          /opt/bitnami/redis/mounted-etc from config (rw)
          /opt/bitnami/scripts/start-scripts from start-scripts (rw)
          /tmp from tmp (rw)
          /var/run/secrets/kubernetes.io/serviceaccount from my-redis-token-92rs6 (ro)
    Conditions:
      Type              Status
      Initialized       True 
      Ready             True 
      ContainersReady   True 
      PodScheduled      True 
    Volumes:
      redis-data:
        Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
        ClaimName:  redis-data-my-redis-master-0
        ReadOnly:   false
      start-scripts:
        Type:      ConfigMap (a volume populated by a ConfigMap)
        Name:      my-redis-scripts
        Optional:  false
      health:
        Type:      ConfigMap (a volume populated by a ConfigMap)
        Name:      my-redis-health
        Optional:  false
      config:
        Type:      ConfigMap (a volume populated by a ConfigMap)
        Name:      my-redis-configuration
        Optional:  false
      redis-tmp-conf:
        Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
        Medium:     
        SizeLimit:  <unset>
      tmp:
        Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
        Medium:     
        SizeLimit:  <unset>
      my-redis-token-92rs6:
        Type:        Secret (a volume populated by a Secret)
        SecretName:  my-redis-token-92rs6
        Optional:    false
    QoS Class:       BestEffort
    Node-Selectors:  <none>
    Tolerations:     node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                     node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
    Events:
      Type     Reason            Age                From               Message
      ----     ------            ----               ----               -------
      Warning  FailedScheduling  40m (x2 over 40m)  default-scheduler  0/3 nodes are available: 3 pod has unbound immediate PersistentVolumeClaims.
      Normal   Scheduled         40m                default-scheduler  Successfully assigned default/my-redis-master-0 to l4
      Normal   Pulling           40m                kubelet            Pulling image "docker.io/bitnami/redis:6.2.7-debian-10-r23"
      Normal   Pulled            40m                kubelet            Successfully pulled image "docker.io/bitnami/redis:6.2.7-debian-10-r23" in 28.401308376s
      Normal   Created           40m                kubelet            Created container redis
      Normal   Started           40m                kubelet            Started container redis
    

    评论栏