가시다님 스터디 쿠버네티스편 [3주차]

강재민·2023년 1월 30일
0

Kubernetes

목록 보기
24/29

[과제 1]

Ingress(with 도메인, 단일 ALB 사용)에 PATH /mario 는 mario 게임 접속하게 설정하고, /tetris 는 tetris 게임에 접속하게 설정하고, SSL 적용 후 관련 스샷 올려주세요



테트리스 또는 테트리스/* 경로에 대해서 해당 대상그룹으로 포워딩
마리오 또는 마리오/* 경로에 대해서 해당 대상그룹으로 포워딩
요청에 응답이 없으면 404에러

[과제 2]

호스트 Path(local-path-provisioner) 실습 및 문제점 확인과 성능 측정 후 관련 스샷 올려주세요

PVC가 특정 노드만 실행 중인 파드는 해당 노드에서만 실행이 가능합니다. 따라서 해당 노드의 문제가 발생하면 파드가 실행되지 않아 서비스 장애가 발생합니다.
정상

Every 2.0s: kubectl get pod,pv,pvc -owide                           Tue Jan 31 23:13:30 2023

NAME                          READY   STATUS    RESTARTS   AGE   IP             NODE
          NOMINATED NODE   READINESS GATES
pod/date-pod-d95d6b8f-rb44q   1/1     Running   0          22s   172.30.58.14   i-00edbcc760f7b92c3   <none>           <none>

NAME                                                        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                     STORAGECLASS   REASON   AGE   VOLUMEMODE
persistentvolume/pvc-f133582f-15b7-4c05-a0cc-ba97ebc82f43   2Gi        RWO            Delete           Bound    default/localpath-claim   local-path              18s   Filesystem

NAME                                    STATUS   VOLUME
CAPACITY   ACCESS MODES   STORAGECLASS   AGE   VOLUMEMODE
persistentvolumeclaim/localpath-claim   Bound    pvc-f133582f-15b7-4c05-a0cc-ba97ebc82f43
2Gi        RWO            local-path     22s   Filesystem

drain 설정 이후 장애

Every 2.0s: kubectl get pod,pv,pvc -owide                           Tue Jan 31 23:14:10 2023

NAME                          READY   STATUS    RESTARTS   AGE   IP       NODE     NOMINATED NODE   READINESS GATES
pod/date-pod-d95d6b8f-njpl5   0/1     Pending   0          12s   <none>   <none>   <none>
        <none>

NAME                                                        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                     STORAGECLASS   REASON   AGE   VOLUMEMODE
persistentvolume/pvc-f133582f-15b7-4c05-a0cc-ba97ebc82f43   2Gi        RWO            Delete           Bound    default/localpath-claim   local-path              58s   Filesystem

NAME                                    STATUS   VOLUME
CAPACITY   ACCESS MODES   STORAGECLASS   AGE   VOLUMEMODE
persistentvolumeclaim/localpath-claim   Bound    pvc-f133582f-15b7-4c05-a0cc-ba97ebc82f43
2Gi        RWO            local-path     62s   Filesystem

에러 로그

(repush:default) [root@kops-ec2 ~]# kubectl describe pod -l app=date | grep Events: -A5
Events:
  Type     Reason            Age   From               Message
  ----     ------            ----  ----               -------
  Warning  FailedScheduling  38s   default-scheduler  0/3 nodes are available: 1 node(s) had untolerated taint {node-role.kubernetes.io/control-plane: }, 1 node(s) had untolerated taint {node.kubernetes.io/unschedulable: }, 1 node(s) were unschedulable, 2 node(s) had volume node affinity conflict. preemption: 0/3 nodes are available: 3 Preemption is not helpful for scheduling.

kubectl describe pv

...
Node Affinity:
  Required Terms:
    Term 0:        kubernetes.io/hostname in [i-00edbcc760f7b92c3]
...

로컬 호스트패스 스토리지 클래스는 다른 스토리지 솔루션에 비해 IOPS 성능이 뛰어나고 구성이 간편합니다. 하지만 특정 노드의 디스크를 사용하므로 해당 노드에 장애가 발생하면 서비스 장애가 발생합니다. 따라서 애플리케이션 레벨의 볼륨 복제 설정이 필요합니다.

W1PIP=3.39.6.65
W2PIP=13.209.43.177

(repush:default) [root@kops-ec2 ~]# kubestr fio -f fio-read.fio -s local-path --size 10G
PVC created kubestr-fio-pvc-9g6nq
Pod created kubestr-fio-pod-glttn
Running FIO test (fio-read.fio) on StorageClass (local-path) with a PVC of Size (10G)
Elapsed time- 2m32.640633959s
FIO test results:

FIO version - fio-3.30
Global options - ioengine=libaio verify= direct=1 gtod_reduce=

JobName:
  blocksize= filesize= iodepth= rw=
read:
  IOPS=3021.462891 BW(KiB/s)=12085
  iops: min=2476 max=8992 avg=3022.627686
  bw(KiB/s): min=9904 max=35968 avg=12090.510742

Disk stats (read/write):
  nvme0n1: ios=362293/463 merge=0/153 ticks=6717336/9585 in_queue=6726920, util=99.954987%

[과제 3]

AWS EBS를 PVC로 사용 후 온라인 볼륨 증가 후 관련 스샷 올려주세요

(repush:default) [root@kops-ec2 ~]# kubectl get pvc ebs-claim -o jsonpath={.spec.resources.requests.storage} ; echo
4Gi
(repush:default) [root@kops-ec2 ~]# kubectl get pvc ebs-claim -o jsonpath={.status.capacity.storage} ; echo
4Gi
(repush:default) [root@kops-ec2 ~]# kubectl exec -it app -- sh -c 'df -hT --type=ext4'
Filesystem     Type  Size  Used Avail Use% Mounted on
/dev/nvme2n1   ext4  9.8G   28K  9.7G   1% /data
/dev/root      ext4  124G  5.2G  119G   5% /etc/hosts
(repush:default) [root@kops-ec2 ~]# kubectl df-pv

 PV NAME                                   PVC NAME   NAMESPACE  NODE NAME            POD NAME  VOLUME MOUNT NAME   SIZE  USED  AVAILABLE  %USED  IUSED  IFREE   %IUSED
 pvc-84db7f09-0d5b-466e-9d4d-5f274c5a754c  ebs-claim  default    i-032553c87eb1da454  app       persistent-storage  9Gi   28Ki  9Gi        0.00   12     655348  0.00

[과제 4]

AWS Volume SnapShots 실습 후 관련 스샷 올려주세요

파일 내용 추가 저장 확인

(repush:default) [root@kops-ec2 ~]# kubectl exec app -- tail -f /data/out.txt
Tue Jan 31 15:06:28 UTC 2023
Tue Jan 31 15:06:33 UTC 2023
Tue Jan 31 15:06:38 UTC 2023
Tue Jan 31 15:06:43 UTC 2023

VolumeSnapshot 확인

(repush:default) [root@kops-ec2 ~]# kubectl get volumesnapshot ebs-volume-snapshot -o jsonpath={.status.boundVolumeSnapshotContentName}
snapcontent-8ce8a40f-db02-4099-bd14-9c9bdddebaa5

AWS EBS 스냅샷 확인

(repush:default) [root@kops-ec2 ~]# aws ec2 describe-snapshots --owner-ids self --query 'Snapshots[]' --output table
----------------------------------------------------------------------------------------------------------------
|                                               DescribeSnapshots                                              |
+-----------------+--------------------------------------------------------------------------------------------+
|  Description    |  Created by CreateImage(i-05ac6756498f91d2d) for ami-0a921992cb6eed04c                     |
|  Encrypted      |  False                                                                                     |
|  KmsKeyId       |                                                                                            |
|  OwnerId        |  442156947938                                                                              |
|  Progress       |  100%                                                                                      |
|  SnapshotId     |  snap-0733ac672a23a851b                                                                    |
|  StartTime      |  2022-12-17T13:46:29.477000+00:00                                                          |
|  State          |  completed                                                                                 |
|  StorageTier    |  standard                                                                                  |
|  VolumeId       |  vol-007a6e562984be055                                                                     |
|  VolumeSize     |  8                                                                                         |
+-----------------+--------------------------------------------------------------------------------------------+
|                                               DescribeSnapshots                                              |
+---------------+----------------------------------------------------------------------------------------------+
|  Description  |  Created by AWS EBS CSI driver for volume vol-007f9ce3bc07b94a2                              |
|  Encrypted    |  True                                                                                        |
|  KmsKeyId     |  arn:aws:kms:ap-northeast-2:442156947938:key/9fd996b1-affb-4042-9779-da9c68ec1ab5            |
|  OwnerId      |  442156947938                                                                                |
|  Progress     |  2%                                                                                          |
|  SnapshotId   |  snap-0997a66f81a5473f2                                                                      |
|  StartTime    |  2023-01-31T15:01:01.865000+00:00                                                            |
|  State        |  pending                                                                                     |
|  StorageTier  |  standard                                                                                    |
|  VolumeId     |  vol-007f9ce3bc07b94a2                                                                       |
|  VolumeSize   |  4                                                                                           |
+---------------+----------------------------------------------------------------------------------------------+
||                                                    Tags                                                    ||
|+-------------------------------------+----------------------------------------------------------------------+|
||                 Key                 |                                Value                                 ||
|+-------------------------------------+----------------------------------------------------------------------+|
||  ebs.csi.aws.com/cluster            |  true                                                                ||
||  kubernetes.io/cluster/repush.click |  owned                                                               ||
||  CSIVolumeSnapshotName              |  snapshot-8ce8a40f-db02-4099-bd14-9c9bdddebaa5                       ||
||  Name                               |  repush.click-dynamic-snapshot-8ce8a40f-db02-4099-bd14-9c9bdddebaa5  ||
||  KubernetesCluster                  |  repush.click                                                        ||
|+-------------------------------------+----------------------------------------------------------------------+|

강제로 장애 재현

(repush:default) [root@kops-ec2 ~]# kubectl delete pod app && kubectl delete pvc ebs-claim
pod "app" deleted
persistentvolumeclaim "ebs-claim" deleted
(repush:default) [root@kops-ec2 ~]# kubectl exec app -- tail -f /data/out.txt
Tue Jan 31 15:06:28 UTC 2023
Tue Jan 31 15:06:33 UTC 2023
Tue Jan 31 15:06:38 UTC 2023
Tue Jan 31 15:06:43 UTC 2023		<<<
Tue Jan 31 15:09:23 UTC 2023
Tue Jan 31 15:09:28 UTC 2023
Tue Jan 31 15:09:33 UTC 2023
Tue Jan 31 15:09:38 UTC 2023

0개의 댓글