StatefulSet

application의 stateful을 관리하는데 사용하는 워크로드 API 오브젝트
파드들의 순서 및 고유성을 보장

pet vs cattle
고유성의 차이

headless service 와 statefullSet을 같이 사용 (headless service필수)

Headless Service

myweb-svc.yaml

apiVersion: v1
kind: Service
metadata:
  name: myweb-svc
spec:
  type: ClusterIP
  selector:
    app: web
  ports:
    - port: 80
      targetPort: 8080

myweb-svc-headless.yaml

apiVersion: v1
kind: Service
metadata:
  name: myweb-svc-headless
spec:
  type: ClusterIP
  clusterIP: None # <-- Headless Service
  selector:
    app: web
  ports:
    - port: 80
      targetPort: 8080

myweb-rs.yaml

apiVersion: apps/v1
kind: ReplicaSet
metadata:
  name: myweb-rs
spec:
  replicas: 3
  selector:
    matchLabels:
      app: web
      env: dev
  template:
    metadata:
      labels:
        app: web
        env: dev
    spec:
      containers:
        - name: myweb
          image: ghcr.io/c1t1d0s7/go-myweb
          ports:
            - containerPort: 8080
              protocol: TCP

kubectl run nettool -it --image ghcr.io/c1t1d0s7/network-multitool --rm

> host myweb-svc
> host myweb-svc-headless 

StatefulSet

예제1

myweb-svc-headless.yaml

apiVersion: v1
kind: Service
metadata:
  name: myweb-svc-headless
spec:
  type: ClusterIP
  clusterIP: None # <-- Headless Service
  selector:
    app: web
  ports:
    - port: 80
      targetPort: 8080

myweb-sts.yaml

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: myweb-sts
spec:
  replicas: 3
  serviceName: myweb-svc-headless
  selector:
    matchLabels:
      app: web
      env: dev
  template:
    metadata:
      labels:
        app: web
        env: dev
    spec:
      containers:
        - name: myweb
          image: ghcr.io/c1t1d0s7/go-myweb
          ports:
            - containerPort: 8080
              protocol: TCP

kubectl run nettool -it --image ghcr.io/c1t1d0s7/network-multitool --rm

> host myweb-svc-headless
> host myweb-sts-0.myweb-svc-headless
> host myweb-sts-1.myweb-svc-headless
> host myweb-sts-2.myweb-svc-headless

pod가 삭제 되어도 똑같은 이름으로 다시 생성
이름이 고정적(이름뒤에 서수가 붙음 -예측가능)
순서대로 생성되고 삭제됨

• template: pod생성
• volumeClaimTemplates: 볼륨 pvc생성

예제2: PVC 템플릿

myweb-sts-vol.yaml

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: myweb-sts-vol
spec:
  replicas: 3
  serviceName: myweb-svc-headless
  selector:
    matchLabels:
      app: web
      env: dev
  template:
    metadata:
      labels:
        app: web
        env: dev
    spec:
      containers:
        - name: myweb
          image: ghcr.io/c1t1d0s7/go-myweb:alpine
          ports:
            - containerPort: 8080
              protocol: TCP
          volumeMounts:
            - name: myweb-pvc
              mountPath: /data
  volumeClaimTemplates:
    - metadata:
        name: myweb-pvc
      spec:
        accessMode:
          - ReadWriteOnce
        resources:
          requests:
            storage: 1G
        storageClassName: nfs-client

pod가 삭제되어도 pv,pvc는 보존

예제3: mysql

참고링크
https://kubernetes.io/docs/tasks/run-application/run-replicated-stateful-application/

configMap

wget https://k8s.io/examples/application/mysql/mysql-configmap.yaml
(바로 시작 안하고 받아서 수정 후 apply)

primaty.cnf 와 replica.cnf의 
datadir 라인 제거 

kubectl create -f mysql-configmap.yaml

service

kubectl apply -f https://k8s.io/examples/application/mysql/mysql-services.yaml

statefulset

kubectl apply -f https://k8s.io/examples/application/mysql/mysql-statefulset.yaml

kubectl get sts,po,pv,pvc

넷툴로 접속 (--rm 을 사용하면 종료 시 자동으로 삭제됨)

kubectl run nettool -it --image ghcr.io/c1t1d0s7/network-multitool --rm

host확인

host mysql
host mysql-read

node0 master확인 (database 추가)

mysql -h mysql-0.mysql -u root
(0번이 master )
show databases;
create database encore;
exit

node1 (database 확인, 동기화)

mysql -h mysql-1.mysqp -u root
show databases;
(encore 존재 확인)
exit

(database 삭제, 동기화)

mysql -h mysql-0.mysql -u root
drop databases eoncore;
show databases;

mysql -h mysql-1.mysqp -u root
show databases;
(encore 존재 확인)
exit

mysql -h mysql-0.mysql -u root
create database encore;
use database encore;
use encore;
create table encore.message (message VARCHAR(50));
show tables;
insert into encore.message values ("hello mysql");
select * from message;
exit

다른 pod를 생성하여도 볼륨에 바로 동기화가 됨

---

Auto Scaling

Resource Request & Limit

요청: request
제한: limit

요청 =< 제한

QoS(서비스 품질) Class:
1. BestEffort: 가장 나쁨
2. Burstable
3. Guaranteed: 가장 좋음

• 요청/제한 설정되어 있지 않으면: BestEffort
• 요청 < 제한: Bustable
• 요청 = 제한: Guaranteed

pod.spec.containers.resources

• requests
  - cpu
  - memory
• limits
  - cpu
  - memory

CPU 요청 & 제한: milicore
ex) 1500m -> 1.5개, 1000m -> 1개
ex) 1.5, 0.1
Memory 요청 & 제한: M, G, T, Mi, Gi, Ti

myweb-reqlim.yaml

apiVersion: v1
kind: Pod
metadata:
  name: myweb-reqlim
spec:
  containers:
    - name: myweb
      image: ghcr.io/c1t1d0s7/go-myweb
      resources:
        requests:
          cpu: 200m
          memory: 200M
        limits:
          cpu: 200m
          memory: 200M

replace는 resource 변경이 안되나 뒤에 --force를 붙여 강제로 가능 (삭제하고 재생성)
제한(limit)만 설정하면 요청(request)에 같은 값이 설정됨
(요청만 설정하면 제한은 설정이 안됨)

노드별 CPU/Memory 사용량 확인

kubectl top nodes

파드별 CPU/Memory 사용량 확인

kubectl top pods
kubectl top pods -A

리소스 모니터링(인프라 모니터링)
Heapster:
-> metric-server: 실시간 cpu/memory 모니터링
-> prometheus: 실시간/이전 cpu/memory/network/disk 모니터링

노드별 요청/제한 양 확인

kubectl describe nodes node1

실행 할 수 없는 리소스
myweb-big.yaml

apiVersion: v1
kind: Pod
metadata:
  name: myweb-big
spec:
  containers:
    - name: myweb
      image: ghcr.io/c1t1d0s7/go-myweb
      resources:
        limits:
          cpu: 3000m
          memory: 4000M

pending 이 오래걸리면 스케줄링이 되지 않는지
볼륨이 연결되지않는지, 이미지를 받지 못하는지 의심할 것

HPA: Horisontal Pod AutoScaler

AutoScaling

• Pod
  - HPA
  - VPA: Vertical Pod Autoscaler
• Node
  - ClusterAutoScaler

HPA: Deployment, ReplicaSet, StatefulSet의 복제본 개수를 조정

스케일 아웃: 180초
스케인 인: 300초

myweb-deploy.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: myweb-deploy
spec:
  replicas: 2
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      containers:
        - name: myweb
          image: ghcr.io/c1t1d0s7/go-myweb:alpine
          ports:
            - containerPort: 8080
          resources:
            requests:
              cpu: 200m
            limits:
              cpu: 200m

HPA를 위해 최소 request는 설정되여 함

myweb-hpa.yaml

apiVersion: autoscaling/v1
kind: HorizontalPodAutoscaler
metadata:
  name: myweb-hpa
spec:
  minReplicas: 1
  maxReplicas: 10
  targetCPUUtilizationPercentage: 50
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: myweb-deploy

부하 (강제로 부하걸기)

kubectl exec <POD> -- sha256sum /dev/zero
kubectl exec myweb-deploy-xx~ --sha256sum /dev/zero

원하는 레플리카수 = ceil[ 현재 레플리카 수 * (현재 메트릭 값/원하는 메트릭 값)]
(ceil -> celing천장함수-> 올림)

---

beta2버전
myweb-hpa-v2beta2.yaml

apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
  name: myweb-hpa
spec:
  minReplicas: 1
  maxReplicas: 10
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          avarageUtilization: 50
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: myweb-deploy