harbor 레지스트리 설치과정 -1

하버 레지스트리가 컨테이너 보안, RBAC, https 지원 레지스트리 UI도 좋아서 도입하게 됐습니다.
더 찾아보니 복제 기능도 있어서 private docker registry의 이미지도 옮길 계획 중.
개발환경은 온프레미스 환경에서 적용시킨 것이기 때문에 AWS, Azure, GCP와 같은 클라우드인 경우 Ingress 설정 옵션이 다를 수 있음을 알아주셨으면 합니다.

설치과정

helm chart 파일 받기

helm repo add harbor https://helm.goharbor.io
helm fetch harbor/harbor --untar
#폴더 확인
cd harbor
ls 
#목록 확인
Chart.yaml	conf	LICENSE	README.md	templates	values.yaml

Install 하기 전에 해둬야할 것

#Harbor 네임스페이스 생성
kubectl create ns harbor

#Harbor TLS 등록
kubectl create secret tls harbor-tls --key path/key --cert  path/crt --dry-run=client o yaml > tls–domain.yaml

ip 작성란은 xxx.xxx.xxx.xxx로 표기 양해부탁드립니다.

PV.yaml

apiVersion: v1
kind: PersistentVolume
metadata:
  name: harbor-registry
  namespace: harbor
spec:
  capacity:
    storage: 10Gi
  volumeMode: Filesystem
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: nas-storage
  nfs:
    server: xxx.xxx.xxx.xxx
    path: /path/registry
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: harbor-jobservice
  namespace: harbor
spec:
  capacity:
    storage: 5Gi
  volumeMode: Filesystem
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: nas-storage
  nfs:
    server: xxx.xxx.xxx.xxx
    path: /path/jobservice
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: harbor-database
  namespace: harbor
spec:
  capacity:
    storage: 5Gi
  volumeMode: Filesystem
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: nas-storage
  nfs:
    server: xxx.xxx.xxx.xxx
    path: /path/database
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: harbor-redis
  namespace: harbor
spec:
  capacity:
    storage: 5Gi
  volumeMode: Filesystem
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: nas-storage
  nfs:
    server: xxx.xxx.xxx.xxx
    path: /path/redis
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: harbor-trivy
  namespace: harbor
spec:
  capacity:
    storage: 5Gi
  volumeMode: Filesystem
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: nas-storage
  nfs:
    server: xxx.xxx.xxx.xxx
    path: /path/trivy

PVC.yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  annotations:
    helm.sh/resource-policy: keep
  labels:
    app: harbor
    component: database
  name: harbor-database
  namespace: harbor
spec:
  storageClassName: nas-storage
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi
  volumeMode: Filesystem
  volumeName: harbor-database
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  annotations:
    helm.sh/resource-policy: keep
  labels:
    app: harbor
    component: jobservice
  name: harbor-jobservice
  namespace: harbor
spec:
  storageClassName: nas-storage
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi
  volumeMode: Filesystem
  volumeName: harbor-jobservice
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  annotations:
    helm.sh/resource-policy: keep
  labels:
    app: harbor
    component: registry
  name: harbor-registry
  namespace: harbor
spec:
  storageClassName: nas-storage
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi
  volumeMode: Filesystem
  volumeName: harbor-registry
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  annotations:
    helm.sh/resource-policy: keep
  labels:
    app: harbor
    component: redis
  name: harbor-redis
  namespace: harbor
spec:
  storageClassName: nas-storage
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi
  volumeMode: Filesystem
  volumeName: harbor-redis
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  annotations:
    helm.sh/resource-policy: keep
  labels:
    app: harbor
    component: trivy
  name: harbor-trivy
  namespace: harbor
spec:
  storageClassName: nas-storage
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi
  volumeMode: Filesystem
  volumeName: harbor-trivy

SC.yaml

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: nas-storage
provisioner: nas-storage
parameters:
  server: xxx.xxx.xxx.xxx
  path: /path
  readOnly: "false"

PV, PVC ,SC 세팅과정에서 스토리지 클래스가 일치하지않으면 PVC는 pending상태가 되며
정상적으로 생성이 되면 Bound 상태여야 함

kubectl get pv,pvc -n harbor

라인	의미	수정예시
4	인그레스 타입	type: ingress
28 33	secret TLS 명칭	secretname:“harbor-tls” notarySecretName:“harbor-tls”
36 37	host 값	ingress.hosts.core: harbor.domain.com ingress.hosts.notary: notary.harbor.domain.com
47	ingressClass값	className: "nginx"
127	외부에서 접속가능한 URL	externalURL: https://harbor.domain.com
208	PVC 연결	existingClaim: "harbor-registry"
219		existingClaim: "harbor-jobservice"
228		existingClaim: "harbor-databse"
237		existingClaim: "harbor-redis"
244		existingClaim: "harbor-trivy"
362	초기비밀번호 설정	harborAdminPassword: "my_password"
415 437 474 550 597 714 747 779	노드 지정	nodeSelector.kubernetes.io/hostname: hostname
440 476 552 599 716 749 781	Affinity 설정	아래참고

affinity: 
      podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchExpressions:
              - key: component
                operator: In
                values:
                - notary-signer
            topologyKey: kubernetes.io/hostname        

설치준비 끝

#values.yaml이 있는 곳에서 helm install (7분소요)
helm install harbor . -n harbor 

이제 DNS 추가하고 접속 시도 해보세요!

✔/etc/docker/daemon.json 수정해야함

{
  "storage-driver": "overlay2",
        "storage-opts": [
          "overlay2.override_kernel_check=true"
        ]
  "insecure-registries": ["harbor.domain.com"]
}

위와 같이 설정을 해줘야 나중에 CI/CD tool에서 실패를 안하며 push가 정상적으로 동작함
이 때 core.harbor.domain과 externalURL이 일치해야할 필요가 있음

참고문서

values.yaml 문서 작성 참고사이트
values.yaml 및 registry 설정 문서 작성 참고 사이트
노드 셀렉터 - values.yaml에서 적용하는 법