Jenkins & Argocd & Gitlab & Kustomize & Harbor (feat. 사설 인증서)

1. Jenkins

설치

helm repo add jenkinsci https://charts.jenkins.io
helm repo update

helm show values jenkinsci/jenkins > values.yaml

vi values.yaml

# 서비스 변경
serviceType: NodePort
...
...

helm install jenkins -n ci jenkinsci/jenkins -f values.yaml

비밀번호

kubectl exec --namespace ci -it svc/jenkins -c jenkins -- /bin/cat /run/secrets/additional/chart-admin-password && echo

Credentials

gitlab Personal Access Token

Username and Password (gitlab)

Username and Password (Harbor)

세개를 등록해준다.
만약 gitlab Personal Access Token 입력할 수 없다면
gitlab 플러그인을 설치하자.

Pipeline

• Secret 토큰과 Webhook url을 추후 Gitlab에 입력해야함.

Gitlab webhook URL문구 뒤에 주소가 나오는데 해당 주소를 기억해야한다.

Build Triggers 안에 Advanced를 누르면 Secret Token이 나오는데
Generate버튼을 눌러 토큰 발급하고 기억해야한다.

• docker config

echo harbor_user:harbor_password | base64

>>> incoded_string


vi config.json

{
  "auths": {
    "harbor domain: {
      "auth": "incoded_string"
    }
  }
}

cat config.json | base64

>>> incoded_string

• secret

apiVersion: v1
kind: Secret
metadata:
  name: docker-config
  namespace: ci
type: Opaque
data:
  config.json: incoded_string

• Jenkinsfile

pipeline {
    agent {
        kubernetes {
            // Kaniko 컨테이너용 Pod 템플릿 정의
            yaml '''
apiVersion: v1
kind: Pod
metadata:
  name: kaniko-agent
  namespace: ci
  labels:
    app: kaniko
spec:
  hostAliases:
  - ip: <harbor ip>
    hostnames:
    - "<harbor domain>"
  containers:
  - name: kaniko
    image: gcr.io/kaniko-project/executor:debug
    tty: true
    command:
    - /busybox/cat
    volumeMounts:
    - name: docker-config
      mountPath: /kaniko/.docker
    - name: ca
      mountPath: /kaniko/ssl/certs/ca.crt
    - name: registry-cert
      mountPath: /kaniko/ssl/certs/<harbor_domain>.key
    - name: registry-key
      mountPath: /kaniko/ssl/certs/<harbor_domain>.cert
    - name: workspace-volume
      mountPath: /workspace
  volumes:
  - name: docker-config
    secret:
      secretName: docker-config
  - name: kaniko-secret
    secret:
      secretName: regcred
  - name: ca
    hostPath:
      path: harbor_ca.crt
      type: FileOrCreate
  - name: registry-cert
    hostPath:
      path: <harbor_domain>.cert
      type: FileOrCreate
  - name: registry-key
    hostPath:
      path: <harbor_domain>.key
      type: FileOrCreate
  - name: workspace-volume
    emptyDir: {}
'''
        }
    }
    environment {
        HARBOR_REGISTRY = 'harbor_domain'
        IMAGE_NAME = 'image name'            
        TAG = "${env.GIT_COMMIT}"
    }
    stages {
        stage('Build and Push Image') {
            steps {
                container('kaniko') {
                    script {
                        def context = "."
                        def dockerfile = "docker file path"
                        def image = "${HARBOR_REGISTRY}/${IMAGE_NAME}:${TAG}"

                        sh "echo Building and pushing image to ${image}"
                        sh "/kaniko/executor --context ${context} --dockerfile ${dockerfile} --destination ${image} --verbosity=debug"
                    }
                }
            }
        }
        stage('Pull git repository') {
            steps {
                withCredentials([usernamePassword(credentialsId: 'gitlab-login', usernameVariable: 'GIT_USERNAME', passwordVariable: 'GIT_PASSWORD')]) {
                    git branch: 'main', url: 'http://<gitlab repo>', credentialsId: 'gitlab-login'
                }
            }
        }
        stage('Install kustomize') {
            steps {
                sh """
                    if [ ! -f "${env.WORKSPACE}/kustomize" ]; then
                        curl -s "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh" | bash
                    fi
                    ${env.WORKSPACE}/kustomize version
                """
            }
        }

        stage('Change imag tag') {
            steps {
                sh """
                    cd overlays/prod
                    ${env.WORKSPACE}/kustomize edit set image argocd-image=${HARBOR_REGISTRY}/pma/${IMAGE_NAME}:${TAG}
                    ${env.WORKSPACE}/kustomize build .
                """
            }
        }

        stage('Push to manifest repo') {
            steps {
                script {
                    withCredentials([usernamePassword(credentialsId: 'gitlab-login', passwordVariable: 'gitPassword', usernameVariable: 'gitUsername')]) {
                        def encodedPassword = URLEncoder.encode("$gitPassword",'UTF-8')
                        sh """
                            git config user.email "<gitlab email>"
                            git config user.name ${env.gitUsername}
                            git add -A
                            git commit -m '[jenkins] update image tag = ${HARBOR_REGISTRY}/${IMAGE_NAME}:${TAG}'
                            git push http://${env.gitUsername}:${encodedPassword}@<gitlab manifest repo>
                        """
                    }
                }

            }
        }
    }

    post {
        always {
            cleanWs()
        }
    }
}

2. Gitlab Webhook 설정

레포지토리 별로 웹훅을 설정할 수 있기에 원하는 레포에 가서
Settings > webhook > Add new webhook으로 등록한다.

• 이때, URL은 위에서 받은 웹훅 URL,
• Secret token도 위에서에서 발급한 Secret 토큰을 입력한다.
  

3. Argocd 설치

kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

• 초기 비밀번호

kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath=``"{.data.password}" | base64 -d; echo

프로젝트 설정

Allow List는 필요한 것들 허용해주기

Repo 설정

Gitlab이나 github 연동 설정 해주기 (SSH or HTTPS with PAT)

클러스터 상태가 Unknown인 이유는

Application을 만들지 않았기 때문이다.