AWS CodeBuild로 docker image build 하는 project 생성하는 방법

jb·2023년 3월 19일
CI/CDawscodebuild
1

AWS로 CICD 구축하기

목록 보기
1/1
post-thumbnail

배경과 목표

AWS CodeBuild를 활용하여 docker 빌드 환경을 만들어보자.

특히, AWS console을 활용해서 CodeCommit git repo를 기반으로 build한 image를 ECR에 PUSH 해보자.

전제

  • ECR private registry
  • CodeCommit repository
  • CodeBuild Project에서 사용할 service role 생성 권한
  • service role의 policy 설정 권한

개념

  • CodeBuild는 각 인스턴스를 project라고 함.
  • CodeBuild가 실제 build 작업을 수행하는데 따르는 명세는 buildspec.yml에 명시됨. buildspec.yml은 git repo 어느 곳에 있어도 상관 없음. 단, 기본적으로 root 경로를 바라보는데 만약, 다른 하위 폴더에 존재한다면 project를 생성할 때, buildspec 설정에서 Buildspec name을 경로로 지정해줘야 함. (e.g. /configuration/buildspec.yml) 그러므로 하나의 git repo에는 여러 개의 buildspec.yml이 있어도 가능함.
  • git repo에 push 했을 때 CodeBuild project가 trigger 되게 하기 위해서는 CodePipeline도 사용해야 함.

console에서 CodeBuild 생성

console → codebuild → build projects → create build project 에서 아래와 같은 설정으로 생성함.

  • project configuration
    • project name: jbpark-test-docker-build
  • source
    • source provider: AWS CodeCommit
    • repository:
    • reference type: Branch
    • branch : dev-build
  • environment
    • environment image: managed image
    • operating system: Ubuntu
    • runtime: standard
    • image: aws/codebuild/standard:4.0
    • image version: latest
    • environment type: Linux
    • privileged: enable
    • service role: new service role
      • Allow AWS CodeBuild to modify this service role so it can be used with this build project checked
    • role name:
    • environment variables
      • AWS_DEFAULT_REGION: ap-northeast-2
      • AWS_ACCOUNT_ID:
      • IMAGE_TAG: latest
      • IMAGE_REPO_NAME: jbpark-test-codebuild
  • logs
    • cloudwatch logs: enabled

service role과 policy 확인

role name: codebuild-jbpark-test-docker-build-2-service-role
trust relationship 

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "codebuild.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

CodeBuildBasePolicy-jbpark-test-docker-build

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Resource": [
                "arn:aws:logs:ap-northeast-2:<your aws account id>:log-group:/aws/codebuild/jbpark-test-docker-build-2",
                "arn:aws:logs:ap-northeast-2:<your aws account id>:log-group:/aws/codebuild/jbpark-test-docker-build-2:*"
            ],
            "Action": [
                "logs:CreateLogGroup",
                "logs:CreateLogStream",
                "logs:PutLogEvents"
            ]
        },
        {
            "Effect": "Allow",
            "Resource": [
                "arn:aws:s3:::codepipeline-ap-northeast-2-*"
            ],
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:GetObjectVersion",
                "s3:GetBucketAcl",
                "s3:GetBucketLocation"
            ]
        },
        {
            "Effect": "Allow",
            "Resource": [
                "arn:aws:codecommit:ap-northeast-2:<your aws account id>:dt-kf"
            ],
            "Action": [
                "codecommit:GitPull"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "codebuild:CreateReportGroup",
                "codebuild:CreateReport",
                "codebuild:UpdateReport",
                "codebuild:BatchPutTestCases",
                "codebuild:BatchPutCodeCoverages"
            ],
            "Resource": [
                "arn:aws:codebuild:ap-northeast-2:<your aws account id>:report-group/jbpark-test-docker-build-2-*"
            ]
        }
    ]
}

(옵션) inline policy push-to-ecr

{
  "Statement": [
    {
      "Action": [
        "ecr:BatchCheckLayerAvailability",
        "ecr:CompleteLayerUpload",
        "ecr:GetAuthorizationToken",
        "ecr:InitiateLayerUpload",
        "ecr:PutImage",
        "ecr:UploadLayerPart"
      ],
      "Resource": "*",
      "Effect": "Allow"
    }
  ],
  "Version": "2012-10-17"
}
  • ecr에 push하기 위한 권한을 추가함. 즉, ecr을 안 쓰고 dockerhub 등을 쓴다면 필요없는 policy임.
  • 위 예제는 Resource가 *로 설정되어 있는데 필요에 따라 사용할 ECR에 대한 ARN을 명시해주는게 더 좋음.

소스 생성

아래 코드를 / 경로에 생성 후에 commit하자.

Dockerfile

FROM golang:1.12-alpine AS build
#Install git
RUN apk add --no-cache git
#Get the hello world package from a GitHub repository
RUN go get github.com/golang/example/hello
WORKDIR /go/src/github.com/golang/example/hello
# Build the project and send the output to /bin/HelloWorld 
RUN go build -o /bin/HelloWorld

FROM golang:1.12-alpine
#Copy the build's output binary from the previous build container
COPY --from=build /bin/HelloWorld /bin/HelloWorld
ENTRYPOINT ["/bin/HelloWorld"]
  • helloworld를 출력하는 golang 코드를 빌드함.

buildspec.yml

version: 0.2

phases:
  pre_build:
    commands:
      - echo Logging in to Amazon ECR...
      - echo "IMAGE_REPO_NAME - $IMAGE_REPO_NAME"
      - echo "IMAGE_TAG - $IMAGE_TAG"
      - echo "AWS_ACCOUNT_ID - $AWS_ACCOUNT_ID"
      - echo "AWS_DEFAULT_REGION - $AWS_DEFAULT_REGION"
      - aws ecr get-login-password --region $AWS_DEFAULT_REGION | docker login --username AWS --password-stdin $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com
  build:
    commands:
      - echo Build started on `date`
      - echo Building the Docker image...
      - docker build -t $IMAGE_REPO_NAME:$IMAGE_TAG .
      - docker tag $IMAGE_REPO_NAME:$IMAGE_TAG $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/$IMAGE_REPO_NAME:$IMAGE_TAG      
  post_build:
    commands:
      - echo Build completed on `date`
      - echo Pushing the Docker image...
      - docker push $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/$IMAGE_REPO_NAME:$IMAGE_TAG
      - echo "updated line for codebuild ci"
  • pre_build: 설정한 환경변수를 출력하고 ecr에 대해 로그인하는 작업을 수행함.
  • build: 앞서 작성한 helloworld golang 코드를 docker를 사용하여 image로 build함.
  • post_build: build가 끝나면 앞서 로그인한 ecr에 push함.

CodeBuild에서 build project 생성, 실행, 확인

console → codebuild → build projects → 해당 프로젝트인 jbpark-test-docker-build 클릭 → 우측 상단에 Start build 클릭

로그 확인

[Container] 2023/03/18 13:36:53 Waiting for agent ping
2	[Container] 2023/03/18 13:36:54 Waiting for DOWNLOAD_SOURCE
3	[Container] 2023/03/18 13:37:01 Phase is DOWNLOAD_SOURCE
4	[Container] 2023/03/18 13:37:01 CODEBUILD_SRC_DIR=/codebuild/output/src632144123/src/git-codecommit.ap-northeast-2.amazonaws.com/v1/repos/<your codecommit repo name>
5	[Container] 2023/03/18 13:37:01 YAML location is /codebuild/output/src632144123/src/git-codecommit.ap-northeast-2.amazonaws.com/v1/repos/<your codecommit repo name>/buildspec.yml
6	[Container] 2023/03/18 13:37:01 Not setting HTTP client timeout for source type codecommit
7	[Container] 2023/03/18 13:37:01 Processing environment variables
8	[Container] 2023/03/18 13:37:02 No runtime version selected in buildspec.
9	[Container] 2023/03/18 13:37:04 Moving to directory /codebuild/output/src632144123/src/git-codecommit.ap-northeast-2.amazonaws.com/v1/repos/<your codecommit repo name>
10	[Container] 2023/03/18 13:37:04 Configuring ssm agent with target id: codebuild:c3605ca9-b3dc-4cba-9160-114066af3cf4
11	[Container] 2023/03/18 13:37:04 Successfully updated ssm agent configuration
12	[Container] 2023/03/18 13:37:04 Registering with agent
13	[Container] 2023/03/18 13:37:04 Phases found in YAML: 3
14	[Container] 2023/03/18 13:37:04  POST_BUILD: 4 commands
15	[Container] 2023/03/18 13:37:04  PRE_BUILD: 6 commands
16	[Container] 2023/03/18 13:37:04  BUILD: 4 commands
17	[Container] 2023/03/18 13:37:04 Phase complete: DOWNLOAD_SOURCE State: SUCCEEDED
18	[Container] 2023/03/18 13:37:04 Phase context status code:  Message: 
19	[Container] 2023/03/18 13:37:04 Entering phase INSTALL
20	[Container] 2023/03/18 13:37:04 Phase complete: INSTALL State: SUCCEEDED
21	[Container] 2023/03/18 13:37:04 Phase context status code:  Message: 
22	[Container] 2023/03/18 13:37:04 Entering phase PRE_BUILD
23	[Container] 2023/03/18 13:37:04 Running command echo Logging in to Amazon ECR...
24	Logging in to Amazon ECR...
25	
26	[Container] 2023/03/18 13:37:04 Running command echo "IMAGE_REPO_NAME - $IMAGE_REPO_NAME"
27	IMAGE_REPO_NAME - <your ECR registry name>
28	
29	[Container] 2023/03/18 13:37:04 Running command echo "IMAGE_TAG - $IMAGE_TAG"
30	IMAGE_TAG - latest
31	
32	[Container] 2023/03/18 13:37:04 Running command echo "AWS_ACCOUNT_ID - $AWS_ACCOUNT_ID"
33	AWS_ACCOUNT_ID - <your AWS account id>
34	
35	[Container] 2023/03/18 13:37:04 Running command echo "AWS_DEFAULT_REGION - $AWS_DEFAULT_REGION"
36	AWS_DEFAULT_REGION - <your AWS region name>
37	
38	[Container] 2023/03/18 13:37:04 Running command aws ecr get-login-password --region $AWS_DEFAULT_REGION | docker login --username AWS --password-stdin $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com
39	WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
40	Configure a credential helper to remove this warning. See
41	https://docs.docker.com/engine/reference/commandline/login/#credentials-store
42	
43	Login Succeeded
44	
45	[Container] 2023/03/18 13:37:18 Phase complete: PRE_BUILD State: SUCCEEDED
46	[Container] 2023/03/18 13:37:18 Phase context status code:  Message: 
47	[Container] 2023/03/18 13:37:18 Entering phase BUILD
48	[Container] 2023/03/18 13:37:18 Running command echo Build started on `date`
49	Build started on Sat Mar 18 13:37:18 UTC 2023
50	
51	[Container] 2023/03/18 13:37:18 Running command echo Building the Docker image...
52	Building the Docker image...
53	
54	[Container] 2023/03/18 13:37:18 Running command docker build -t $IMAGE_REPO_NAME:$IMAGE_TAG .
55	Sending build context to Docker daemon  22.58MB
56	
57	Step 1/8 : FROM golang:1.12-alpine AS build
58	1.12-alpine: Pulling from library/golang
59	c9b1b535fdd9: Pulling fs layer
60	cbb0d8da1b30: Pulling fs layer
61	d909eff28200: Pulling fs layer
62	665fbbf998e4: Pulling fs layer
63	4985b1919860: Pulling fs layer
64	665fbbf998e4: Waiting
65	4985b1919860: Waiting
66	d909eff28200: Verifying Checksum
67	d909eff28200: Download complete
68	cbb0d8da1b30: Download complete
69	c9b1b535fdd9: Verifying Checksum
70	c9b1b535fdd9: Download complete
71	c9b1b535fdd9: Pull complete
72	cbb0d8da1b30: Pull complete
73	d909eff28200: Pull complete
74	4985b1919860: Verifying Checksum
75	4985b1919860: Download complete
76	665fbbf998e4: Verifying Checksum
77	665fbbf998e4: Download complete
78	665fbbf998e4: Pull complete
79	4985b1919860: Pull complete
80	Digest: sha256:3f8e3ad3e7c128d29ac3004ac8314967c5ddbfa5bfa7caa59b0de493fc01686a
81	Status: Downloaded newer image for golang:1.12-alpine
82	 ---> 76bddfb5e55e
83	Step 2/8 : RUN apk add --no-cache git
84	 ---> Running in 8b68649601bb
85	fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/main/x86_64/APKINDEX.tar.gz
86	fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/community/x86_64/APKINDEX.tar.gz
87	(1/5) Installing nghttp2-libs (1.40.0-r1)
88	(2/5) Installing libcurl (7.79.1-r0)
89	(3/5) Installing expat (2.2.9-r1)
90	(4/5) Installing pcre2 (10.34-r1)
91	(5/5) Installing git (2.24.4-r0)
92	Executing busybox-1.31.1-r9.trigger
93	OK: 22 MiB in 20 packages
94	Removing intermediate container 8b68649601bb
95	 ---> b312e1cf6c8d
96	Step 3/8 : RUN go get github.com/golang/example/hello
97	 ---> Running in 2330d5cdcc0d
98	Removing intermediate container 2330d5cdcc0d
99	 ---> 7c7bdd3fd922
100	Step 4/8 : WORKDIR /go/src/github.com/golang/example/hello
101	 ---> Running in 4cbe3a5271fb
102	Removing intermediate container 4cbe3a5271fb
103	 ---> 427d12ba76ee
104	Step 5/8 : RUN go build -o /bin/HelloWorld
105	 ---> Running in 7f2f45c4ee93
106	Removing intermediate container 7f2f45c4ee93
107	 ---> 7cfea8434c29
108	Step 6/8 : FROM golang:1.12-alpine
109	 ---> 76bddfb5e55e
110	Step 7/8 : COPY --from=build /bin/HelloWorld /bin/HelloWorld
111	 ---> ab89ab67d678
112	Step 8/8 : ENTRYPOINT ["/bin/HelloWorld"]
113	 ---> Running in 7d6ab854f5ab
114	Removing intermediate container 7d6ab854f5ab
115	 ---> bd657498d35f
116	Successfully built bd657498d35f
117	Successfully tagged <your image repo name>:latest
118	
119	[Container] 2023/03/18 13:37:37 Running command docker tag $IMAGE_REPO_NAME:$IMAGE_TAG $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/$IMAGE_REPO_NAME:$IMAGE_TAG
120	
121	[Container] 2023/03/18 13:37:37 Phase complete: BUILD State: SUCCEEDED
122	[Container] 2023/03/18 13:37:37 Phase context status code:  Message: 
123	[Container] 2023/03/18 13:37:37 Entering phase POST_BUILD
124	[Container] 2023/03/18 13:37:37 Running command echo Build completed on `date`
125	Build completed on Sat Mar 18 13:37:37 UTC 2023
126	
127	[Container] 2023/03/18 13:37:37 Running command echo Pushing the Docker image...
128	Pushing the Docker image...
129	
130	[Container] 2023/03/18 13:37:37 Running command docker push $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/$IMAGE_REPO_NAME:$IMAGE_TAG
131	The push refers to repository [<your AWS account id>.dkr.ecr.<your AWS region name>.amazonaws.com/<your ECR registry name>]
132	33415f0de5df: Preparing
133	7306dca01e79: Preparing
134	3957f7032fc4: Preparing
135	12c4e92b2d48: Preparing
136	45182158f5da: Preparing
137	5216338b40a7: Preparing
138	5216338b40a7: Waiting
139	7306dca01e79: Layer already exists
140	3957f7032fc4: Layer already exists
141	45182158f5da: Layer already exists
142	12c4e92b2d48: Layer already exists
143	5216338b40a7: Layer already exists
144	33415f0de5df: Pushed
145	latest: digest: sha256:642e165f1cd961864281b241ca3ac04f06ca4354e8f667f5005a151dfce0345d size: 1576
146	
147	[Container] 2023/03/18 13:37:39 Running command echo "updated line for codebuild ci"
148	updated line for codebuild ci
149	
150	[Container] 2023/03/18 13:37:39 Phase complete: POST_BUILD State: SUCCEEDED
151	[Container] 2023/03/18 13:37:39 Phase context status code:  Message: 
152
  • buildspec.yml에 명시한 command들 수행에 따라 로깅이 잘 되어 있음.

기타

aws cli로 CodeBuild project 생성 샘플

$ aws codebuild create-project --cli-input-json
'{
  "name": "sample-docker-project",
  "source": {
    "type": "S3",
    "location": "codebuild-region-ID-account-ID-input-bucket/DockerSample.zip"
  },
  "artifacts": {
    "type": "NO_ARTIFACTS"
  },
  "environment": {
    "type": "LINUX_CONTAINER",
    "image": "aws/codebuild/standard:4.0",
    "computeType": "BUILD_GENERAL1_SMALL",
    "environmentVariables": [
      {
        "name": "AWS_DEFAULT_REGION",
        "value": "region-ID"
      },
      {
        "name": "AWS_ACCOUNT_ID",
        "value": "account-ID"
      },
      {
        "name": "IMAGE_REPO_NAME",
        "value": "Amazon-ECR-repo-name"
      },
      {
        "name": "IMAGE_TAG",
        "value": "latest"
      }
    ],
    "privilegedMode": true
  },
  "serviceRole": "arn:aws:iam::account-ID:role/role-name",
  "encryptionKey": "arn:aws:kms:region-ID:account-ID:key/key-ID"
}'
  • 위 예제는 s3를 소스로 사용함.

trust relationship 설정

  • CodeBuild에서 사용하는 service role은 trust relationship으로 codebuild 서비스 하나만 assumerole을 allow해야 함. ec2나 다른 서비스도 포함 되어 있다면 설정 변경 등에서 언제든 에러날 가능성이 있음.
profile
jb
기록하는 엔지니어 되기 💪

0개의 댓글