Kubernetes install on Ubuntu

• Kubernetes version: 1.29
• Ubuntu version: 20.04 LTS(HVM), SSD Volume Type
• Kubernetes 포트: https://kubernetes.io/ko/docs/reference/networking/ports-and-protocols
  (네트워크, 보안 설정 시 참고)
  

Master/Worker 공통

---

hostname 설정

# master mode
sudo hostnamectl set-hostname master01
# worker node
sudo hostnamectl set-hostname worker01

sudo vi /etc/hosts
# IP HOSTNAME 설정 

update

sudo apt update
sudo apt -y upgrade

swap-off 설정

sudo swapoff -a && sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab

방화벽 헤제 설정

sudo ufw disable

커널 파라미터 설정

sudo tee /etc/modules-load.d/containerd.conf <<EOF
overlay
br_netfilter
EOF

sudo modprobe overlay
sudo modprobe br_netfilter

sudo tee /etc/sysctl.d/kubernetes.conf <<EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
EOF

sudo sysctl --system

containerd 설치

sudo apt-get update

sudo apt install -y \
    curl \
    gnupg2 \
    software-properties-common \
    apt-transport-https \
    ca-certificates

sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmour -o /etc/apt/trusted.gpg.d/docker.gpg
sudo add-apt-repository -y "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"

sudo apt update
sudo apt install -y containerd.io

containerd config default | sudo tee /etc/containerd/config.toml >/dev/null 2>&1
sudo sed -i 's/SystemdCgroup \= false/SystemdCgroup \= true/g' /etc/containerd/config.toml

sudo systemctl restart containerd
sudo systemctl enable containerd

cat /etc/containerd/config.toml | grep SystemdCgroup
sudo systemctl status containerd

kubeadm 설치

Note: The legacy package repositories (apt.kubernetes.io and yum.kubernetes.io) have been deprecated and frozen starting from September 13, 2023. Using the new package repositories hosted at pkgs.k8s.io is strongly recommended and required in order to install Kubernetes versions released after September 13, 2023. The deprecated legacy repositories, and their contents, might be removed at any time in the future and without a further notice period. The new package repositories provide downloads for Kubernetes versions starting with v1.24.0.

Note: There's a dedicated package repository for each Kubernetes minor version. If you want to install a minor version other than v1.29, please see the installation guide for your desired minor version.

kubernetes 패키지 레포가 deprecated되면서, 설치 방식이 조금 변경이 있다. kubernetes 공식 문서 를 참조하자.

1. Update the apt package index and install packages needed to use the Kubernetes apt repository:

sudo apt-get update
# apt-transport-https may be a dummy package; if so, you can skip that package
sudo apt-get install -y apt-transport-https ca-certificates curl gpg

2. Download the public signing key for the Kubernetes package repositories. The same signing key is used for all repositories so you can disregard the version in the URL:

# If the directory `/etc/apt/keyrings` does not exist, it should be created before the curl command, read the note below.
# sudo mkdir -p -m 755 /etc/apt/keyrings
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.29/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg

3. Add the appropriate Kubernetes apt repository. Please note that this repository have packages only for Kubernetes 1.29; for other Kubernetes minor versions, you need to change the Kubernetes minor version in the URL to match your desired minor version (you should also check that you are reading the documentation for the version of Kubernetes that you plan to install).

# This overwrites any existing configuration in /etc/apt/sources.list.d/kubernetes.list
echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.29/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list

4. Update the apt package index, install kubelet, kubeadm and kubectl, and pin their version:

sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl

Master 구성

---

sudo kubeadm init --control-plane-endpoint "10.0.xxx.xxx:6443" --upload-certs

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

You can now join any number of the control-plane node running the following command on each as root:

  kubeadm join 10.0.xxx.xxx:6443 --token m8dtlr.xvfiuyywapeb4mic \
        --discovery-token-ca-cert-hash sha256:e5c661b48c8028e03bff7ba01c17a6ba726b3dddc70ac64c60a1de3e1e8c672a \
        --control-plane --certificate-key 28e28f092b2b6100a317a687eb6557edf1dace61a45c33a8aa76f071883f371b

Please note that the certificate-key gives access to cluster sensitive data, keep it secret!
As a safeguard, uploaded-certs will be deleted in two hours; If necessary, you can use
"kubeadm init phase upload-certs --upload-certs" to reload certs afterward.

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 10.0.xxx.xxxx:6443 --token m8dtlr.xvfiuyywapeb4mic \
        --discovery-token-ca-cert-hash sha256:e5c661b48c8028e03bff7ba01c17a6ba726b3dddc70ac64c60a1de3e1e8c672a

Worker 노드 구성

---

kubeadm join 10.0.xxx.xxx:6443 --token pom78r.fozl885tzlwzq0cg \
        --discovery-token-ca-cert-hash sha256:aeef26ae4c1dff61e6c65b7484e9b91d9fd3bc14a29209cb324f32799255a88b

CNI(calico) 구성

---

kubectl apply -f https://raw.githubusercontent.com/projectcalico/calico/v3.25.0/manifests/calico.yaml

노드 상태 확인

kubectl get nodes
NAME       STATUS   ROLES           AGE     VERSION
master01   Ready    control-plane   15m     v1.29.3
worker01   Ready    <none>          3m13s   v1.29.3
worker02   Ready    <none>          3m8s    v1.29.3

pod 정상 배포 상태 확인

kubectl get pods -n kube-system
NAME                                       READY   STATUS    RESTARTS   AGE     IP               NODE       NOMINATED NODE   READINESS GATES
calico-kube-controllers-658d97c59c-v6vrw   1/1     Running   0          59s     192.168.241.66   master01   <none>           <none>
calico-node-2j9ff                          1/1     Running   0          59s     10.0.3.158       worker01   <none>           <none>
calico-node-d4ckn                          1/1     Running   0          59s     10.0.15.44       worker02   <none>           <none>
calico-node-hlg4b                          1/1     Running   0          59s     10.0.8.164       master01   <none>           <none>
coredns-76f75df574-9m2nd                   1/1     Running   0          15m     192.168.241.65   master01   <none>           <none>
coredns-76f75df574-dx9zk                   1/1     Running   0          15m     192.168.241.67   master01   <none>           <none>
etcd-master01                              1/1     Running   0          15m     10.0.8.164       master01   <none>           <none>
kube-apiserver-master01                    1/1     Running   0          15m     10.0.8.164       master01   <none>           <none>
kube-controller-manager-master01           1/1     Running   0          15m     10.0.8.164       master01   <none>           <none>
kube-proxy-b7dlm                           1/1     Running   0          2m52s   10.0.15.44       worker02   <none>           <none>
kube-proxy-cdnx8                           1/1     Running   0          2m57s   10.0.3.158       worker01   <none>           <none>
kube-proxy-cqlgp                           1/1     Running   0          15m     10.0.8.164       master01   <none>           <none>
kube-scheduler-master01                    1/1     Running   0          15m     10.0.8.164       master01   <none>           <none>