Post

Ansible로 Kubernetes Cluster 구축하기

Posted Updated
Preview Image
By KKamJi
views 19 min read
Ansible로 Kubernetes Cluster 구축하기
Ansible playbook은 control plane을 먼저 만들고 worker를 join시킵니다

control plane 초기화가 만든 join token을 worker 단계에서 사용하므로 순서가 고정됩니다.

Inventory control plane과 worker group을 나눠 둡니다.
대상 group 순서 결정
Ansible control node bootstrap play를 실행하고 결과를 모읍니다.
kubeadm init
Control plane 먼저 초기화하고 worker join token을 생성합니다.
token으로 join
Worker nodes control plane이 발급한 token으로 cluster에 합류합니다.
노드 준비 완료
Kubernetes cluster workload를 schedule할 수 있는 상태가 됩니다.

PNG 다운로드 원본 이미지 열기

저번 포스트에서 Ansible의 기본 개념, 사용 방법에 대해 다뤘습니다. 이번 시간에는 Ansible과 Kubeadm을 사용해 Kubernetes Cluster를 구축하는 과정을 다루겠습니다.

전체 코드 - https://github.com/KKamJi98/Ansible-Kubernetes

WSL => Ubuntu 24.04(LTS), RAM 8GB, 172.28.8.28/20
Master Node => Ubuntu 24.04(LTS), RAM 6GB, 10.0.0.100/24, Kubernetes v1.29
Worker Nodes => Ubuntu 24.04(LTS), RAM 4GB, 10.0.0.200-201/24, Kubernetes v1.29

TL;DR

  • kubeadm 클러스터 구축을 swap 해제, containerd 설치, kubeadm/kubelet/kubectl 설치, 클러스터 초기화, Calico 설치, worker join 순서로 Ansible playbook에 옮깁니다.
  • CLI 절차를 그대로 코드로 옮기는 작업이 아니었습니다. 실패했을 때 되돌리는 방법, 설정이 반영됐는지 확인하는 방법, join token을 넘기는 방법을 playbook 안에서 따로 설계해야 했습니다.
  • 전체 코드는 KKamJi98/Ansible-Kubernetes 저장소에 있습니다.

1. 사전 준비물

1.1. Master Node 준비

CPU - 코어 2개 이상
RAM - 2GB 이상
Storage - 8GB 이상 여유
Ansible 호스트 & 인터넷 통신 가능

1.2. Worker Nodes 준비

CPU - 코어 1개 이상
RAM - 1GB 이상
Storage - 8GB 이상 여유
Ansible 호스트 & 인터넷 통신 가능

2. 실습 계획

참고자료 - https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/

Kubeadm을 사용하여 쿠버네티스 클러스터를 구축하는 과정은 다음과 같습니다.

  1. Master, Worker 스왑 메모리 해제
  2. CRI(Container Runtime Interface) - Containerd 설치
    1. Overlay Network를 위한 iptables 구성
    2. Containerd 설치
    3. systemd cgroup driver 구성
  3. kubeadm, kubelet, kubectl 설치
  4. 클러스터 초기화 [In Master Node]
  5. 네트워크 플러그인 설치(Calico) [In Master Node]
  6. 워커노드 추가 [In Worker Nodes]

3. Inventory 구성 및 통신 확인

3.1. Inventory 구성

1
2
3
4
5
6
7
8
9
10
11
12

❯ cat /etc/ansible/hosts
[worker_nodes]
10.0.0.201 # worker01
10.0.0.202 # worker02

[master_nodes]
10.0.0.100 # master01

[nodes]
10.0.0.201
10.0.0.202
10.0.0.100

3.2. Inventory 구성 요소 내 통신 확인

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

❯ ansible all -m ping
10.0.0.201 | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python3"
    },
    "changed": false,
    "ping": "pong"
}
10.0.0.100 | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python3"
    },
    "changed": false,
    "ping": "pong"
}
10.0.0.202 | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python3"
    },
    "changed": false,
    "ping": "pong"
}

4. Master, Worker Nodes 스왑 메모리 해제

스왑 메모리는 물리적 메모리(RAM)가 부족할 때 디스크를 임시 메모리로 사용하는 기능입니다. Kubernetes에서는 성능, 안정성 문제 등 여러 가지 문제를 방지하기 위해 스왑 메모리를 비활성화하는 것을 권장합니다. 또한 기본적으로 스왑 메모리가 활성화되어 있으면 kubelet이 동작하지 않도록 되어 있습니다. cgroup v2를 사용하면 스왑 메모리를 사용할 수 있지만 해당 기능은 아직 베타 기능이며 사용을 추천하지 않습니다.

4.1. 플레이북 구성 (1-swap-off-playbook.yaml)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

- name: Disable all nodes swap memory
  hosts: nodes
  become: true
  tasks:
    - name: Gather facts
      ansible.builtin.setup:

    - name: Disable swap
      ansible.builtin.command: swapoff -a
      when: ansible_swaptotal_mb > 0

    - name: Ensure swap is disabled in /etc/fstab
      ansible.builtin.replace:
        path: /etc/fstab
        regexp: '^(.+\s+swap\s+.+)$'
        replace: '# \1'

4.2. 플레이북 실행

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

❯ ansible-playbook swap-off-playbook.yaml

PLAY [Disable all nodes swap memory] *************************************************************************************************************************************************************

TASK [Gathering Facts] ***************************************************************************************************************************************************************************
ok: [10.0.0.202]
ok: [10.0.0.100]
ok: [10.0.0.201]

TASK [Gather facts] ******************************************************************************************************************************************************************************
ok: [10.0.0.201]
ok: [10.0.0.202]
ok: [10.0.0.100]

TASK [Disable swap] ******************************************************************************************************************************************************************************
changed: [10.0.0.201]
changed: [10.0.0.202]
changed: [10.0.0.100]

TASK [Ensure swap is disabled in /etc/fstab] *****************************************************************************************************************************************************
changed: [10.0.0.100]
changed: [10.0.0.201]
changed: [10.0.0.202]

PLAY RECAP ***************************************************************************************************************************************************************************************
10.0.0.100              : ok=4    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
10.0.0.201              : ok=4    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
10.0.0.202              : ok=4    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

5. CRI(Container Runtime Interface) - Containerd 설치

5.1. Network 구성

5.1.1. 플레이북 구성 (2-install-containerd-playbook.yaml)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

- name: Install containerd
  hosts: nodes
  become: true
  tasks:
    ## Step 2.1
    - name: Gather facts
      ansible.builtin.setup:

    - name: Load overlay module
      ansible.builtin.shell: modprobe overlay
      register: overlay_result

    - name: Load br_netfilter module
      ansible.builtin.shell: modprobe br_netfilter
      register: br_netfilter_result

    - name: Ensure sysctl params are set for Kubernetes
      ansible.builtin.copy:
        dest: /etc/sysctl.d/k8s.conf
        content: |
          net.bridge.bridge-nf-call-iptables  = 1
          net.bridge.bridge-nf-call-ip6tables = 1
          net.ipv4.ip_forward                 = 1

    - name: Apply sysctl params without reboot
      ansible.builtin.shell: sysctl --system

    - name: Verify br_netfilter module is loaded
      ansible.builtin.shell: lsmod | grep br_netfilter
      register: br_netfilter_check
      failed_when: br_netfilter_check.rc != 0

    - name: Verify overlay module is loaded
      ansible.builtin.shell: lsmod | grep overlay
      register: overlay_check
      failed_when: overlay_check.rc != 0

    - name: Verify sysctl settings
      ansible.builtin.shell: |
        sysctl net.bridge.bridge-nf-call-iptables net.bridge.bridge-nf-call-ip6tables net.ipv4.ip_forward
      register: sysctl_check
      failed_when:
        - sysctl_check.stdout.find('net.bridge.bridge-nf-call-iptables = 1') == -1
        - sysctl_check.stdout.find('net.bridge.bridge-nf-call-ip6tables = 1') == -1
        - sysctl_check.stdout.find('net.ipv4.ip_forward = 1') == -1

5.1.2. 플레이북 실행

1
2
3
4
5
6
7
8
9
10
11
12
13
14

❯ ansible-playbook 2-install-containerd-playbook.yaml

PLAY [Install conatinerd] ************************************************************************************************************************************************************************

TASK [Gathering Facts] ***************************************************************************************************************************************************************************
ok: [10.0.0.202]
ok: [10.0.0.201]
ok: [10.0.0.100]
...
...
...
10.0.0.100              : ok=11   changed=6    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
10.0.0.201              : ok=11   changed=6    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
10.0.0.202              : ok=11   changed=6    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

5.2. Containerd 설치

참고자료 - https://docs.docker.com/engine/install/ubuntu/

5.2.1. 플레이북 구성 (2-install-containerd-playbook.yaml)에 추가

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

    ## Step 2.2
    - name: Add Docker's official GPG key
      ansible.builtin.shell: |
        sudo apt-get update
        sudo apt-get install -y ca-certificates curl gnupg
        sudo install -m 0755 -d /etc/apt/keyrings
        curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
        sudo chmod a+r /etc/apt/keyrings/docker.gpg
    
    - name: Set up the Docker repository
      ansible.builtin.shell: |
        mkdir -p /etc/apt/keyrings
        echo \
          "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
          $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
          sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
        sudo apt-get update

    - name: Install containerd.io
      ansible.builtin.apt:
        name: containerd.io
        state: present

5.2.2. 플레이북 실행

이전 내용은 생략했습니다.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

❯ ansible-playbook 2-install-containerd-playbook.yaml

PLAY [Install conatinerd] ************************************************************************************************************************************************************************

TASK [Gathering Facts] ***************************************************************************************************************************************************************************
ok: [10.0.0.202]
ok: [10.0.0.100]
ok: [10.0.0.201]
...
...
...
TASK [Add Docker''s official GPG key] *************************************************************************************************************************************************************
changed: [10.0.0.202]
changed: [10.0.0.100]
changed: [10.0.0.201]

TASK [Set up the Docker repository] **************************************************************************************************************************************************************
changed: [10.0.0.100]
changed: [10.0.0.202]
changed: [10.0.0.201]

TASK [Install containerd.io] *********************************************************************************************************************************************************************
changed: [10.0.0.202]
changed: [10.0.0.100]
changed: [10.0.0.201]

PLAY RECAP ***************************************************************************************************************************************************************************************
10.0.0.100              : ok=14   changed=9    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
10.0.0.201              : ok=14   changed=9    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
10.0.0.202              : ok=14   changed=9    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

5.3. systemd cgroup driver 구성

5.3.1. 플레이북 구성 (2-install-containerd-playbook.yaml)에 추가

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52

    ## Step 2.3
    - name: Setup containerd config directory
      ansible.builtin.file:
        path: /etc/containerd
        state: directory
        owner: root
        group: root
        mode: "0755"

    - name: Generate default containerd configuration
      ansible.builtin.command:
        cmd: "containerd config default"
      register: containerd_default_config

    - name: Save default containerd configuration to config.toml
      ansible.builtin.copy:
        dest: /etc/containerd/config.toml
        content: ""

    - name: Update sandbox_image in config.toml
      ansible.builtin.replace:
        path: /etc/containerd/config.toml
        regexp: 'sandbox_image = ".*"'
        replace: 'sandbox_image = "registry.k8s.io/pause:3.9"'

    - name: Update SystemdCgroup in config.toml
      ansible.builtin.replace:
        path: /etc/containerd/config.toml
        regexp: 'SystemdCgroup = false'
        replace: 'SystemdCgroup = true'
        
    - name: Restart containerd
      ansible.builtin.systemd:
        name: containerd
        state: restarted

    - name: Create crictl config file
      copy:
        dest: /etc/crictl.yaml
        content: |
          runtime-endpoint: unix:///run/containerd/containerd.sock
        mode: '0644'

    - name: Verify crictl configuration
      command: crictl stats
      register: crictl_stats
      ignore_errors: yes

    - name: Display crictl stats output
      debug:
        var: crictl_stats.stdout_lines
      when: crictl_stats is defined and crictl_stats.stdout_lines is defined

5.3.2. 플레이북 실행

이전 내용은 생략했습니다.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66

❯ ansible-playbook 2-install-containerd-playbook.yaml

...
...
...
TASK [Setup containerd config directory] ********************************************************************************************************************************************************************************
ok: [worker01]
ok: [master]
ok: [worker02]

TASK [Generate default containerd configuration] ************************************************************************************************************************************************************************
changed: [master]
changed: [worker01]
changed: [worker02]

TASK [Save default containerd configuration to config.toml] *************************************************************************************************************************************************************
changed: [worker02]
changed: [master]
changed: [worker01]

TASK [Update sandbox_image in config.toml] ******************************************************************************************************************************************************************************
changed: [worker01]
changed: [master]
changed: [worker02]

TASK [Update SystemdCgroup in config.toml] ******************************************************************************************************************************************************************************
changed: [worker01]
changed: [master]
changed: [worker02]

TASK [Restart containerd] ***********************************************************************************************************************************************************************************************
changed: [worker01]
changed: [worker02]
changed: [master]

TASK [Create crictl config file] ****************************************************************************************************************************************************************************************
changed: [master]
changed: [worker02]
changed: [worker01]

TASK [Verify crictl configuration] **************************************************************************************************************************************************************************************
changed: [master]
changed: [worker01]
changed: [worker02]

TASK [Display crictl stats output] **************************************************************************************************************************************************************************************
ok: [master] => {
    "crictl_stats.stdout_lines": [
        "\u001b[2J\u001b[HCONTAINER           NAME                CPU %               MEM                 DISK                INODES"
    ]
}
ok: [worker01] => {
    "crictl_stats.stdout_lines": [
        "\u001b[2J\u001b[HCONTAINER           NAME                CPU %               MEM                 DISK                INODES"
    ]
}
ok: [worker02] => {
    "crictl_stats.stdout_lines": [
        "\u001b[2J\u001b[HCONTAINER           NAME                CPU %               MEM                 DISK                INODES"
    ]
}

PLAY RECAP **************************************************************************************************************************************************************************************************************
master                     : ok=21   changed=15   unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
worker01                   : ok=21   changed=15   unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
worker02                   : ok=21   changed=15   unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

6. kubeadm, kubelet, kubectl 설치

6.1. 플레이북 구성 (3-kube-series-playbook.yaml)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55

- name: Install kubeadm, kubectl and kubelet
  hosts: nodes
  become: true
  tasks:
    - name: Gather facts
      ansible.builtin.setup:

    - name: Update apt package index
      ansible.builtin.apt:
        update_cache: yes

    - name: Install packages needed for Kubernetes apt repository
      ansible.builtin.apt:
        name:
          - apt-transport-https
          - ca-certificates
          - curl
          - gpg
        state: present

    - name: Create directory for apt keyrings if it does not exist
      ansible.builtin.file:
        path: /etc/apt/keyrings
        state: directory
        mode: '0755'
        
    - name: Download Kubernetes public signing key
      ansible.builtin.shell: |
        curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.29/deb/Release.key | gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg

    - name: Add Kubernetes apt repository
      ansible.builtin.shell: |
        echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.29/deb/ /' | tee /etc/apt/sources.list.d/kubernetes.list

    - name: Update apt package index again
      ansible.builtin.apt:
        update_cache: yes

    - name: Install kubeadm, kubelet and kubectl
      ansible.builtin.apt:
        name:
          - kubeadm
          - kubelet
          - kubectl
        state: present

    - name: Hold kubelet, kubeadm and kubectl packages
      ansible.builtin.shell: |
        sudo apt-mark hold kubelet kubeadm kubectl

    - name: Enable and start kubelet service
      ansible.builtin.systemd:
        name: kubelet
        enabled: yes
        state: started

6.2. 플레이북 실행

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58

❯ ansible-playbook 3-kube-series-playbook.yaml

PLAY [Install kubeadm, kubectl and kubelet] **********************************************************************************************************************************************************

TASK [Gathering Facts] *******************************************************************************************************************************************************************************
ok: [10.0.0.202]
ok: [10.0.0.201]
ok: [10.0.0.100]

TASK [Gather facts] **********************************************************************************************************************************************************************************
ok: [10.0.0.201]
ok: [10.0.0.202]
ok: [10.0.0.100]

TASK [Update apt package index] **********************************************************************************************************************************************************************
ok: [10.0.0.201]
ok: [10.0.0.202]
ok: [10.0.0.100]

TASK [Install packages needed for Kubernetes apt repository] *****************************************************************************************************************************************
changed: [10.0.0.201]
changed: [10.0.0.100]
changed: [10.0.0.202]

TASK [Create directory for apt keyrings if it does not exist] ****************************************************************************************************************************************
ok: [10.0.0.202]
ok: [10.0.0.201]
ok: [10.0.0.100]

TASK [Download Kubernetes public signing key] ********************************************************************************************************************************************************
changed: [10.0.0.201]
changed: [10.0.0.100]
changed: [10.0.0.202]

TASK [Add Kubernetes apt repository] *****************************************************************************************************************************************************************
changed: [10.0.0.201]
changed: [10.0.0.202]
changed: [10.0.0.100]

TASK [Update apt package index again] ****************************************************************************************************************************************************************
changed: [10.0.0.201]
changed: [10.0.0.202]
changed: [10.0.0.100]

TASK [Install kubeadm, kubelet and kubectl] **********************************************************************************************************************************************************
changed: [10.0.0.201]
changed: [10.0.0.100]
changed: [10.0.0.202]

TASK [Hold kubelet, kubeadm and kubectl packages] ****************************************************************************************************************************************************
changed: [10.0.0.202]
changed: [10.0.0.100]
changed: [10.0.0.201]

PLAY RECAP *******************************************************************************************************************************************************************************************
10.0.0.100              : ok=10   changed=6    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
10.0.0.201              : ok=10   changed=6    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
10.0.0.202              : ok=10   changed=6    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

7. 클러스터 초기화 (In Master)

7.1. 플레이북 (4-init-cluster-playbook.yaml)

클러스터 초기화 후 조인 명령어를 홈 디렉토리에 저장하고, 콘솔로도 출력하도록 했습니다.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

- name: Initialize Kubernetes Cluster
  hosts: master_nodes
  become: true
  tasks:
    - name: Initialize the Kubernetes cluster
      command: kubeadm init --pod-network-cidr=10.244.0.0/16 --apiserver-advertise-address=10.0.0.100 --cri-socket=unix:///run/containerd/containerd.sock
      register: kubeadm_init_output
      ignore_errors: true

    - name: Create .kube directory
      file:
        path: "/.kube"
        state: directory
        mode: '0755'

    - name: Copy admin.conf to user's kube config
      copy:
        src: /etc/kubernetes/admin.conf
        dest: "/.kube/config"
        remote_src: yes
        owner: ""
        mode: '0644'
      when: kubeadm_init_output.changed

    - name: Generate join command
      shell: kubeadm token create --print-join-command
      register: join_command
      when: kubeadm_init_output.changed

    - name: Save join command to file
      copy:
        content: ""
        dest: "/kubeadm_join_cmd.sh"
      when: kubeadm_init_output.changed

    - name: Output join command to console
      debug:
        msg: ""
      when: kubeadm_init_output.changed

7.2. 플레이북 실행

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

❯ ansible-playbook 4-init-cluster-playbook.yaml

PLAY [Initialize Kubernetes Cluster] ************************************************************************************************************************************************************************************

TASK [Gathering Facts] **************************************************************************************************************************************************************************************************
ok: [master]

TASK [Initialize the Kubernetes cluster] ********************************************************************************************************************************************************************************
changed: [master]

TASK [Create .kube directory] *******************************************************************************************************************************************************************************************
ok: [master]

TASK [Copy admin.conf to user's kube config] ****************************************************************************************************************************************************************************
changed: [master]

TASK [Generate join command] ********************************************************************************************************************************************************************************************
changed: [master]

TASK [Save join command to file] ****************************************************************************************************************************************************************************************
changed: [master]

TASK [Output join command to console] ***********************************************************************************************************************************************************************************
ok: [master] => {
    "msg": "kubeadm join 10.0.0.100:6443 --token 7wrnqt.8r6p42ice03v8cnu --discovery-token-ca-cert-hash sha256:<DISCOVERY_CA_CERT_HASH> "
}

PLAY RECAP **************************************************************************************************************************************************************************************************************
master                     : ok=7    changed=4    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

8. 네트워크 플러그인 설치(Calico) (In Master)

8.1. 플레이북 (5-install-calico-playbook.yaml)

1
2
3
4
5
6
7
8
9
10
11
12

- name: Install Calico
  hosts: master_nodes
  become: true
  tasks:
    - name: Download Calico manifest
      get_url:
        url: https://docs.projectcalico.org/manifests/calico.yaml
        dest: /usr/local/bin/calico.yaml

    - name: Apply Calico manifest
      command: kubectl apply -f /usr/local/bin/calico.yaml
      register: calico_install

8.2. 플레이북 실행

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

❯ ansible-playbook 5-install-calico.playbook.yaml

PLAY [Install Calico] ***************************************************************************************************************************************************************************************************

TASK [Gathering Facts] **************************************************************************************************************************************************************************************************
ok: [master]

TASK [Download Calico manifest] *****************************************************************************************************************************************************************************************
ok: [master]

TASK [Apply Calico manifest] ********************************************************************************************************************************************************************************************
changed: [master]

PLAY RECAP **************************************************************************************************************************************************************************************************************
master                     : ok=3    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

9. 워커노드 추가 (In Worker)

9.1. 플레이북 (6-join-worker-nodes-playbook.yaml)

클러스터 초기화 할 때 생성된 조인 명령어를 사용해주셔야 합니다.

1
2
3
4
5
6
7

- name: Join Worker Nodes to Kubernetes Cluster
  hosts: worker_nodes
  become: true
  tasks:
    - name: Join the node to the cluster
      command: {join commands - 수정필요}
      register: join_command

9.2. 플레이북 실행

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

❯ ansible-playbook 6-join-worker-nodes-playbook.yaml

PLAY [Join Worker Nodes to Kubernetes Cluster] **************************************************************************************************************************************************************************

TASK [Gathering Facts] **************************************************************************************************************************************************************************************************
ok: [worker02]
ok: [worker01]

TASK [Join the node to the cluster] *************************************************************************************************************************************************************************************
changed: [worker01]
changed: [worker02]

PLAY RECAP **************************************************************************************************************************************************************************************************************
worker01                   : ok=2    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
worker02                   : ok=2    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

10. 확인

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

❯ ssh Master
Welcome to Ubuntu 24.04 LTS (GNU/Linux 6.8.0-36-generic x86_64)
...
Last login: Wed Jun 26 06:09:17 2024 from 10.0.0.1

--------------------------------------------------------------------------------------------

❯ kubectl get nodes
NAME       STATUS   ROLES           AGE     VERSION
master     Ready    control-plane   2m41s   v1.29.6
worker01   Ready    <none>          106s    v1.29.6
worker02   Ready    <none>          106s    v1.29.6

--------------------------------------------------------------------------------------------

❯ kubectl get pods -A
NAMESPACE     NAME                                       READY   STATUS    RESTARTS   AGE
kube-system   calico-kube-controllers-658d97c59c-7bw5w   1/1     Running   0          2m17s
kube-system   calico-node-dq6xv                          1/1     Running   0          109s
kube-system   calico-node-nnkcr                          1/1     Running   0          109s
kube-system   calico-node-nq4ss                          1/1     Running   0          2m17s
kube-system   coredns-76f75df574-2k4hb                   1/1     Running   0          2m27s
kube-system   coredns-76f75df574-lzl2x                   1/1     Running   0          2m27s
kube-system   etcd-master                                1/1     Running   8          2m41s
kube-system   kube-apiserver-master                      1/1     Running   8          2m43s
kube-system   kube-controller-manager-master             1/1     Running   8          2m41s
kube-system   kube-proxy-2fpsg                           1/1     Running   0          109s
kube-system   kube-proxy-b4kmd                           1/1     Running   0          2m28s
kube-system   kube-proxy-s2bkd                           1/1     Running   0          109s
kube-system   kube-scheduler-master                      1/1     Running   8          2m41s

11. 마무리

CLI 콘솔에서 구축하는 것을 Ansible Code로 옮기면 되는 간단한 작업일 것이라 생각했습니다. 하지만 잘못된 작업을 했을 때 어떻게 되돌릴 것인지, 설정이 제대로 되었는지 어떻게 확인할 것인지, 특정 파일의 특정 부분을 어떻게 수정할 것인지, Cluster Join에 필요한 토큰을 어떻게 저장하고 사용할 것인지 등 다양한 문제를 직면하게 되었고, 문제의 원인이 무엇인지, 해당 문제를 해결하기 위해서 어떤 방법을 사용해야 하는지 찾아보면서 몰랐던 기능과 해당 기능을 사용하는 방법을 습득하면서 Ansible에 대한 이해를 넓힐 수 있었습니다.

핵심 정리

12. Reference

궁금하신 점이나 추가해야 할 부분은 댓글이나 아래의 링크를 통해 문의해주세요.
Written with KKamJi

IaC, Ansible
ansible kubernetes inventory playbook kubeadm k8s kubelet systemd calico kubectl docker containerd
This post is licensed under CC BY 4.0 by the author.