CKA Kubernetes ( K8S ) Cluster Maintenance

 The preparation for the CKA (Certified Kubernetes Administrator) requires lots of practice and practice. Fortunately, we have lots of online playgrounds to keep practicing, there are lots of free courseware available and lots of paid as well are available. In addition to that, we get two attempts to clear the exam

• There is training conducted by the https://training.linuxfoundation.org/ itself who conducts the CKA exams. 
• But I felt this course from Udemy is good https://www.udemy.com/course/certified-kubernetes-administrator-with-practice-tests/, it has videos that explain each concept and have lots of practical sessions and mock exams.
• The https://killer.sh/ is another tool where you can practice and the questions asked in the killer.sh are complex and tests real skill.  If we subscribe to the CKA exam, we will get two free sessions to solve the problems in the killer.sh
• https://www.katacoda.com/courses/kubernetes this is very useful if you are a beginner in kubernetes. It starts with a very simple usecase/scenario and we can solve in our own pace and try any number of times
• https://killercoda.com/ is another useful online tool that we can use like a playground to solve scenario-based problems

In the CKA exam definitely we can expect questions on the cluster maintenance, also this will be high score question. So if we practice well on this then we can save time and score high on these questions.

OS Upgrades

• When a node in the cluster is brought down then the pods inside will become inaccessible

• The pods will be accessible again if the node comes back within 5 mins

• Same way to do is

○ kubectl drain node01 -> This will move the pods to a different node and mark the node as cordon which means no new scheduling of pod can happen in this node

○ kubectl uncordon node01 -> after the OS upgrade, the node should be uncordon so scheduling of pod can happen in this node

○ kubectl cordon node01 -> Like without draining the pod, we can simply mark as cordon so no new scheduling happens here

Kubernetes release

• The versioning is v1.13.0 -> major.minor.bug_fixes

• https://github.com/kubernetes/kubernetes

• Only latest 3 versions of k8s will be supported and the older releases will become unsupported

• When master node is upgraded the control plane component is not accessible

• Upgrade Strategy to worker node

○ All at once - will have a down time

○ Rolling updates 

○ Bring new nodes with new version and remove old nodes

Backup and Restore

• Can take backup in three different ways

○ Resource Configuration

§ kubectl get all -A -o yaml > all-resources.yaml

§ There are tools like ARK / Velero

○ ETCD 

§ the storage directory can be backed up as it is

§ ETCD comes up with its snapshot tool

§ export ETCDCTL_API=3

§ etcdctl snapshot save snapshot.db

§ service kube-apiserver stop

§ etcdctl snapshot restore snapshot.db

§ systemctl daemon-reload

§ service etcd restart

§ service kube-apiserver start

CKA Kubernetes ( K8S ) Application Lifecycle Management

 The preparation for the CKA (Certified Kubernetes Administrator) requires lots of practice and practice. Fortunately, we have lots of online playgrounds to keep practicing, there are lots of free courseware available and lots of paid as well are available. In addition to that, we get two attempts to clear the exam

• There is training conducted by the https://training.linuxfoundation.org/ itself who conducts the CKA exams. 
• But I felt this course from Udemy is good https://www.udemy.com/course/certified-kubernetes-administrator-with-practice-tests/, it has videos that explain each concept and have lots of practical sessions and mock exams.
• The https://killer.sh/ is another tool where you can practice and the questions asked in the killer.sh are complex and tests real skill.  If we subscribe to the CKA exam, we will get two free sessions to solve the problems in the killer.sh
• https://www.katacoda.com/courses/kubernetes this is very useful if you are a beginner in kubernetes. It starts with a very simple usecase/scenario and we can solve in our own pace and try any number of times
• https://killercoda.com/ is another useful online tool that we can use like a playground to solve scenario-based problems

Rolling updates and rollback

• Rollout is the process of updating all the pods/replicas for a defined deployment

• Revision is a change happened in the deployment

• Deployment Strategey

○ Recreate - Destroys all the existing pod and then start creating new version of pods

○ Rolling Update - Each pod gets deleted and recreated with new version in a sequence 

• Under the hood, the deployment actually creates another ReplicaSet with the new version of the image and will bring down the pods in older ReplicaSet

• For reverting the new changes, we need to execute 'kubectl rollout undo deployment/my-deployment'. Now it actually switches back to old ReplicaSet

Command and Argument

• Difference between CMD and ENTRYPOINT in docker is

○ CMD will be executed as soon as the container starts. And we can override the whole command by passing in the 'docker run'

○ ENTRYPOINT - if given, the final command that will be executed will be the concatenation of ENTRYPOINT and CMD

• We can override both of these properties in docker run command

• We can override these in kubernetes container by specifying the 'command' and 'args' fields

spec:

  containers:

   -  image: ubuntu

      command: ["sleep"]

      args: ["1000"]

     

Environment Variables

• We can define using the 'env' property

• We can get the env var value from ConfigMap and Secret too

spec:

  containers:

   -  image: ubuntu

      command: ["sleep"]

      args: ["1000"]

      env:

      - name: APP_COLOR

         value: BLUE

ConfigMaps

• We can create using imperative and declarative way

• 2 steps

○ First we need to create the ConfigMap

○ Refer it in the POD

apiVersion: v1

kind: ConfigMap

metadata:

   name: app-config

data:

  APP_COLOR: blue

  APP_MODE: prod

apiVersion: v1

kind: pod

metadata:

spec:

   containers:

   - name: webapp

     image: webapp:v1

     envFrom:

     - configMapRef:

         name: app-config

Secret

• Secret are used to store sensitive information

• Can be Created in both imitative and declarative way

• In declarative way we should encode the value in base64 format

• The ConfigMap and Secret can be mounted as volume as well. In that case the properties needs to be accessed like a file /opt/app-secret-volume/DB_Host

• The value in the file should be encoded like echo -n 'root' | base64

apiVersion: v1

kind: Secret

metadata:

    name: app-secret

data:

   DB_PASSWORD: hjuvhw=

apiVersion: v1

kind: pod

metadata:

spec:

   containers:

   - name: webapp

     image: webapp:v1

     envFrom:

     -secretRef:

         name: app-secret

     volumes:

     - name: app-secret-volume

       secret: 

         secretName: app-secret

Multicontainer pod

• All the containers in the pod shares the same

○ network - accessible by localhost

○ storage

○ lefecycle

• InitContainers

○ The InitContainers are special containers which will be executed before the actual container starts

○ We can define multiple InitContainers and all of them will run in a sequence 

CKA Kubernetes ( K8S ) Logging and Monitoring

 The preparation for the CKA (Certified Kubernetes Administrator) requires lots of practice and practice. Fortunately, we have lots of online playgrounds to keep practicing, there are lots of free courseware available and lots of paid as well are available. In addition to that, we get two attempts to clear the exam

• There is training conducted by the https://training.linuxfoundation.org/ itself who conducts the CKA exams. 
• But I felt this course from Udemy is good https://www.udemy.com/course/certified-kubernetes-administrator-with-practice-tests/, it has videos that explain each concept and have lots of practical sessions and mock exams.
• The https://killer.sh/ is another tool where you can practice and the questions asked in the killer.sh are complex and tests real skill.  If we subscribe to the CKA exam, we will get two free sessions to solve the problems in the killer.sh
• https://www.katacoda.com/courses/kubernetes this is very useful if you are a beginner in kubernetes. It starts with a very simple usecase/scenario and we can solve in our own pace and try any number of times
• https://killercoda.com/ is another useful online tool that we can use like a playground to solve scenario-based problems

Monitor Cluster

• How many nodes are there

• How many are healthy

• CPU, Memory, disk utilization

• pod metrics

• Varieties of tools are available for monitoring

○ Metric server

○ Prometheus

○ Elastic Stack

○ DataDog

○ dynatrace

• There is a component called as cAdvisor inside kubelet which periodically gets status of the pod and pushes to monitoring services

Monitoring POD

• We can see the sysout logs using the kubectl logs command

CKA Kubernetes ( K8S ) Scheduling

 The preparation for the CKA (Certified Kubernetes Administrator) requires lots of practice and practice. Fortunately, we have lots of online playgrounds to keep practicing, there are lots of free courseware available and lots of paid as well are available. In addition to that, we get two attempts to clear the exam

• There is training conducted by the https://training.linuxfoundation.org/ itself who conducts the CKA exams. 
• But I felt this course from Udemy is good https://www.udemy.com/course/certified-kubernetes-administrator-with-practice-tests/, it has videos that explain each concept and have lots of practical sessions and mock exams.
• The https://killer.sh/ is another tool where you can practice and the questions asked in the killer.sh are complex and tests real skill.  If we subscribe to the CKA exam, we will get two free sessions to solve the problems in the killer.sh
• https://www.katacoda.com/courses/kubernetes this is very useful if you are a beginner in kubernetes. It starts with a very simple usecase/scenario and we can solve in our own pace and try any number of times
• https://killercoda.com/ is another useful online tool that we can use like a playground to solve scenario-based problems

Manual Scheduling

• We can decide on in which node the pod should be scheduled or should be running

• The pod.yaml contains nodeName under spec where we can specify the node name

• We cannot modify the nodeName of any running pod

• If we want to run the pod in a different node then we need to use the Binding object

Taints and Tolerance

• Decides which pod should be placed in which node

• It it to enforce restriction on which pods can be placed in which node only but not like security etc

• tolerant pod can be placed on any non tainted nodes

• only tolerant pod be placed in a tainted nodes

• Taints are set to Nodes and Tolerance are set to Pods

• Only the Tolerant pod can be deployed into a tainted nodes\

• 3 traint effect that can be set to a pod when it is tainted

○ NoSchedule - no new pod will be allowed to schedule but existing running pod will continue to run

○ PreferNoSchedule - no guarrentee

○ NoExecute - Once the taint is applied then if the running pod cannot tolerate the taint then it will be evicted

• By default the master node has the taint, so the scheduler will not place any pod in the master node

spec:

  tolerations:

      - key : "app"

         operator: "Equal"

         value: "blue"

         effect: "NoSchedule"

Node selector

• Setting the limitation on the pod so it will be executed in a desired node

• Node will be set with the labels and which will be used in pods yaml file

○ kubectl label nodes node01 size=large

• It has the limitation that we cannot have OR condition or NOT condition in nodeSelector section

spec:

    nodeSelector:

         size:large

Node Affinity

• Ensure pods are hosted in particular node

• We can define complex conditions/rules on in which node the pod should run

• 3 types of affinity can be defined

○ requiredDuringSchedulingIgnoredDuringExecution

○ preferredDuringSchedulingIgnoredDuringExecution

○ requiredDuringSchedulingRequiredDuringExecution

• There are different operators exists like Equal, In, Exists, NoEqual etc

spec:

      affinity:

        nodeAffinity:

          requiredDuringSchedulingIgnoredDuringExecution:

            nodeSelectorTerms:

            - matchExpressions:

              - key: color

                operator: In

                values:

                - blue

Resource Requests

• There are 3 resource that will be utilized by a POD

○ CPU

○ Memory

○ Disk

• We can define two entity for a container

○ requests - a minimum reqt

○ limits - max limit the container can use

• Default limit for any container in kubernetes can be defined using LimitRange resource

○ CPU - 1vCPU

○ Memory - 512 Mi

spec:

   containers:

   - name: publisher

      resources:

          requests:

             memory: "1Gi"

             cpu: 1

         limits:

             memory: "2Gi"c

             cpu: 2

Daemon Set

• Ensures one copy of the pod is always present in each node in the cluster

• Usecases:

○ Monitoring agent

○ Logging

• kube-proxy is one example of DaemonSet

• The definition of DaemonSet is same as ReplicaSet

apiVersion: apps/v1

kind: DaemonSet

metadata:

   name: monitoring-daemon

spec:

   selector:

     matchLabels:

        app: monitoring-agent

   template:

     metadata:

        labels:

           app: monitoring-agent

     spec:

        containers:

           - name: monitoring-agent

              image: monitoring-agent

Static POD

• Ability to create a pod within the worker node without any intervention/help by the master node (kube-apiserver)

• All the pod definition yaml should be kept in /etc/kubernetes/manifests

• kubelet in the worker node periodically checks the files in this directory and create/recreate/delete a pod

• Only pod can be created, we cannot create ReplicaSet, Deployement etc by placing the yaml in the above directory

• We can mention the manifest directory

○ while bringing up the POD using the  --pod-manifest-path args OR

○ we can pass --config=kubeconfig.yaml. And this kubeconfig.yaml will have a key 'staticPodPath' with the location of manifest file

• Usecases

○ To deploy the control-plane components like etcd.yaml, controller-manager, apiserver etc

• ps -aux | grep kubelet

• ps auxw | grep kubelet

Multiple Scheduler

• We can run additional scheduler in the master node

• Also we can specify what the pod placement algorithm the scheduler should follow

• While creating the pod we can tell which scheduler should be used for deployment

spec:

   containers:

   - image: nginx

     name: nginx-app

   schedulerName: my-custom-scheduler  

CKA Kubernetes ( K8S ) Cluster Architecture

 The preparation for the CKA (Certified Kubernetes Administrator) requires lots of practice and practice. Fortunately, we have lots of online playgrounds to keep practicing, there are lots of free courseware available and lots of paid as well are available. In addition to that, we get two attempts to clear the exam

• There is training conducted by the https://training.linuxfoundation.org/ itself who conducts the CKA exams. 
• But I felt this course from Udemy is good https://www.udemy.com/course/certified-kubernetes-administrator-with-practice-tests/, it has videos that explain each concept and have lots of practical sessions and mock exams.
• The https://killer.sh/ is another tool where you can practice and the questions asked in the killer.sh are complex and tests real skill.  If we subscribe to the CKA exam, we will get two free sessions to solve the problems in the killer.sh
• https://www.katacoda.com/courses/kubernetes this is very useful if you are a beginner in kubernetes. It starts with a very simple usecase/scenario and we can solve in our own pace and try any number of times
• https://killercoda.com/ is another useful online tool that we can use like a playground to solve scenario-based problems

Even though the CKA exam is a performance-based exam, we should know the concepts thoroughly so we can solve the problems for troubleshooting and fixing.

Types of K8S deployment

• Manual from scratch

○ All components will be installed directly in the master nodes as a linux service

• Using kubeadm command

○ All components of master will be running as a pod under kube-system namespace

Two types of Nodes

• Master Node 

○ ETCD Cluster - Stores information about the cluster

○ K8S Scheduler - Used to for deploying containers in worker nodes

○ Node Controller - 

○ Replication Controller

○ kube-apiserver - for orchestrating communication between services/containers within the cluster

• Worker Nodes

○ kubelet - is a agent runs on each worker node and listens for instruction from master node

○ kube-proxy - is used for communication between containers within worker node or across worker nodes

Containers

• Even the components running in the master node is also runs inside a container

• K8S supports multiple different container runtime engines

○ docker

○ containerd

○ rkt

ETCD

• Distributed reliable key-value store

• Open source and we need to bring up the service by executing the binary

• by default the service comes up in port 2379

• and comes with a default client called etcdctl

ETCD in K8S

• It stores all the details of the cluster like

○ Nodes

○ Pods

○ Configs

○ Secret

○ Accounts

○ Roles

○ Binding

kube-apiserver

• We can use following clients to talk to kube-apiserver

○ kubectl

○ REST APIs

• Eg: when we execute "kubectl get pods", the kube-apiserver receives the commands and checks the etcd cluster and responses

• The kube-apiserver is responsible for

○ Authenticating the user

○ Validating the request

○ Update the ETCD

○ Talks to scheduler, kubelet etc

Controller Manager : kube-controller-manager

• Node controller responsibility is to monitor the node state and keep it healthy

○ checks the health every 5 sec

○ node eviction time is 5 m

○ kubectl get nodes

• Replication Conroller

○ responsible to keep the desired number of pods in node

• Deployment Controller

• Namespace Controller

• Job controller

• Service account controller

• Endpoint controller

• Stateful set

• Cron job

kube-scheduler

• Will decide which pod should be placed in which node and then the kubelet will take care of deployment

• It has some selection criteria in choosing the right pod

○ Filter Nodes

○ Rank Nodes

kubelet

• Registers the node to kubunetes master

• Create POD

• Monitor Node and POD

• kubeadm does not deploy the kubelet service in the worker node

kube-proxy

• Responsible for hosting pods like a service and exposing it through an IP

• It takes care of establishing a connection between pods across multiple different nodes

• It routes traffic using the iptable it maintains

CKA Kubernetes ( K8S ) Linux Networking Cheat sheet

The preparation for the CKA (Certified Kubernetes Administrator) requires lots of practice and practice. Fortunately, we have lots of online playgrounds to keep practicing, there are lots of free courseware available and lots of paid as well are available. In addition to that, we get two attempts to clear the exam

• There is training conducted by the https://training.linuxfoundation.org/ itself who conducts the CKA exams. 
• But I felt this course from Udemy is good https://www.udemy.com/course/certified-kubernetes-administrator-with-practice-tests/, it has videos that explain each concept and have lots of practical sessions and mock exams.
• The https://killer.sh/ is another tool where you can practice and the questions asked in the killer.sh are complex and tests real skill.  If we subscribe to the CKA exam, we will get two free sessions to solve the problems in the killer.sh
• https://www.katacoda.com/courses/kubernetes this is very useful if you are a beginner in kubernetes. It starts with a very simple usecase/scenario and we can solve in our own pace and try any number of times
• https://killercoda.com/ is another useful online tool that we can use like a playground to solve scenario-based problems

It is good to have some basic understanding of Linux networking concepts and commands as in the CKA exam we will be asked to solve the network related issue

• To get the physical Ethernet available

○ ip link

○ ip addr

• To add an IP address to a interface

○ ip addr add 192.168.1.11/24 dev <ens3>

• To get the list of IP routing tables

○ route

○ ip route

• To add a routing to target via a gateway

○ ip route add <192.168.2.0/24> via <192.168.1.1>

§ 192.168.1.1 - is the IP address where the 'Router' is connected to the network and acts like a gateway 

§ 192.168.2.0/24 - is the CIDR of the target network

• To forward traffic/packets from one interface to another interface in the router or gateway

○ Edit /proc/sys/net/ipv4/ip_forward

§ Set the value to '1' but this will not be preserved on reboots

○  Edit /etc/sysctl.conf --> add an entry 'net.ipv4.ip_forward=1'

§ This change will be preserved on reboots

• To know which DNS server our machine is talking to

○ /etc/resolv.conf

§ We can have multiple nameservers defined

§ Add entry like 'nameserver 8.8.8.8' to point to a public DNS server hosted by google

• Search domain

○ We can specify the domain name that we want to append with user given URL

○ An entry like below in /etc/resolv.conf solves this

§ 'search mycompany.com'

• The order to resolve when duplicate entries are seen in local /etc/hosts and in DNS server

○ /etc/nsswitch.conf

§ Add entry like 'hosts:          files dns' --> Here the first preference goes to local /etc/hosts and then to DNS

• To test DNS resolution

○ nslookup www.google.com

§ nslookup will not consider the entries in /etc/hosts file

○ dig www.google.com

§ This will give more details

• Network namespaces

○ ip netns add <red>

§ For creating the namespaces

○ ip netns

§ for listing the namespaces

○ ip netns exec <red> ip link

OR

○ ip -n <red> ip link

§ For executing commands inside namespaces

• Address Resolution Protocol (ARP) table

○ arp

○ ip netns exec red arp

• Virtual Ethernet

○ ip link add <veth-red> type veth

§ For creating a virtual ethernet

○ ip link add <veth-blue> type veth peer name <veth-red>

○ ip link set veth-blue netns blue

§ For assigning the veth to a namespace

○ ip -n red link del veth-red

• Assigning IP addresses for virtual ethernet interfaces

○ ip -n <red> addr add 192.168.15.1 dev <veth-red>

○ ip -n <red> link set veth-red up

• Virtual Bridge

○ ip link add v-net-0 type bridge

§ This will acts like a virtual switch where the network namespaces can connect to

• netstat -nptl

○ To know the list of process and in which port it is listening to

• netstat -anp

○ To know the list of active connection etc

• iptables -L -t net | grep db-service

• host web-service

Will print the Fully Qualified Domain Name (FQDN) where it is accessible from