Virtualization and cloud technologies

Jakub Klinkovský

:: Czech Technical University in Prague
:: Faculty of Nuclear Sciences and Physical Engineering
:: Department of Software Engineering

Kubernetes

• Kubernetes is a portable, extensible, open source platform for managing containerized workloads and services, that facilitates both declarative configuration and automation. It has a large, rapidly growing ecosystem. Kubernetes services, support, and tools are widely available.

• The name Kubernetes originates from Greek, meaning helmsman or pilot. K8s as an abbreviation results from counting the eight letters between the "K" and the "s".

• Google open-sourced the Kubernetes project in 2014. Kubernetes combines over 15 years of Google's experience running production workloads at scale with best-of-breed ideas and practices from the community.

Main Features of Kubernetes

• Service discovery and load balancing If traffic to a container is high, Kubernetes is able to load balance and distribute the network traffic.
• Automated scaling and failover If the current number of containers is not enough, others can be automatically started. If some containers become unavailable, others can take over.
• Automated rollouts and rollbacks You can describe the desired state for your deployed containers using Kubernetes, and it can change the actual state to the desired state at a controlled rate.
• Self-healing Kubernetes restarts or replaces containers that fail or don't respond.
• Designed for extensibility Add features to your Kubernetes cluster without changing upstream source code.

The Illustrated Children's Guide to Kubernetes

Kubernetes Components

Control plane components

• kube-apiserver
  The core component server that exposes the Kubernetes HTTP API
• etcd
  Consistent and highly-available key value store for all API server data
• kube-scheduler
  Looks for Pods not yet bound to a node, and assigns each Pod to a suitable node
• kube-controller-manager
  Runs controllers to implement Kubernetes API behavior
• cloud-controller-manager (optional)
  Integrates with underlying cloud provider(s) (e.g. AWS, Google Cloud, etc.)

Node components

Run on every Node and provide the Kubernetes runtime environment:

• kubelet
  Ensures that Pods are running, including their containers.
• kube-proxy (optional)
  Maintains network rules on nodes to implement Services.
• Container runtime
  Software responsible for running containers (e.g. CRI-O, containerd, Docker).

Addons

Addons extend the functionality of Kubernetes. A few important examples include:
DNS, Web UI (dashboard), Container resource monitoring, Cluster-level logging

References

From the Kubernetes documentation:

• Kubernetes Components
• Cluster Architecture

External:

• Understanding Kubernetes Architecture with Diagrams

Objects in Kubernetes

Kubernetes objects are persistent entities in the Kubernetes system. Kubernetes uses these entities to represent the state of the cluster. Specifically, they can describe:

• What containerized applications are running and which resources they need
• How the applications can be accessed from the outside and from within the cluster
• Policies around how those applications behave, such as restart policies, upgrades, and fault-tolerance

A Kubernetes object is a record of intent – once you create the object, the Kubernetes system will constantly work to ensure that the object exists.

Objects can be created, modified, or deleted through the Kubernetes API (the kube-apiserver component) using kubectl or another program using some client library.

Common Types of Objects

• • Basic objects: Pod, Service, Volume
  • High-level workloads: Deployment, ReplicaSet, StatefulSet, DaemonSet, Job, and CronJob
  • Configuration objects: ConfigMap and Secret
  • Policies: NetworkPolicy, LimitRange, ResourceQuota

• There are also cluster resources that are used internally and represent parts of the actual cluster:

  • Node, Namespace, APIService, IPAddress, Event, etc.

• The specification is recursive and open-ended. The CustomResourceDefinition type allows to define custom objects that should be available through the Kubernetes API.

Manifest Files

Manifest is a specification of a Kubernetes API object in JSON or YAML format.

A manifest specifies the desired state of an object that Kubernetes will maintain when you apply the manifest. For YAML format, each file can contain multiple manifests.

kubectl (pronounced like "cube control") is a command line tool for communicating with a Kubernetes cluster's control plane, using the Kubernetes API.

You can use kubectl to create, inspect, update, and delete Kubernetes objects. It can also read, validate, and apply manifest files.

Pods

Pods are the smallest deployable units of computing that you can create and manage in Kubernetes.

A Pod (as in a pod of whales or pea pod) is a group of one or more containers, with shared storage and network resources, and a specification for how to run the containers. A Pod models an application-specific "logical host": it contains one or more application containers which are relatively tightly coupled.

Example pod specification

A pod can be specified using a YAML file, e.g. simple-pod.yaml:

apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:
  containers:
  - name: nginx
    image: nginx:1.14.2
    ports:
    - containerPort: 80

Then it can be created by running the following command:

kubectl apply -f simple-pod.yaml

Example pod status

The command kubectl get pods nginx -o yaml shows the spec and status:

apiVersion: v1
kind: Pod
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"v1","kind":"Pod","metadata":{"annotations":{},"name":"nginx","namespace":"default"},"spec":{"containers":[{"image":"nginx:1.14.2","name":"nginx","ports":[{"containerPort":80}]}]}}
    kubernetes.io/limit-ranger: 'LimitRanger plugin set: cpu, memory request for container
      nginx; cpu, memory limit for container nginx'
  creationTimestamp: "2025-05-13T11:43:49Z"
  name: nginx
  namespace: default
  resourceVersion: "132223514"
  uid: 8e295722-e557-4723-9452-2e73a96ab708
spec:
  containers:
  - image: nginx:1.14.2
    imagePullPolicy: IfNotPresent
    name: nginx
    ports:
    - containerPort: 80
      protocol: TCP
    resources:
      limits:
        cpu: "1"
        memory: 1Gi
      requests:
        cpu: "1"
        memory: 1Gi
    terminationMessagePath: /dev/termination-log
    terminationMessagePolicy: File
    volumeMounts:
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: kube-api-access-rvkcz
      readOnly: true
  dnsPolicy: ClusterFirst
  enableServiceLinks: true
  nodeName: k8s.fjfi.cvut.cz
  preemptionPolicy: PreemptLowerPriority
  priority: 0
  restartPolicy: Always
  schedulerName: default-scheduler
  securityContext: {}
  serviceAccount: default
  serviceAccountName: default
  terminationGracePeriodSeconds: 30
  tolerations:
  - effect: NoExecute
    key: node.kubernetes.io/not-ready
    operator: Exists
    tolerationSeconds: 300
  - effect: NoExecute
    key: node.kubernetes.io/unreachable
    operator: Exists
    tolerationSeconds: 300
  volumes:
  - name: kube-api-access-rvkcz
    projected:
      defaultMode: 420
      sources:
      - serviceAccountToken:
          expirationSeconds: 3607
          path: token
      - configMap:
          items:
          - key: ca.crt
            path: ca.crt
          name: kube-root-ca.crt
      - downwardAPI:
          items:
          - fieldRef:
              apiVersion: v1
              fieldPath: metadata.namespace
            path: namespace

status:
  conditions:
  - lastProbeTime: null
    lastTransitionTime: "2025-05-13T11:43:55Z"
    status: "True"
    type: PodReadyToStartContainers
  - lastProbeTime: null
    lastTransitionTime: "2025-05-13T11:43:49Z"
    status: "True"
    type: Initialized
  - lastProbeTime: null
    lastTransitionTime: "2025-05-13T11:43:55Z"
    status: "True"
    type: Ready
  - lastProbeTime: null
    lastTransitionTime: "2025-05-13T11:43:55Z"
    status: "True"
    type: ContainersReady
  - lastProbeTime: null
    lastTransitionTime: "2025-05-13T11:43:49Z"
    status: "True"
    type: PodScheduled
  containerStatuses:
  - containerID: cri-o://8aeeaa3b8d971796bdbd14ecee2a904adacb57c3bdc456f5aed04c4a2eb28ebc
    image: docker.io/library/nginx:1.14.2
    imageID: docker.io/library/nginx@sha256:706446e9c6667c0880d5da3f39c09a6c7d2114f5a5d6b74a2fafd24ae30d2078
    lastState: {}
    name: nginx
    ready: true
    restartCount: 0
    started: true
    state:
      running:
        startedAt: "2025-05-13T11:43:54Z"
    volumeMounts:
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: kube-api-access-rvkcz
      readOnly: true
      recursiveReadOnly: Disabled
  hostIP: 147.32.22.10
  hostIPs:
  - ip: 147.32.22.10
  phase: Running
  podIP: 10.240.14.131
  podIPs:
  - ip: 10.240.14.131
  qosClass: Guaranteed
  startTime: "2025-05-13T11:43:49Z"

Pods and controllers

• You can use workload resources to create and manage multiple Pods for you. A controller for the resource can handle replication, rollout, and automatic healing in case of pod failure.

  For example, if a Node fails, a controller notices that Pods on that Node have stopped working and creates a replacement Pod. The scheduler places the replacement Pod onto a healthy Node.

• Some examples of workload resources that manage one or more Pods:

  • ReplicaSet – ensures that a specified number of pod replicas are running at any given time (guarantees the availability of a specified number of identical Pods)
  • Deployment – provides declarative updates for Pods and ReplicaSets (rollout and rollback)
  • StatefulSet – runs a group of Pods and maintains a sticky identity for each of them (useful for managing applications that need persistent storage or a stable, unique network identity)

Services

Service is a method for exposing a network application that is running as one or more Pods in the cluster.

For example:

apiVersion: v1
kind: Service
metadata:
  name: my-service
spec:
  selector:
    app.kubernetes.io/name: MyApp
  ports:
    - protocol: TCP
      port: 80
      targetPort: 9376

How it works

• Kubernetes creates a new Service object named my-service
• Kubernetes assigns this Service an IP address within the cluster
  (Kubernetes has a virtual network where the traffic is handled by the kube-proxy components or an alternative network plugin)
• The controller for the Service continuously scans for Pods that match its selector, and then makes any necessary updates to EndpointSlices of the Service
  • EndpointSlices are objects that represent connections between the Service and backend Pods
  • The previous Service uses the TCP protocol, is accessible on port 80 (Service port), and targets the port 9376 on any Pod with the label app.kubernetes.io/name: MyApp

Service types

The default Service type is ClusterIP, which was described in the previous example.

Other types are:

• NodePort – Exposes the Service on the IP address of every Node in the cluster, using a port number unique in the cluster
• LoadBalancer – Exposes the Service externally using an external load balancer. Kubernetes does not directly offer a load balancing component, but many cloud providers do.
• ExternalName – Maps the Service to an external DNS name, not to a typical Pod selector.

Exposing a Service Externally

The Ingress object can be used to expose HTTP and HTTPS routes from outside the cluster to services within the cluster. Traffic routing is controlled by rules defined on the Ingress resource.

An Ingress may be configured to give Services externally-reachable URLs, load balance traffic, terminate SSL / TLS, and offer name-based virtual hosting.

Ingress controller

In order for an Ingress to work, there must be an ingress controller running in the cluster. It is responsible for configuring routers within the cluster, often even the external load balancer, according to the Ingress specifications.

The Kubernetes project maintains controllers for AWS and Google Cloud, and a generic controller using nginx. There are also many additional controllers.

Kubernetes Distributions

Kubernetes is very flexible and it consists of many components. There are even several alternatives for specific components (especially network plugins). Overall it is quite hard to install and administer the whole cluster.

Similarly to Linux, there are distributions of Kubernetes that select specific components, installation method and some default configuration to provide a simplified user experience for common use cases. For example:

• Rancher
• k3s
• minikube – allows to install Kubernetes on your laptop (using e.g. Docker)

Furthermore, many cloud providers have their own pre-installed Kubernetes services: Google Kubernetes Engine (GKE), Azure Kubernetes Service (AKS), Amazon Elastic Kubernetes Service (EKS), etc.

Installing Software in a Kubernetes Cluster

Helm is a popular package manager for Kubernetes – it allows you to install applications using so called helm charts, which help you define, install, and upgrade even the most complex applications.

• Simple sharing – charts are published in public (or private) repositories
• Easy updates – download new chart and install it with one command
• Rollback – helm tracks the history of all deployments and allows to move back in time if the new release has problems

Moving from Docker to Kubernetes

Kompose is a conversion tool for Docker Compose to container orchestrators such as Kubernetes.

$ kompose convert -f compose.yaml

$ kubectl apply -f .

$ kubectl get pods
NAME                            READY     STATUS              RESTARTS   AGE
frontend-591253677-5t038        1/1       Running             0          10s
redis-leader-2410703502-9hshf   1/1       Running             0          10s
redis-replica-4049176185-hr1lr  1/1       Running             0          10s

See also this tutorial.

References

• Kubernetes Documentation
  • Concepts
  • Glossary
• Kubernetes Tutorials
  • Hello Minikube
    Note: minikube is installed on gp8.fjfi.cvut.cz but you need to configure it for podman before starting:

    minikube config set rootless true
    minikube config set driver podman
    minikube config set container-runtime cri-o
    minikube start
    

• KodeKloud Public Playgrounds