Kubernetes Application Delivery and GitOps Operations Model

At first glance, Kubernetes appears to be a container deployment tool that uses a lot of YAML. However, if you follow it a little further, the core of Kubernetes is closer to “declaring what state the application should run in and continuously adjusting that state” rather than “running the container.”

If you think about operating a backend service, what you need is not only container execution. You need to decide what image to deploy, how many instances to maintain, what name the traffic will come under, where to place settings and sensitive information, what health checks are needed to avoid sending requests to unprepared pods, and how to rollback when deployment fails.

Rather than memorizing Kubernetes, AKS, Helm, Operator, ArgoCD, and GitOps separately, this article is a record of the flow of an application from a container image to the operating state of a cluster.

Transition point from container to orchestration

Containers allow applications and execution environments to be bundled and deployed as images. Unlike VMs, startup is fast and resource usage is low because each instance does not contain an entire OS but shares the host OS kernel. It is also advantageous in reducing the difference between the development environment and the operating environment.

However, running one container well and operating dozens of services are different problems. As the number of services increases, the following questions arise.

• If a container dies, who relaunches it?
• When there are multiple servers, which node should they be placed on?
• Which pod will the traffic be sent to?
• How to gradually deploy new versions
• How to separate settings and secrets
• Where to collect logs and metrics

Kubernetes is a container orchestration platform that addresses this problem. The container execution itself is handled by a runtime such as containerd, but Kubernetes looks at the overall state of the cluster and deploys pods, recreates failed pods, and maintains service endpoints.

Separate view of Control Plane and Worker Node

Kubernetes clusters are largely divided into Control Plane and Worker Nodes. The Control Plane manages the state of the cluster, and Worker Nodes run the actual application pods.

Control Plane
- kube-apiserver: API entry point for all cluster requests
- scheduler: decides which Node should run a new Pod
- controller-manager: control loops that reconcile desired and actual state
- etcd: cluster state store

Worker Node
- kubelet: manages Pod state on each Node
- kube-proxy: handles Service network routing
- container runtime: runs containers through containerd or a similar runtime

In managed Kubernetes such as AKS or EKS, much of the control plane is managed by the cloud provider. Users mainly deal with Node Pool, network, storage, application resources, access rights, and observability configuration. Being a managed service does not mean that Kubernetes operations disappear. Although the burden of operating API Server and etcd is reduced, incorrect deployment, excessive resource requests, poor probes, open ingress, and uncleaned secrets still lead to service problems.

Deployment is not a deployment file, but an operating contract

The most frequently encountered resources in Kubernetes are Pod, Deployment, Service, Ingress, ConfigMap, and Secret. These resources are not simple YAML files, but rather express “what the service should be like.”

apiVersion: apps/v1
kind: Deployment
metadata:
  name: api
spec:
  replicas: 3
  selector:
    matchLabels:
      app: api
  template:
    metadata:
      labels:
        app: api
    spec:
      containers:
        - name: api
          image: ghcr.io/example/api:1.0.0
          ports:
            - containerPort: 8080
          readinessProbe:
            httpGet:
              path: /health/ready
              port: 8080
            initialDelaySeconds: 10
            periodSeconds: 5
          livenessProbe:
            httpGet:
              path: /health/live
              port: 8080
            initialDelaySeconds: 30
            periodSeconds: 10
          resources:
            requests:
              cpu: "250m"
              memory: "256Mi"
            limits:
              cpu: "500m"
              memory: "512Mi"

Here, replicas: 3 is a declaration to maintain three pods, and readinessProbe is a device for not sending traffic to pods that are not yet ready. livenessProbe serves as a basis for restarting the application when it enters an unrecoverable state. resources.requests is used by the scheduler to make placement decisions, and limits limits one container from taking excessive node resources.

At first, it is easy to check that it is deployed with kubectl apply -f deployment.yaml. However, from an operational perspective, the following is more important:

• Don't readiness and liveness see the same endpoint?
• Is the application startup time longer than the probe settings?
• Is the CPU limit too low and causing throttling?
• Is OOMKilled not repeated due to exceeding the memory limit?
• Is the minimum number of available pods maintained during rolling updates?

Kubernetes leaves many hints when a problem occurs.

kubectl get pods -o wide
kubectl describe pod api-7c9d8d9f8b-x2k7s
kubectl logs deploy/api --tail=100
kubectl rollout status deployment/api
kubectl rollout history deployment/api
kubectl rollout undo deployment/api

Service and Ingress hide pod replacement

Pods can be created and destroyed at any time. Therefore, operation is difficult if you rely directly on the Pod IP. A Service provides a reliable access point to a set of Pods based on a label selector.

apiVersion: v1
kind: Service
metadata:
  name: api
spec:
  type: ClusterIP
  selector:
    app: api
  ports:
    - port: 80
      targetPort: 8080
`

ClusterIPis for internal access to the cluster, andNodePortis exposed to the outside through the node port. In a cloud environment, using theLoadBalancer` type connects to an external load balancer such as Azure Load Balancer or AWS Load Balancer. HTTP layer routing, TLS, and host/path-based routing are usually handled by the Ingress Controller.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: api
spec:
  ingressClassName: nginx
  rules:
    - host: api.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: api
                port:
                  number: 80

In AKS, there are options such as Azure CNI, Azure Load Balancer, and Application Gateway Ingress Controller, and in EKS, you can connect to ALB/NLB with AWS Load Balancer Controller. Ultimately, the Kubernetes network is not just a matter of resources within the cluster, but also connects to VNet/VPC, Subnet, NSG/Security Group, DNS, and TLS certificates.

ConfigMap and Secret make the deployment unit lightweight

If all settings are entered when creating an image, the image must be recreated for each environment. In Kubernetes, settings can be separated into ConfigMap and Secret.

apiVersion: v1
kind: ConfigMap
metadata:
  name: api-config
data:
  SPRING_PROFILES_ACTIVE: prod
  LOG_LEVEL: info

apiVersion: v1
kind: Secret
metadata:
  name: api-secret
type: Opaque
stringData:
  DATABASE_URL: postgresql://user:password@postgres:5432/app

Secret has a good name and purpose, but that doesn't mean that the default Kubernetes Secret is a robust secret management system. etcd encryption, RBAC, External Secrets, cloud secret manager integration, and Secret rotation must be seen together to create an operational configuration. In particular, when applying GitOps, methods such as SOPS, Sealed Secrets, and External Secrets Operator should be reviewed to avoid uploading plaintext secrets to Git.

Helm reduces YAML duplication, but also creates abstraction costs

As Kubernetes resources increase, YAML management for each environment becomes complicated. Helm bundles Kubernetes resources into a package called Chart and injects environment-specific values ​​with values.yaml. Like apt, yum, and pip, it is close to a package manager that installs and upgrades Kubernetes applications.

my-api-chart/
  Chart.yaml
  values.yaml
  templates/
    deployment.yaml
    service.yaml
    ingress.yaml

For example, the development environment and the operating environment can use the same template, but only have different replicas, image tags, resources, and ingress hosts.

# values-prod.yaml
image:
  repository: ghcr.io/example/api
  tag: "1.0.0"

replicaCount: 3

resources:
  requests:
    cpu: 250m
    memory: 256Mi
  limits:
    cpu: 500m
    memory: 512Mi

ingress:
  enabled: true
  host: api.example.com

Distribution is performed as follows.

helm lint ./charts/api
helm template api ./charts/api -f values-prod.yaml
helm upgrade --install api ./charts/api -n production -f values-prod.yaml

Helm is convenient, but if the template becomes too complicated, it becomes a deployment system that is difficult to read. A good chart is not a chart that abstracts everything, but a chart that separates values ​​that the team frequently changes from structures that should not be changed. It is also important to make a habit of checking the YAML that is actually created with the helm template.

Kustomize is a different direction from Helm. Combine YAML using base and overlay rather than a template language. Helm is strong in packaging and parameterization, and Kustomize is good at keeping the existing YAML relatively intact and adding environment-specific differences. One is always better than the other; you should choose whichever is easier for your team to read and review changes.

Operator is a method of transferring operating knowledge to the controller.

Operator is a pattern that extends Kubernetes. It goes beyond simply installing Deployment and Service, and puts the operating procedures of a specific system into the controller. The core is CRD (Custom Resource Definition), CR (Custom Resource), and Custom Controller.

CRD: defines a custom resource type named Prometheus
CR: requests an actual Prometheus instance
Controller: watches the CR, creates the required StatefulSet, Service, and Config, and reconciles state

Taking Prometheus Operator as an example, users do not directly create complex StatefulSets and ConfigMap to configure the Prometheus server. Create Custom Resources such as Prometheus, ServiceMonitor, and PodMonitor, and the Operator looks at them and configures the necessary sub-resources.

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: api
spec:
  selector:
    matchLabels:
      app: api
  endpoints:
    - port: http
      path: /metrics
      interval: 30s

The Operator becomes a controller who is comfortable but has strong authority within the cluster. You need to check which CRDs are installed, which namespaces are monitored, and whether RBAC permissions are excessive. Operator Lifecycle Manager (OLM) or Operator Hub assists with installation and lifecycle management, but does not eliminate operational responsibility.

GitOps is more about change control than deployment automation

GitOps is an operating method that sets the Git repository as the desired state of the cluster and synchronizes it with the actual cluster state. ArgoCD is a representative tool that implements this model in Kubernetes.

The general flow is as follows.

1. Application code changes
2. CI runs tests and builds the image
3. The image is pushed to the registry
4. Helm values or manifest image tags change in the deployment repository
5. A Pull Request is reviewed
6. Argo CD detects the change after merge
7. Cluster state synchronizes with the desired state in Git

The important thing about this structure is that it reduces the need to manually fix the cluster. If the operator makes any modifications with kubectl edit deployment, ArgoCD detects the difference between Git and the actual state as drift. Depending on the Sync Policy, it can be automatically reverted or synchronized after manual approval.

Installing ArgoCD is relatively simple.

kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

Initially, you can replace the argocd-server service with LoadBalancer for external access. For practice or local verification, port-forward is safer and simpler.

kubectl patch svc argocd-server -n argocd -p '{"spec": {"type": "LoadBalancer"}}'

kubectl port-forward svc/argocd-server -n argocd 8080:443

Check the initial password in Secret or look it up using CLI.

kubectl -n argocd get secret argocd-initial-admin-secret \
  -o jsonpath="{.data.password}" | base64 -d

argocd admin initial-password -n argocd
argocd login <ARGOCD_SERVER>
argocd account update-password

You can check the basic operation of ArgoCD by deploying the Guestbook example.

kubectl config set-context --current --namespace=argocd

argocd app create guestbook \
  --repo https://github.com/argoproj/argocd-example-apps.git \
  --path guestbook \
  --dest-server https://kubernetes.default.svc \
  --dest-namespace default

argocd app get guestbook
argocd app sync guestbook

In a production environment, the storage structure is more important than the examples. You need to decide whether to separate the application source repository, deployment configuration repository, and platform add-on repository.

app-source-repo
- application code
- Dockerfile
- tests

app-deploy-repo
- Helm chart or manifests
- values-dev.yaml
- values-staging.yaml
- values-prod.yaml

platform-addons-repo
- ingress controller
- cert-manager
- external-secrets
- prometheus stack
- logging stack

Splitting up repositories makes it easier to separate change responsibilities, but also increases flow. Putting them all in one repository is simple, but it can blur the boundaries between permissions and reviews. What is more important in GitOps than the tool name is the structure in which changes remain as Pull Requests, only approved settings are reflected in the cluster, and rollbacks are linked to Git history.

ArgoCD Application can also be declared according to this structure.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: api-prod
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/example/deployments.git
    targetRevision: main
    path: apps/api/chart
    helm:
      valueFiles:
        - values-prod.yaml
  destination:
    server: https://kubernetes.default.svc
    namespace: production
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true
`

```prune: true` means that resources that have disappeared from Git will be removed from the cluster. Although it is convenient, it is also a dangerous setting. If the structure requires resource deletion, you must check not only “newly added resources” but also “disappearing resources” in the PR review. `selfHeal: true` returns settings manually changed in the cluster based on Git. Since the content that the operator hastily modified with `kubectl edit` can be restored, there must be a procedure to eventually reflect emergency changes in Git.

## Terraform and Ansible manage cluster transfer steps

Kubernetes resources can be managed with GitOps, but the Kubernetes cluster itself has a separate infrastructure lifecycle. Terraform is best left to creating AKS or EKS, configuring node pools, connecting networks and permissions, and issuing kubeconfig.

```hcl
resource "azurerm_resource_group" "platform" {
  name     = "rg-platform-prod"
  location = "koreacentral"
}

resource "azurerm_kubernetes_cluster" "main" {
  name                = "aks-platform-prod"
  location            = azurerm_resource_group.platform.location
  resource_group_name = azurerm_resource_group.platform.name
  dns_prefix          = "platform-prod"

  default_node_pool {
    name       = "system"
    node_count = 3
    vm_size    = "Standard_D2s_v5"
  }

  identity {
    type = "SystemAssigned"
  }
}

When using Terraform, the important thing is not “the cluster has been created” but whether it can be created again with the same definition. Bundling resource groups, VNet/Subnet, AKS, ACR, Key Vault, Log Analytics Workspace, DNS, and permission assignments in code improves environment reproducibility. Conversely, as the number of settings manually changed in the portal increases, the Terraform state and actual resources begin to diverge.

Ansible is in a slightly different position. In managed Kubernetes, Ansible's role is reduced, but in configurations such as VM-based clusters or Kubespray, it is useful for node preparation tasks. For example, disabling swap, kernel parameters, package installation, SSH access, and container runtime configuration are between declarative infrastructure and operational scripts.

- name: Prepare Kubernetes nodes
  hosts: kube_nodes
  become: true
  tasks:
    - name: Disable swap
      command: swapoff -a

    - name: Enable bridge netfilter
      modprobe:
        name: br_netfilter
        state: present

    - name: Configure sysctl for Kubernetes networking
      copy:
        dest: /etc/sysctl.d/k8s.conf
        content: |
          net.bridge.bridge-nf-call-iptables = 1
          net.ipv4.ip_forward = 1

From an operational perspective, it is important not to mix the responsibilities of Terraform, Ansible, and ArgoCD.

Terraform
- cloud resources
- Kubernetes cluster
- network, IAM, registry, logging workspace

Ansible
- VM node bootstrap
- OS package and kernel settings
- self-managed cluster preparation

ArgoCD / GitOps
- Kubernetes workloads
- Helm values and manifests
- platform add-ons

Setting these boundaries will make your questions clear even when obstacles arise. If there is no node or the network is incorrect, it is the Terraform area. If the node OS settings are incorrect, it is the Ansible area. If Deployment or Ingress is different from Git, it is the ArgoCD area.

AKS provisioning involves leaving operational input rather than creating a cluster.

When creating AKS with Terraform, looking at just one azurerm_kubernetes_cluster resource is not enough. Actual operational inputs are determined together with network, node pool, ACR connection, log workspace, Key Vault access, upgrade policy, and permission model. What is more important than creating a cluster is leaving code and variables so that the same judgment can be repeated in the next environment.

variable "environment" {
  type    = string
  default = "prod"
}

variable "location" {
  type    = string
  default = "koreacentral"
}

resource "azurerm_log_analytics_workspace" "platform" {
  name                = "log-platform-${var.environment}"
  location            = var.location
  resource_group_name = azurerm_resource_group.platform.name
  sku                 = "PerGB2018"
  retention_in_days   = 30
}

resource "azurerm_kubernetes_cluster" "main" {
  name                = "aks-platform-${var.environment}"
  location            = var.location
  resource_group_name = azurerm_resource_group.platform.name
  dns_prefix          = "platform-${var.environment}"

  default_node_pool {
    name                = "system"
    vm_size             = "Standard_D2s_v5"
    node_count          = 3
    vnet_subnet_id      = azurerm_subnet.aks.id
    type                = "VirtualMachineScaleSets"
    orchestrator_version = null
  }

  identity {
    type = "SystemAssigned"
  }

  oms_agent {
    log_analytics_workspace_id = azurerm_log_analytics_workspace.platform.id
  }

  network_profile {
    network_plugin    = "azure"
    load_balancer_sku = "standard"
    outbound_type     = "loadBalancer"
  }
}

In this example, the key is the surrounding dependencies rather than AKS itself. If Log Analytics is omitted, failure analysis becomes difficult, and if the subnet is not clearly specified, the network policy and IP plan become unstable later. From the moment you attach ACR, Key Vault, and DNS Zone, the cluster becomes part of the platform rather than an independent resource.

Terraform execution flow should also be left as an operating procedure.

terraform init -upgrade
terraform fmt -recursive
terraform validate
terraform plan -out main.tfplan
terraform apply main.tfplan

az aks get-credentials \
  --resource-group rg-platform-prod \
  --name aks-platform-prod \
  --overwrite-existing

kubectl get nodes -o wide
kubectl get pods -A

What you need to look at in the terraform plan results is not only the number of creations. You should look for changes that could lead to failures, such as re-creating a node pool, changing a subnet, replacing an identity, or deleting a role assignment. In particular, AKS can lead to replacement rather than simple modification if the node pool and network settings change. So in PR, it is better to treat Terraform diff as part of the code review.

State management cannot be left out either. Local state is convenient for individual practice, but there is a risk of conflict and loss in a collaborative environment.

terraform {
  backend "azurerm" {
    resource_group_name  = "rg-tfstate-prod"
    storage_account_name = "sttfstateprod001"
    container_name       = "tfstate"
    key                  = "platform/aks-prod.tfstate"
  }
}

Using remote state makes it easier to track who applied what infrastructure changes and when. However, because the state file may contain resource identifiers and some sensitive values, Storage Account access rights should also be minimized.

Kubespray asks different questions than managed Kubernetes

In managed Kubernetes such as AKS/EKS, the burden of operating the control plane is taken over by the cloud. On the other hand, configuring a cluster on top of a VM with Kubespray requires more direct understanding of the control plane, etcd, worker nodes, container runtime, certificates, and upgrade procedures. This experience is helpful even if you use a managed service. This is because when a failure occurs, you can more quickly narrow down which layer the problem is, rather than saying “Kubernetes is not working.”

self-managed cluster preparation

1. prepare Linux nodes and SSH access
2. disable swap and configure kernel modules
3. configure container runtime
4. define inventory for control plane and workers
5. run Kubespray playbook
6. verify kubeconfig and cluster components
7. apply CNI, storage class, ingress, monitoring

The advantage of Kubespray is that repeatable cluster installation can be left with Ansible inventory and playbooks. However, operational responsibility also increases. You must manually check etcd backups, certificate expiration, control plane upgrades, node OS patches, and even CNI compatibility.

[kube_control_plane]
cp-01 ansible_host=10.10.1.10
cp-02 ansible_host=10.10.1.11
cp-03 ansible_host=10.10.1.12

[kube_node]
worker-01 ansible_host=10.10.2.10
worker-02 ansible_host=10.10.2.11

[etcd:children]
kube_control_plane

[k8s_cluster:children]
kube_control_plane
kube_node

Whether you choose managed Kubernetes or build your own is a matter of operating model, not technology preference. If a small team needs to quickly operate a product, AKS/EKS is realistic, and if the environment requires strong control of the network and OS layers, self-construction or private PaaS should be considered. In either case, it is safer to separate the scope of GitOps and infrastructure provisioning.

Deployment strategy is about determining the scope of failure, not YAML options

Kubernetes Deployment supports RollingUpdate by default. Creates a new ReplicaSet and reduces the existing pods when the new pods are ready.

strategy:
  type: RollingUpdate
  rollingUpdate:
    maxUnavailable: 0
    maxSurge: 1
`

```maxUnavailable: 0` means that the existing available pods will not be reduced even during deployment, and `maxSurge: 1` means that one more pod can be temporarily launched. This setting requires resource availability, but is stable for services with traffic.

If you need Blue/Green or Canary, you can look into tools like Argo Rollouts. Canary is a method of sending only a portion of traffic to a new version and gradually increasing it by looking at the metrics. At this time, it can be automatically stopped based on error rate or latency by linking with Prometheus. The important thing is not to “use a canary,” but to decide which indicators to stop deployment when they get worse.

```txt
Canary decision signals
- 5xx error rate
- p95 / p99 latency
- pod restart count
- CPU throttling
- business transaction failure

Observability should go into resource design, not after deployment

When looking at problems in Kubernetes, Pod status alone is not enough. Deployment rollout, Service endpoint, Ingress routing, Node resource, application log, metrics, and trace must be viewed together.

kubectl get deploy,rs,pod,svc,ingress
kubectl get endpoints api
kubectl top pod
kubectl top node
kubectl describe ingress api
kubectl logs deploy/api -f

Prometheus/Grafana reports metrics, Loki or OpenSearch help with log searches, and Tempo or Jaeger provide distributed tracing. By attaching OpenTelemetry to your application, you can follow HTTP requests, DB queries, and external API calls as a single trace.

Istio, a service mesh, provides mTLS, traffic policy, retry, timeout, circuit breaking, and telemetry. However, adding a mesh from the beginning increases operational complexity. It is better to first review the Service, Ingress, Probe, Resource, and logs/metrics when they are stably organized and the need for traffic control becomes clear.

Cleanup

Kubernetes starts out as a tool to launch containers, but in reality, it is more of a platform for declaring and coordinating the operating state of an application. Deployment defines rollout and recovery units, Service and Ingress hide pod replacement, and ConfigMap and Secret separate configuration from the image. Helm manages differences in each environment, and Operator moves repetitive operating procedures to the controller.

GitOps and ArgoCD are not a method of creating one more deployment button, but an operating model that matches change history and cluster status based on Git. For this model to work properly, we need to keep asking more basic questions than just using a lot of YAML. Which state will be the desired state, who will approve changes, revert when drift occurs, and stop deployment when some indicator gets worse.

Ultimately, Kubernetes operations do not end with “successful deployment.” Requests come in, pods are ready, services can deliver traffic, deployments can progress incrementally, and when problems arise, logs and metrics should be able to narrow down the cause. When the entire flow is managed with Git and declarative settings, Kubernetes becomes a platform beyond a simple execution environment.