Resource Requests and Limits Explained

Effectively managing resources within a Kubernetes cluster is critical for both application stability and cost optimization. Kubernetes employs a robust…

Effectively managing resources within a Kubernetes cluster is critical for both application stability and cost optimization. Kubernetes employs a robust mechanism of resource requests and limits to enable granular control over how pods consume CPU and memory. Understanding the nuances of these configurations is fundamental for network engineers optimizing service mesh performance, sysadmins maintaining node health, cloud consultants right-sizing clusters, and developers ensuring their applications run reliably.

This article delves into the specifics of CPU and memory requests and limits, their impact on pod scheduling and runtime behavior, and best practices for their implementation in production environments.

CPU Requests and Limits

CPU resources in Kubernetes are measured in "CPU units," where 1 CPU unit is equivalent to 1 vCPU on a public cloud VM, 1 AWS EC2 Compute Unit (ECU), or 1 core on a bare-metal machine. Fractional values are also supported, such as 0.5 or 500m (500 millicores).

CPU Requests: Guaranteed Minimums for Scheduling

A CPU request specifies the minimum amount of CPU resources a container needs. The Kubernetes scheduler uses this value to determine which node can accommodate a pod. If a node does not have enough unallocated CPU capacity to meet all the requests of a pod's containers, the pod will not be scheduled on that node. This mechanism ensures that a scheduled pod has a guaranteed minimum CPU share available.

• Impact on Scheduling: The scheduler reserves this amount for the pod. Without sufficient available requested CPU on a node, the pod remains in a Pending state.
• No Runtime Enforcement: CPU requests themselves do not restrict a pod's CPU usage at runtime. A pod can burst beyond its requested CPU if there are idle CPU cycles available on the node.

CPU Limits: Enforced Maximums for Runtime

A CPU limit defines the maximum amount of CPU resources a container is allowed to use. If a container attempts to use CPU beyond its limit, the Linux kernel's Completely Fair Scheduler (CFS) will throttle the process, reducing its CPU allocation. This prevents a single misbehaving container from monopolizing the node's CPU and impacting other workloads.

• Runtime Enforcement: Implemented via CFS quotas (cpu.cfs_period_us and cpu.cfs_quota_us in cgroups v1, or cpu.max in cgroups v2).
• Throttling: Exceeding the CPU limit leads to throttling. The process will experience increased latency and reduced throughput, but it will not be terminated.
• Unit Consistency: CPU limits use the same units as CPU requests (e.g., 1, 500m).

apiVersion: v1
kind: Pod
metadata:
  name: cpu-intensive-pod
spec:
  containers:
  - name: my-app
    image: myrepo/my-app:1.0.0
    resources:
      requests:
        cpu: "200m"  # Request 0.2 CPU core
      limits:
        cpu: "500m"  # Limit to 0.5 CPU core
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: kubernetes.io/arch
            operator: In
            values:
            - amd64

Memory Requests and Limits

Memory resources in Kubernetes are measured in bytes. Standard suffixes like Mi (mebibytes) and Gi (gibibytes) are commonly used (e.g., 256Mi, 2Gi).

Memory Requests: Guaranteed Minimums for Scheduling

A memory request specifies the minimum amount of memory a container needs. Similar to CPU requests, the scheduler uses this value to place pods on nodes that have sufficient unallocated memory. The requested memory is reserved for the pod, meaning other pods cannot use this portion of the node's memory capacity for their own requests.

• Impact on Scheduling: The scheduler considers a node's allocatable memory (total memory minus OS/kubelet overhead) and the sum of all requested memory by pods already on that node.
• No Runtime Enforcement (Directly): While requested memory is reserved, a container can attempt to use more memory than its request if it's available and below its limit.

Memory Limits: Enforced Maximums with OOMKills

A memory limit defines the maximum amount of memory a container is allowed to use. If a container attempts to allocate memory beyond its limit, the Linux Out-Of-Memory (OOM) killer will terminate the container. This is a critical difference from CPU limits: exceeding a memory limit results in termination, not just throttling.

• Runtime Enforcement: Implemented via cgroups memory limits (memory.limit_in_bytes).
• OOMKilled: When a container exceeds its memory limit, its process is sent a SIGKILL signal, causing the container to terminate. The pod will then show a OOMKilled status in its container state.
• Pod Restart Policy: If a pod's restartPolicy is Always or OnFailure, the kubelet will attempt to restart the OOMKilled container, potentially leading to a restart loop if the memory issue is not resolved.

apiVersion: v1
kind: Pod
metadata:
  name: memory-hungry-pod
spec:
  containers:
  - name: data-processor
    image: myrepo/data-processor:latest
    resources:
      requests:
        memory: "512Mi"  # Request 512 MiB
      limits:
        memory: "1Gi"    # Limit to 1 GiB

Quality of Service (QoS) Classes

Kubernetes assigns a Quality of Service (QoS) class to each pod based on its resource requests and limits. This classification influences how the kubelet prioritizes pod eviction during node resource contention.

Understanding QoS classes is crucial for sysadmins managing heterogeneous workloads. BestEffort pods are the first to be terminated if a node experiences memory pressure, followed by Burstable pods. Guaranteed pods are only evicted as a last resort, typically when a critical system component requires resources.

Recommendations and Best Practices

Implementing resource requests and limits effectively requires careful planning and continuous monitoring.

• Always Set Requests: This is fundamental for reliable scheduling and ensuring your applications have minimum resources. Without requests, your pods are BestEffort and can be preempted or evicted easily.
• Start Without Limits (Initial Learning): For new applications, consider deploying without memory limits initially (but with requests). This allows you to observe the pod's actual memory consumption under various loads without risking OOMKills. Use tools like Prometheus and Grafana (specifically container_memory_usage_bytes) or kubectl top pod to gather data.
• Set Conservative Limits First: Once you have a baseline of actual usage, set memory limits slightly above the observed peak usage (e.g., 1.2x peak). This provides a buffer for spikes while still protecting the node from runaway processes.
• CPU Limits Trade-offs:
  • Hard CPU Limits (Request = Limit): Provides predictable performance but might leave CPU idle if other processes aren't using it. Can prevent bursting. Use for critical, latency-sensitive applications.
  • Soft CPU Limits (Limit > Request): Allows bursting up to the limit, utilizing idle CPU. This is often a good balance for many applications. However, if many pods burst simultaneously, it can lead to throttling for all of them.
  • No CPU Limits: Allows a pod to consume all available CPU if needed. Can be dangerous as a single misbehaving pod can starve the entire node. Only use for very specific, trusted, and isolated workloads.
• Monitor and Adjust: Resource requirements are rarely static. Continuously monitor application performance, resource usage metrics, and OOMKills. Adjust requests and limits as application load patterns change or new versions are deployed. Vertical Pod Autoscalers (VPAs) can automate this process based on observed usage.
• Node Allocatable: Be aware of the allocatable capacity on your nodes. Kubernetes reserves some CPU and memory for the OS and kubelet processes. Your pods can only consume up to the allocatable amount, not the node's total physical resources. Check with kubectl describe node <node-name> under the "Allocatable" section.
• Resource Overhead: Account for sidecar containers (e.g., Istio proxies, logging agents) when calculating total pod resources. Each container within a pod contributes to the pod's overall request and limit.

Common Pitfalls and Troubleshooting

• OOMKilled Pods: The most common memory-related issue. Check kubectl describe pod <pod-name> for the OOMKilled status and the exit code (often 137 or 1). Review application logs for signs of memory leaks or increased memory consumption under load. Increase the memory limit cautiously.
• CPU Throttling: Indicated by performance degradation without termination. Monitor container_cpu_cfs_throttled_periods_total and container_cpu_cfs_throttled_seconds_total metrics (from cAdvisor/kubelet via Prometheus). If throttling is high, either the CPU limit is too low for the workload, or the application needs optimization.
• Pod Pending State: If pods remain in Pending, it's usually due to insufficient resources on available nodes. Run kubectl describe pod <pod-name> and look for scheduler messages like "0/N nodes are available: insufficient cpu, insufficient memory." This indicates that your requests are higher than what's available on any node, or your cluster is overall undersized.
• Ignoring QoS Classes: Running critical applications as BestEffort or Burstable when they require guaranteed resources can lead to unexpected evictions during node pressure, especially for stateful workloads.
• Setting Arbitrary Limits: Guessing resource values without profiling can lead to either wasted resources (limits too high) or instability (limits too low). Always base your limits on observed usage patterns.