Kubernetes HPA: CPU, memory, and custom-metric autoscaling

The Horizontal Pod Autoscaler (HPA) in Kubernetes automatically adjusts the number of pod replicas in a deployment, statefulset, or replica set based on…

The Horizontal Pod Autoscaler (HPA) in Kubernetes automatically adjusts the number of pod replicas in a deployment, statefulset, or replica set based on observed metrics. This dynamic scaling ensures application performance and resource efficiency by provisioning only the necessary resources to handle current load. This article explores configuring HPA for CPU, memory, and custom metrics, detailing prerequisites, configurations, and common operational considerations.

HPA Fundamentals and Prerequisites

HPA operates by regularly querying metrics and comparing them against defined targets. If the metrics exceed (or fall below) the target, HPA initiates a scaling action to increase (or decrease) the number of pods. For HPA to function correctly, several components must be in place:

• Kubernetes Metrics Server: This cluster add-on collects resource metrics (CPU, memory) from all nodes and exposes them via the Metrics API (/apis/metrics.k8s.io/v1beta1). HPA relies on Metrics Server for default CPU and memory-based autoscaling. Ensure it's installed and healthy in your cluster. You can verify its status with kubectl get apiservice v1beta1.metrics.k8s.io.
• Resource Requests/Limits: For CPU and memory-based HPA, your containers must have resource requests defined in their pod specifications. HPA calculates utilization as a percentage of these requested resources. Without requests, HPA cannot determine utilization and will report <unknown> for the target metric.
• Custom Metrics Provider: For scaling based on application-specific or infrastructure metrics (e.g., HTTP requests per second, queue length), a custom metrics API server is required. Common choices include:
  • Prometheus Adapter: Integrates Prometheus with the Kubernetes custom metrics API. It scrapes metrics from Prometheus and exposes them in a format HPA understands.
  • KEDA (Kubernetes Event-driven Autoscaling): Extends HPA capabilities to scale based on a wide range of external event sources like message queues, databases, and serverless functions, often providing more robust integration patterns than a pure Prometheus Adapter.

Kubernetes API Version for HPA

As of Kubernetes 1.23+, autoscaling/v2 (formerly autoscaling/v2beta2) is the stable API version for HPA, supporting CPU, memory, pod-level custom metrics, and external metrics all within a single resource definition. Earlier versions like autoscaling/v1 only supported CPU utilization and had a more limited feature set.

CPU and Memory-Based HPA Configuration

Scaling based on CPU and memory utilization is the most common and straightforward HPA use case. The HPA calculates the average utilization across all pods managed by the target controller and adjusts the replica count to bring this average closer to the defined target.

apiVersion: autoscaling.k8s.io/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web-hpa
  namespace: default
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web-app
  minReplicas: 2
  maxReplicas: 20
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: 80
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300 # Wait 5 minutes before scaling down
      policies:
      - type: Percent
        value: 50
        periodSeconds: 60
      - type: Pods
        value: 2
        periodSeconds: 60
    scaleUp:
      stabilizationWindowSeconds: 0 # Scale up immediately
      policies:
      - type: Percent
        value: 100
        periodSeconds: 15
      - type: Pods
        value: 4
        periodSeconds: 15

Explanation of fields:

• scaleTargetRef: Specifies the target resource to scale (e.g., a Deployment named web-app).
• minReplicas: The minimum number of replicas the HPA will maintain.
• maxReplicas: The maximum number of replicas the HPA will scale up to.
• metrics: An array defining the scaling criteria.
  • type: Resource: Indicates scaling based on built-in Kubernetes resources.
  • resource: { name: cpu, target: { type: Utilization, averageUtilization: 70 } }: Scales to maintain an average CPU utilization of 70% across all pods.
  • resource: { name: memory, target: { type: Utilization, averageUtilization: 80 } }: Scales to maintain an average memory utilization of 80% across all pods. Note that memory autoscaling is often less effective than CPU due to application memory usage patterns (e.g., garbage collection, caches) and can lead to thrashing if not carefully tuned. It's often better to scale primarily on CPU or custom metrics.
• behavior: (Kubernetes 1.18+) Provides fine-grained control over scaling policies.
  • scaleDown.stabilizationWindowSeconds: The number of seconds the HPA must observe metrics below the target before initiating a scale-down. This prevents rapid "flapping" of replica counts. A common value is 300 seconds (5 minutes).
  • scaleUp.stabilizationWindowSeconds: The number of seconds the HPA must observe metrics above the target before initiating a scale-up. A value of 0 means immediate scale-up if metrics dictate.
  • policies: Defines how many pods can be scaled up or down in a given period. The HPA uses the policy that allows the fastest change.
    • type: Percent, value: 50, periodSeconds: 60: Scale down by at most 50% of the current replicas every 60 seconds.
    • type: Pods, value: 2, periodSeconds: 60: Scale down by at most 2 pods every 60 seconds.

Custom Metric Autoscaling (Pod-level)

Pod-level custom metrics allow you to scale based on metrics that are reported by individual pods, typically exposed via an application's Prometheus endpoint. The HPA then averages these metrics across all pods.

apiVersion: autoscaling.k8s.io/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web-app-rps-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web-app
  minReplicas: 3
  maxReplicas: 30
  metrics:
  - type: Pods
    pods:
      metric:
        name: http_requests_total_per_second # This metric must be exposed by your app and scraped by Prometheus
      target:
        type: AverageValue
        averageValue: "50" # Scale to maintain an average of 50 requests per second per pod

Prerequisites for Custom Metrics:

• Prometheus and Prometheus Adapter: Prometheus scrapes application metrics (e.g., http_requests_total_per_second). The Prometheus Adapter then exposes these metrics via the Kubernetes Custom Metrics API. You need to configure the Prometheus Adapter with rules to map Prometheus queries to custom metric names. For example, a rule might look like this:

  - seriesQuery: '{__name__="http_requests_total"}'
    resources:
      template: << .ResourceLabels >>
    name:
      matches: "http_requests_total"
      as: "http_requests_total_per_second" # The name HPA uses
    metricsQuery: sum(rate(<<.Series>>{<<.LabelMatchers>>}[5m])) by (pod)
  

  This example takes the rate of http_requests_total over 5 minutes, sums it per pod, and exposes it as http_requests_total_per_second.
• Application Instrumentation: Your application must expose the desired metrics, usually in the Prometheus exposition format (e.g., on /metrics endpoint).

External Metric Autoscaling

External metrics allow scaling based on metrics originating from outside the Kubernetes cluster, such as queue lengths in a message broker, database connection pools, or object counts in cloud storage. This is particularly useful for event-driven architectures.

apiVersion: autoscaling.k8s.io/v2
kind: HorizontalPodAutoscaler
metadata:
  name: worker-sqs-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: worker-app
  minReplicas: 1
  maxReplicas: 10
  metrics:
  - type: External
    external:
      metric:
        name: sqs_approximate_number_of_messages_visible
        selector:
          matchLabels:
            queue: orders-processing
            region: us-east-1
      target:
        type: AverageValue
        averageValue: "30" # Scale to process messages so that there are on average 30 messages visible per worker pod

Prerequisites for External Metrics:

• Custom Metrics Provider (e.g., KEDA or Prometheus Adapter with external exporters): KEDA excels here, offering built-in scalers for many external services (AWS SQS, Azure Service Bus, Kafka, RabbitMQ, etc.). Prometheus Adapter can also be used if you have a Prometheus exporter that scrapes the external metric and Prometheus itself is configured to scrape that exporter.
  • KEDA Example: KEDA defines ScaledObject resources that link your deployment to an external scaler. When KEDA detects a need to scale, it updates the HPA object in Kubernetes with the appropriate external metric definition.
  • Prometheus Adapter Example: You would need a Prometheus exporter (e.g., sqs_exporter) to scrape SQS metrics, and then configure Prometheus Adapter to expose these metrics.
• selector: Crucial for external metrics. It allows you to select specific instances of an external metric. For example, queue: orders-processing distinguishes between different SQS queues.

Inspecting HPA Status

Once an HPA is deployed, you can monitor its state and activity using kubectl commands.

• kubectl get hpa: Provides a high-level overview of all HPAs, including their current status (target, actual replicas, minimum/maximum replicas).

  NAME                 REFERENCE               TARGETS          MINPODS   MAXPODS   REPLICAS   AGE
  web-hpa              Deployment/web-app      75%/70%, 82%/80%   2         20        5          2d
  web-app-rps-hpa      Deployment/web-app      55/50            3         30        6          1d
  worker-sqs-hpa       Deployment/worker-app   35/30            1         10        4          10h
  

• kubectl describe hpa <hpa-name>: Provides detailed information, including events, current metrics, and status conditions. This is invaluable for debugging.

  Name:                               web-hpa
  Namespace:                          default
  Labels:                             <none>
  Annotations:                        <none>
  CreationTimestamp:                  Mon, 15 Jan 2024 10:00:00 +0000
  Reference:                          Deployment/web-app
  Metrics:                            (current / target)
    resource cpu:                     75% (500m) / 70%
    resource memory:                  82% (1024Mi) / 80%
  Min Replicas:                       2
  Max Replicas:                       20
  Deployment pods:                    5 current / 5 desired
  Conditions:
    Type            Status  Reason            Message
    ----            ------  ------            -------
    AbleToScale     True    ReadyForNewScale  the last global stabilization target was 5
    ScalingActive   True    ValidMetricFound  the HPA was able to compute the replica count based on cpu resource
    ScalingLimited  False   DesiredWithinRange  the desired replica count is within the acceptable range
  Events:
    Type     Reason                        Age    From                       Message
    ----     ------                        ----   ----                       -------
    Normal   SuccessfulRescale             2m     horizontal-pod-autoscaler  New size: 5; reason: All metrics below target
    Normal   SuccessfulRescale             10m    horizontal-pod-autoscaler  New size: 6; reason: cpu resource utilization (90%) above target
    Normal   DesiredReplicasComputed       12m    horizontal-pod-autoscaler  Computed desired replica count of 6 (average CPU utilization (90%) is above target 70%)
  

• kubectl top pods: Shows real-time CPU and memory usage for pods, useful for verifying the metrics HPA is observing. Note this only works if Metrics Server is installed.

Common Pitfalls and Troubleshooting

• No Resource Requests/Limits: If pods lack CPU/memory requests, HPA will report <unknown> for resource metrics and will not scale. Always define requests.
• Metrics Server Not Running/Healthy: HPA cannot get CPU/memory metrics without a functioning Metrics Server. Check its deployment and logs.
• Aggressive Scale-Down Leading to Flapping: Setting stabilizationWindowSeconds too low or having overly aggressive scale-down policies can cause replica counts to rapidly decrease, then immediately increase as load returns. Increase stabilizationWindowSeconds for scale-down to give the system time to stabilize.
• Custom/External Metrics Provider Misconfiguration: If HPA reports "failed to get custom metric" or "failed to get external metric," verify your Prometheus Adapter or KEDA configuration. Check logs of the adapter/KEDA deployment, and ensure your Prometheus queries or KEDA scalers are correctly defined and matching the metric names and selectors in your HPA.
• Scale Up/Down Thrashing: If your application has a slow startup time, rapid scale-up and scale-down can be problematic. Balance stabilizationWindowSeconds and policies carefully. Consider pre-warming instances if startup is very slow.
• Metric Granularity and Latency: The HPA controller typically polls metrics every 15-30 seconds (configurable). If your custom metrics have a significant delay (e.g., Prometheus scrape interval of 60s), HPA reacts based on slightly stale data. Avoid scaling on very short-lived spikes or sub-15-second signals.
• Min/Max Replicas Too Restrictive: Ensure your minReplicas is sufficient for baseline load and your maxReplicas allows enough headroom for peak loads without hitting resource limits of your cluster.
• Memory HPA Instability: Memory utilization can be less predictable than CPU. Applications may hold onto memory even when not actively using it, or garbage collection might cause sudden drops. This can lead to undesirable scaling behavior. Prioritize CPU or request-based custom metrics for autoscaling if possible.