Stackvora.tech

GCP: create a GKE Autopilot cluster

Google Kubernetes Engine (GKE) Autopilot provides a fully managed Kubernetes experience, abstracting away node provisioning, scaling, patching, and…

Google Kubernetes Engine (GKE) Autopilot provides a fully managed Kubernetes experience, abstracting away node provisioning, scaling, patching, and underlying infrastructure management. Unlike GKE Standard, where users manage node pools, Autopilot automatically optimizes cluster resources for your workloads, making it ideal for teams prioritizing operational simplicity and cost efficiency based on actual resource consumption. This article details the process of creating and interacting with a GKE Autopilot cluster, highlighting key operational differences and considerations.

Understanding GKE Autopilot's Operational Model

GKE Autopilot fundamentally alters the cluster management paradigm. Instead of managing node pools, their sizes, and upgrades, Autopilot provisions and scales nodes in response to pod resource requests. This "pay-per-pod" model simplifies cost allocation and aligns expenses directly with application resource consumption.

  • Node Management: Autopilot handles all node lifecycle operations, including provisioning, upgrades, repairs, and scaling. You define pod resource requests (CPU, memory), and Autopilot ensures the underlying nodes can satisfy them.
  • Security Baseline: Autopilot enforces a hardened security configuration by default, including Workload Identity, Shielded GKE Nodes, and automatic node OS patching.
  • Resource Limits & Quotas: Resource requests and limits are mandatory for pods deployed to Autopilot. This is crucial for Autopilot to make intelligent scheduling and scaling decisions. Without them, pods might fail to schedule.
  • Version Management: Autopilot clusters are automatically upgraded to new GKE versions within their release channel, minimizing manual intervention.

Prerequisites for GKE Autopilot

Before creating an Autopilot cluster, ensure the following are in place:

  • GCP Project: You need an active Google Cloud Project.
  • Billing Account: A linked active billing account is required.
  • GKE API: The Kubernetes Engine API must be enabled for your project. You can enable it via the GCP Console (Navigation Menu > Kubernetes Engine > Clusters) or using the gcloud CLI: 
    gcloud services enable container.googleapis.com
  • gcloud CLI: Ensure your Google Cloud SDK is installed and authenticated. Update it to the latest version for access to new GKE features: 
    gcloud components update

Creating a GKE Autopilot Cluster

Creating an Autopilot cluster is straightforward using the gcloud CLI. The key difference from GKE Standard is the --create-auto flag.

Basic Cluster Creation

To create an Autopilot cluster named hello-cluster in the europe-west1 region:

gcloud container clusters create-auto hello-cluster \
    --region=europe-west1 \
    --release-channel=regular \
    --project=your-gcp-project-id \
    --async
  • --region: Specifies the Google Cloud region for the cluster. For multi-regional deployments or higher availability, consider zonal clusters (e.g., europe-west1-b) or separate regional clusters.
  • --release-channel: Autopilot clusters require a release channel. Recommended channels are regular for balanced stability and features, or stable for maximum stability. rapid offers the latest features but might have less testing.
  • --project: Your GCP project ID. While often inferred from your active gcloud config, explicitly specifying it is good practice.
  • --async: Allows the command to return immediately while the cluster creation proceeds in the background. You can check its status with gcloud container operations list.

Cluster creation can take 5-10 minutes. You can monitor its status via the GCP Console (Kubernetes Engine > Clusters) or by omitting the --async flag.

Advanced Autopilot Cluster Configuration

While Autopilot abstracts away many node-level configurations, you can still customize aspects like networking, IP allocation, and Workload Identity.

gcloud container clusters create-auto production-autopilot-cluster \
    --region=us-central1 \
    --release-channel=stable \
    --workload-identity-config=enabled \
    --enable-private-nodes \
    --master-ipv4-cidr-block=172.16.0.0/28 \
    --enable-master-authorized-networks \
    --master-authorized-networks=192.168.1.0/24,10.0.0.0/24 \
    --network=projects/your-gcp-project-id/global/networks/your-vpc-network \
    --subnetwork=projects/your-gcp-project-id/regions/us-central1/subnetworks/your-subnetwork \
    --cluster-secondary-range-name=gke-pods-range \
    --services-secondary-range-name=gke-services-range \
    --project=your-gcp-project-id
  • --workload-identity-config=enabled: Enables Workload Identity, allowing Kubernetes Service Accounts to act as GCP Service Accounts for fine-grained access control to GCP resources. Highly recommended.
  • --enable-private-nodes: Creates a private cluster where nodes have internal IP addresses only. Requires careful network configuration.
  • --master-ipv4-cidr-block: (For private clusters) Specifies the CIDR range for the control plane's internal IP. Must be unique within your VPC. Default is 172.16.0.0/28.
  • --enable-master-authorized-networks: Restricts access to the control plane endpoint to specified CIDR blocks. Crucial for security.
  • --master-authorized-networks: Comma-separated list of CIDR blocks.
  • --network and --subnetwork: Specify a custom VPC network and subnetwork. Mandatory for integrating with existing network infrastructure.
  • --cluster-secondary-range-name and --services-secondary-range-name: Define the secondary ranges within your specified subnetwork for Pods and Services IPs, respectively.

Connecting to Your Autopilot Cluster

Once the cluster is created, you need to get its credentials to interact with it using kubectl.

gcloud container clusters get-credentials hello-cluster \
    --region=europe-west1 \
    --project=your-gcp-project-id

This command configures kubectl to point to your new cluster by updating your ~/.kube/config file. You can verify the connection:

kubectl get nodes

Initially, you might see 0 nodes listed or a small number of system-managed nodes. This is normal for Autopilot. Nodes are provisioned dynamically as you deploy workloads.

NAME                                     STATUS   ROLES    AGE   VERSION
gke-hello-cluster-autopilot-1-f7614e5a   Ready    <none>   5m    v1.27.3-gke.100

The node names in Autopilot follow a pattern like gke-CLUSTER_NAME-autopilot-VERSION_HASH.

Deploying Workloads to Autopilot

When deploying pods to Autopilot, it's essential to define resource requests. Without them, pods may remain in a Pending state as Autopilot cannot provision appropriate nodes.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:latest
        resources:
          requests:
            cpu: "250m"
            memory: "512Mi"
          limits:
            cpu: "500m"
            memory: "1Gi"
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-service
spec:
  selector:
    app: nginx
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
  type: LoadBalancer

Apply this manifest:

kubectl apply -f nginx-deployment.yaml

Monitor the pods and nodes:

kubectl get pods -w
kubectl get nodes

You will observe Autopilot provisioning nodes as needed to accommodate the 3 Nginx pods. The node count might increase and decrease over time based on workload demand and configured pod autoscaling.

Scaling and Cost Management in Autopilot

Autopilot automatically scales nodes based on pod resource requests. You manage application scaling using standard Kubernetes Horizontal Pod Autoscalers (HPA) and Vertical Pod Autoscalers (VPA).

  • Horizontal Pod Autoscaling (HPA): Scales the number of pod replicas based on CPU/memory utilization or custom metrics. When HPA increases replicas, Autopilot automatically provisions more nodes if existing ones are insufficient.
  • Vertical Pod Autoscaling (VPA): Adjusts the CPU and memory requests/limits for individual pods based on historical usage. This helps optimize resource allocation and informs Autopilot's node provisioning decisions. Autopilot supports VPA in recommendation mode. Enabling VPA in auto mode (which sets requests/limits automatically) can be done but requires careful testing, as Autopilot relies on these requests to provision nodes.

Cost Implications: Autopilot bills for the actual CPU, memory, and ephemeral storage requested by your running pods. There is a small flat fee per cluster per hour, plus charges for resources. Nodes themselves are not directly billed; their underlying compute resources are implicitly covered by the pod resource charges. This makes cost management more granular and directly tied to application needs.

Pricing details are crucial and frequently updated. Always refer to the official GKE Autopilot pricing page for the latest information.

Common Pitfalls and Troubleshooting

  • Missing Resource Requests/Limits: This is the most common issue. Pods without explicit resources.requests will likely remain in a Pending state. Autopilot cannot determine node sizing without this information.
  • Insufficient Quotas: Even though Autopilot manages nodes, your GCP project still has quotas for CPU, memory, and IP addresses. If you hit these quotas, node provisioning will fail. Check gcloud compute regions describe europe-west1 for regional quotas and gcloud container operations list for specific error messages.
  • Pod Scheduling Failures: Look at pod events (kubectl describe pod <pod-name>). Common reasons include:
    • FailedScheduling due to insufficient CPU/memory requests.
    • Pending due to missing resources.requests.
    • Unschedulable if node constraints (e.g., taints/tolerations, node selectors) are used in an unsupported way or if Autopilot cannot find a suitable node. Autopilot aims for simplicity, so complex node targeting might be better suited for GKE Standard.
  • Unsupported Workloads: Autopilot has specific requirements. For instance, privileged containers or host path volumes are generally not supported for security and management reasons. Refer to the GKE Autopilot documentation for detailed workload constraints.
  • Networking Issues: If pods cannot communicate with external services or other pods, verify your VPC network configuration, firewall rules, and secondary IP ranges for pods and services. Private clusters require careful consideration of master authorized networks and VPC peering if accessing resources in other networks.

Back to the knowledge base · Ask the AI assistant