How I Cut AWS Compute Costs by 70% with a Multi-Arch EKS Cluster and Karpenter

After watching our monthly AWS bill climb steadily as our Kubernetes workloads grew, I knew we needed a smarter approach to node provisioning. Traditional cluster autoscaling was burning through our budget with over-provisioned nodes sitting idle, while still taking forever to scale up when I needed capacity. I’d heard of Karpenter, but I hadn’t tried it. That incident pushed me to finally give it a shot. In this guide, I'll share how I built a production-ready, multi-architecture EKS cluster that cut our compute costs by 70% while making scaling nearly instantaneous. The complete source code is available on GitHub. The complete source code is available on GitHub. GitHub Switched from Cluster Autoscaler to KarpenterCut compute costs by 70% using spot + GravitonReduced pod scheduling latency from 3 mins to 20 secondsBuilt multi-arch EKS cluster with AMD64 + ARM64Full Terraform + CI/CD pipeline on GitHub Switched from Cluster Autoscaler to Karpenter Cut compute costs by 70% using spot + Graviton Reduced pod scheduling latency from 3 mins to 20 seconds Built multi-arch EKS cluster with AMD64 + ARM64 Full Terraform + CI/CD pipeline on GitHub Ditching Traditional Autoscaling Here's what I did not like about the standard Kubernetes Cluster Autoscaler: I'd have pods sitting in "Pending" state for 2-3 minutes waiting for new nodes, while paying for a bunch of over-provisioned m5.large instances that were barely hitting 20% CPU utilization most of the time. I remember one particularly frustrating incident where a traffic spike hit our app at 2 AM. The cluster autoscaler took over 4 minutes to provision new nodes, and by then our users were getting timeouts. That's when I started seriously looking at Karpenter. What makes Karpenter different is that it doesn't think in terms of fixed node groups. Instead, it looks at your pending pods and says, "OK, you need 2 vCPUs and 4GB RAM? Let me find the cheapest spot instance that fits those requirements." It's like having a really smart provisioning assistant that actually understands your workload patterns. After implementing it, I saw our node provisioning times drop from 3+ minutes to under 20 seconds. And the cost savings — about a 70% reduction in our compute bill, which freed up budget for other infrastructure improvements. Our Multi-Architecture Setup One thing that impressed me about Karpenter was how easily it handled our mixed workload requirements. I had some legacy PHP applications that needed x86 instances, but I also wanted to experiment with ARM64 Graviton instances for our newer microservices. The architecture I ended up with looks like this: ┌─────────────────────────────────────────────────────────────┐ │ EKS Cluster │ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │ │ AMD64 │ │ ARM64 │ │ Karpenter │ │ │ │ NodePool │ │ NodePool │ │ Controller │ │ │ └─────────────┘ └─────────────┘ └─────────────┘ │ │ │ │ ┌─────────────────────────────────────────────────────┐ │ │ │ Spot Instance Handling │ │ │ │ SQS Queue + EventBridge Rules │ │ │ └─────────────────────────────────────────────────────┘ │ └─────────────────────────────────────────────────────────────┘ ┌─────────────────────────────────────────────────────────────┐ │ EKS Cluster │ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │ │ AMD64 │ │ ARM64 │ │ Karpenter │ │ │ │ NodePool │ │ NodePool │ │ Controller │ │ │ └─────────────┘ └─────────────┘ └─────────────┘ │ │ │ │ ┌─────────────────────────────────────────────────────┐ │ │ │ Spot Instance Handling │ │ │ │ SQS Queue + EventBridge Rules │ │ │ └─────────────────────────────────────────────────────┘ │ └─────────────────────────────────────────────────────────────┘ The architecture consists of several key components: The architecture consists of several key components: VPC Infrastructure: Getting the networking right was crucial. I spent some time getting the subnet tagging perfect — those karpenter.sh/discovery tags aren't just nice-to-have, they're essential for Karpenter to find your subnets. I forgot the karpenter.sh/discovery tag once — took an hour to debug why it wouldn’t launch nodes. VPC Infrastructure: Getting the networking right was crucial. I spent some time getting the subnet tagging perfect — those karpenter.sh/discovery tags aren't just nice-to-have, they're essential for Karpenter to find your subnets. I forgot the karpenter.sh/discovery tag once — took an hour to debug why it wouldn’t launch nodes. VPC Infrastructure karpenter.sh/discovery karpenter.sh/discovery EKS Cluster: The OIDC provider integration was probably the trickiest part to get right initially. It's what allows Karpenter to securely talk to AWS APIs without hardcoded credentials. EKS Cluster: The OIDC provider integration was probably the trickiest part to get right initially. It's what allows Karpenter to securely talk to AWS APIs without hardcoded credentials. EKS Cluster Karpenter Controller: This is where the magic happens. Once deployed, it constantly watches for unscheduled pods and intelligently provisions the right instances. Karpenter Controller: This is where the magic happens. Once deployed, it constantly watches for unscheduled pods and intelligently provisions the right instances. Karpenter Controller Multi-Architecture Support: Having separate node pools for AMD64 and ARM64 lets us run different workloads on the most cost-effective hardware. Multi-Architecture Support: Having separate node pools for AMD64 and ARM64 lets us run different workloads on the most cost-effective hardware. Multi-Architecture Support Spot Instance Management: The interruption handling was something I was initially worried about, but AWS's SQS integration makes it surprisingly robust. Spot Instance Management: The interruption handling was something I was initially worried about, but AWS's SQS integration makes it surprisingly robust. Spot Instance Management The Terraform Structure That Actually Works My first attempt at structuring this was a mess. I had everything in one giant main.tf file, and making changes was terrifying. After some refactoring (and a few late nights), I landed on this modular approach: main.tf ├── eks-module/ # EKS cluster creation ├── karpenter/ # Karpenter autoscaler setup ├── vpc/ # VPC infrastructure ├── root/ # Main Terraform execution └── .github/workflows/ # CI/CD pipeline ├── eks-module/ # EKS cluster creation ├── karpenter/ # Karpenter autoscaler setup ├── vpc/ # VPC infrastructure ├── root/ # Main Terraform execution └── .github/workflows/ # CI/CD pipeline VPC: Getting the Networking Foundation Right The VPC setup took me a few iterations to get right. The key insight was that the tagging strategy isn't just documentation — it's functional. Here's what I learned works: Here's what I learned works: resource "aws_vpc" "main" { cidr_block = var.vpc_cidr_block enable_dns_support = true enable_dns_hostnames = true tags = { Name = "${var.project_name}-vpc" "kubernetes.io/cluster/${var.project_name}" = "shared" } } resource "aws_subnet" "private_subnets" { # ... configuration tags = { Name = "private-subnet-${count.index}" "karpenter.sh/discovery" = var.project_name "kubernetes.io/role/internal-elb" = "1" } } resource "aws_vpc" "main" { cidr_block = var.vpc_cidr_block enable_dns_support = true enable_dns_hostnames = true tags = { Name = "${var.project_name}-vpc" "kubernetes.io/cluster/${var.project_name}" = "shared" } } resource "aws_subnet" "private_subnets" { # ... configuration tags = { Name = "private-subnet-${count.index}" "karpenter.sh/discovery" = var.project_name "kubernetes.io/role/internal-elb" = "1" } } EKS and Karpenter: The Heart of the System The EKS setup was straightforward, but getting the OIDC provider right was critical. This is what allows Karpenter to assume IAM roles securely: resource "aws_eks_cluster" "project" { name = var.cluster_name role_arn = aws_iam_role.cluster_role.arn vpc_config { subnet_ids = var.private_subnet_ids endpoint_private_access = true endpoint_public_access = true } access_config { authentication_mode = "API" } } resource "aws_iam_openid_connect_provider" "eks" { client_id_list = ["sts.amazonaws.com"] thumbprint_list = [data.tls_certificate.eks.certificates[0].sha1_fingerprint] url = aws_eks_cluster.project.identity[0].oidc[0].issuer } resource "aws_eks_cluster" "project" { name = var.cluster_name role_arn = aws_iam_role.cluster_role.arn vpc_config { subnet_ids = var.private_subnet_ids endpoint_private_access = true endpoint_public_access = true } access_config { authentication_mode = "API" } } resource "aws_iam_openid_connect_provider" "eks" { client_id_list = ["sts.amazonaws.com"] thumbprint_list = [data.tls_certificate.eks.certificates[0].sha1_fingerprint] url = aws_eks_cluster.project.identity[0].oidc[0].issuer } For the Karpenter deployment, I went with the Helm chart approach. One thing I learned is that you need to install the CRDs first, then the main chart. The interruption queue setup was crucial — it's what prevents your workloads from getting abruptly terminated when spot instances are reclaimed: # Install CRDs first resource "helm_release" "karpenter_crd" { name = "karpenter-crd" chart = "oci://public.ecr.aws/karpenter/karpenter-crd" version = var.karpenter_version namespace = var.namespace } # Then install the main Karpenter chart resource "helm_release" "karpenter" { depends_on = [helm_release.karpenter_crd] name = "karpenter" chart = "oci://public.ecr.aws/karpenter/karpenter" version = var.karpenter_version namespace = var.namespace set { name = "settings.clusterName" value = var.cluster_name } set { name = "settings.aws.interruptionQueueName" value = aws_sqs_queue.karpenter_interruption_queue.name } set { name = "serviceAccount.annotations.eks\\.amazonaws\\.com/role-arn" value = var.controller_role_arn } } # Install CRDs first resource "helm_release" "karpenter_crd" { name = "karpenter-crd" chart = "oci://public.ecr.aws/karpenter/karpenter-crd" version = var.karpenter_version namespace = var.namespace } # Then install the main Karpenter chart resource "helm_release" "karpenter" { depends_on = [helm_release.karpenter_crd] name = "karpenter" chart = "oci://public.ecr.aws/karpenter/karpenter" version = var.karpenter_version namespace = var.namespace set { name = "settings.clusterName" value = var.cluster_name } set { name = "settings.aws.interruptionQueueName" value = aws_sqs_queue.karpenter_interruption_queue.name } set { name = "serviceAccount.annotations.eks\\.amazonaws\\.com/role-arn" value = var.controller_role_arn } } Multi-Architecture NodePools Initially, I was skeptical about ARM64. "Will our applications even work?" But after some testing, I found that our Node.js and Python services ran perfectly on Graviton instances — often with better performance per dollar. Here's how I set up the node pools: AMD64 NodePool apiVersion: karpenter.sh/v1 kind: NodePool metadata: name: amd64-nodepool spec: template: spec: requirements: - key: kubernetes.io/arch operator: In values: ["amd64"] - key: karpenter.sh/capacity-type operator: In values: ["spot", "on-demand"] - key: node.kubernetes.io/instance-type operator: In values: ["m5.large", "m5.xlarge", "c5.large", "c5.xlarge"] limits: cpu: 1000 disruption: consolidationPolicy: WhenEmpty consolidateAfter: 1m apiVersion: karpenter.sh/v1 kind: NodePool metadata: name: amd64-nodepool spec: template: spec: requirements: - key: kubernetes.io/arch operator: In values: ["amd64"] - key: karpenter.sh/capacity-type operator: In values: ["spot", "on-demand"] - key: node.kubernetes.io/instance-type operator: In values: ["m5.large", "m5.xlarge", "c5.large", "c5.xlarge"] limits: cpu: 1000 disruption: consolidationPolicy: WhenEmpty consolidateAfter: 1m ARM64 NodePool: The Graviton Experiment The ARM64 setup requires a bit more thought. I added taints to prevent incompatible workloads from accidentally landing on ARM64 nodes: apiVersion: karpenter.sh/v1 kind: NodePool metadata: name: arm64-nodepool spec: template: spec: requirements: - key: kubernetes.io/arch operator: In values: ["arm64"] - key: node.kubernetes.io/instance-type operator: In values: ["t4g.medium", "c6g.large", "m6g.large"] taints: - key: arm64 value: "true" effect: NoSchedule apiVersion: karpenter.sh/v1 kind: NodePool metadata: name: arm64-nodepool spec: template: spec: requirements: - key: kubernetes.io/arch operator: In values: ["arm64"] - key: node.kubernetes.io/instance-type operator: In values: ["t4g.medium", "c6g.large", "m6g.large"] taints: - key: arm64 value: "true" effect: NoSchedule I learned that specifying exact AMI IDs gives you more control over what gets deployed, especially when testing different ARM64 configurations. You can also use nodeSelector requirements to target specific architectures without needing taints if your applications are architecture-aware. Cost Savings with this Approach Let me share the real impact from our production environment. Before Karpenter, we were running a mix of on-demand instances across multiple node groups — about 15-20 m5.large and c5.xlarge instances running 24/7. After switching to Karpenter with about 70% spot instance usage, our monthly compute costs dropped by 70%. That's a significant reduction that freed up substantial budget for new features and infrastructure improvements. The ARM64 Benefits The ARM64 instances provided an additional bonus. Beyond the ~20% cost savings compared to equivalent x86 instances, I saw better performance on some of our CPU-intensive workloads. Our image processing service, for example, ran about 15% faster on Graviton instances. Here's what I typically see across our clusters: 70% reduction in compute costsSub-20-second scaling instead of minutesBetter resource utilization — no more paying for idle CPU cycles 70% reduction in compute costs 70% reduction Sub-20-second scaling instead of minutes Sub-20-second scaling Better resource utilization — no more paying for idle CPU cycles Better resource utilization The performance monitoring showed we went from average 25% CPU utilization on our fixed nodes up to 70% utilization with Karpenter's right-sizing. Production Lessons Learned Security: IRSA Is Your Friend Getting the security model right was crucial. I use IAM Roles for Service Accounts (IRSA) everywhere, which means no more hardcoded AWS credentials floating around in our cluster: resource "aws_iam_role" "karpenter_controller" { name = "KarpenterController-${var.cluster_name}" assume_role_policy = jsonencode({ Version = "2012-10-17" Statement = [ { Action = "sts:AssumeRoleWithWebIdentity" Effect = "Allow" Principal = { Federated = var.oidc_provider_arn } Condition = { StringEquals = { "${replace(var.oidc_provider_url, "https://", "")}:sub" = "system:serviceaccount:karpenter:karpenter" } } } ] }) } resource "aws_iam_role" "karpenter_controller" { name = "KarpenterController-${var.cluster_name}" assume_role_policy = jsonencode({ Version = "2012-10-17" Statement = [ { Action = "sts:AssumeRoleWithWebIdentity" Effect = "Allow" Principal = { Federated = var.oidc_provider_arn } Condition = { StringEquals = { "${replace(var.oidc_provider_url, "https://", "")}:sub" = "system:serviceaccount:karpenter:karpenter" } } } ] }) } Monitoring After some trial and error, I settled on monitoring these key metrics. Too many dashboards become noise, but these four tell me everything I need to know about our cluster performance: karpenter_nodes_created_total: Are nodes scaling up when they should be?karpenter_nodes_terminated_total: Are nodes scaling down appropriately?karpenter_pods_state: Are pods getting scheduled quickly?karpenter_node_utilization: Are instances being right-sized? karpenter_nodes_created_total: Are nodes scaling up when they should be? karpenter_nodes_created_total karpenter_nodes_terminated_total: Are nodes scaling down appropriately? karpenter_nodes_terminated_total karpenter_pods_state: Are pods getting scheduled quickly? karpenter_pods_state karpenter_node_utilization: Are instances being right-sized? karpenter_node_utilization The node utilization metric was particularly eye-opening. We went from ~25% average utilization up to 70% after Karpenter started right-sizing our instances. CI/CD My deployment pipeline is pretty straightforward. I use GitHub Actions with OIDC authentication to AWS (no more access keys in secrets!). The workflow supports feature branches for development, staging for testing, and main for production: name: Terraform Deploy on: push: branches: - 'feature/**' - 'staging' - 'main' workflow_dispatch: permissions: id-token: write # Required for OIDC JWT contents: read # Required for checkout jobs: deploy: environment: ${{ (github.ref == 'refs/heads/main' && 'prod') || (github.ref == 'refs/heads/staging' && 'staging') || 'dev' }} runs-on: ubuntu-22.04 defaults: run: working-directory: root/ steps: - name: Clone repo uses: actions/checkout@v3 - name: Configure AWS Credentials uses: aws-actions/configure-aws-credentials@v4 with: aws-region: us-east-1 role-to-assume: ${{ vars.IAM_Role }} role-session-name: gha-assignment-session - name: Initialize Terraform run: terraform init -backend-config="bucket=${{ vars.TF_STATE_BUCKET }}" - name: Terraform Plan run: terraform plan -input=false -var-file=${{ vars.STAGE }}.tfvars - name: Terraform Apply run: terraform apply -auto-approve -input=false -var-file=${{ vars.STAGE }}.tfvars name: Terraform Deploy on: push: branches: - 'feature/**' - 'staging' - 'main' workflow_dispatch: permissions: id-token: write # Required for OIDC JWT contents: read # Required for checkout jobs: deploy: environment: ${{ (github.ref == 'refs/heads/main' && 'prod') || (github.ref == 'refs/heads/staging' && 'staging') || 'dev' }} runs-on: ubuntu-22.04 defaults: run: working-directory: root/ steps: - name: Clone repo uses: actions/checkout@v3 - name: Configure AWS Credentials uses: aws-actions/configure-aws-credentials@v4 with: aws-region: us-east-1 role-to-assume: ${{ vars.IAM_Role }} role-session-name: gha-assignment-session - name: Initialize Terraform run: terraform init -backend-config="bucket=${{ vars.TF_STATE_BUCKET }}" - name: Terraform Plan run: terraform plan -input=false -var-file=${{ vars.STAGE }}.tfvars - name: Terraform Apply run: terraform apply -auto-approve -input=false -var-file=${{ vars.STAGE }}.tfvars What I like about this setup is the dynamic environment selection and the use of GitHub variables for different stages. The terraform fmt -check step ensures code consistency across our team. terraform fmt -check Summary Look, I'm not going to pretend this was all smooth sailing. I had some initial hiccups with spot instance interruptions (protip: make sure your applications handle SIGTERM gracefully), and getting the ARM64 workloads right took some iteration. But the results speak for themselves. We're running a more resilient, cost-effective infrastructure that scales intelligently. The substantial cost savings alone paid for the engineering time I spent on this migration within the first month. If you're dealing with unpredictable workloads, rising AWS bills, or just want to try something that feels like "the future of Kubernetes," I'd definitely recommend giving Karpenter a shot. Start with a development cluster, get comfortable with the concepts, and then gradually migrate your production workloads. The complete implementation is available on GitHub. I've included all the Terraform modules, testing scripts, and a troubleshooting guide based on the issues I ran into. Feel free to use it as a starting point for your own implementation — and let me know how it goes! GitHub