Secure Secrets in EKS: Using AWS Secrets Manager with the CSI Driver

Introduction

Containerized workloads often need to consume sensitive data such as database credentials, API keys, or tokens. Storing these values directly in container images or Kubernetes manifests exposes them to risk. AWS Secrets Manager together with the Secrets Store CSI driver provides a way to fetch secrets at runtime and mount them into pods as read‑only files, keeping the secret data out of code and manifests.

Core Components

AWS Secrets Manager – a managed service for storing, rotating, and retrieving secrets.

Secrets Store CSI driver – a Kubernetes Container Storage Interface driver that can mount external secret stores as volumes.

AWS Secrets and Configuration Provider (ASCP) – the provider plugin that enables the CSI driver to communicate with AWS Secrets Manager.

IAM Roles for Service Accounts (IRSA) – grants a pod a specific IAM role, limiting its permissions to only the secrets it needs.

Architecture Overview

An application pod requests a secret by mounting a special volume. The Secrets Store CSI driver, using the ASCP plugin, calls AWS Secrets Manager on behalf of the pod. Authentication is performed via the IAM role associated with the pod’s service account (IRSA). The retrieved secret is written as a file inside the pod’s filesystem (e.g., /mnt/secrets), where the application can read it.

Step‑by‑Step Implementation

1. Install the Secrets Store CSI driver

helm repo add secrets-store-csi-driver https://kubernetes-sigs.github.io/secrets-store-csi-driver/charts
helm install csi-secrets-store secrets-store-csi-driver/secrets-store-csi-driver \
  --namespace kube-system

2. Deploy the AWS provider (ASCP)

kubectl apply -f https://raw.githubusercontent.com/aws/secrets-store-csi-driver-provider-aws/main/deployment/aws-provider-installer.yaml

3. Create an IAM policy and role for IRSA

Define a policy that allows the pod to read a specific secret:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "secretsmanager:GetSecretValue",
      "Resource": "arn:aws:secretsmanager:REGION:ACCOUNT_ID:secret:SECRET_NAME-???????"
    }
  ]
}

Attach the policy to an IAM role and associate the role with a Kubernetes service account using the IRSA annotation:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: my-app-sa
  namespace: default
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::ACCOUNT_ID:role/SecretsManagerRole

4. Define a SecretProviderClass

The custom resource tells the CSI driver which provider to use and which secret objects to fetch:

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: my-app-secrets
spec:
  provider: aws
  parameters:
    objects: |
      - objectName: "my-database-credentials"
        objectType: "secretsmanager"

5. Mount the secret volume in a pod (or deployment)

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      serviceAccountName: my-app-sa  # IRSA‑linked service account
      containers:
      - name: my-app-container
        image: my-app-image
        volumeMounts:
        - name: secrets-volume
          mountPath: "/mnt/secrets"
          readOnly: true
      volumes:
      - name: secrets-volume
        csi:
          driver: secrets-store.csi.k8s.io
          readOnly: true
          volumeAttributes:
            secretProviderClass: "my-app-secrets"
            # Optional: sync the secret to a native Kubernetes Secret
            # secretProviderClass: "my-app-secrets"
            # syncSecret: "true"

When the pod starts, the secret values appear as files under /mnt/secrets. Applications can read the files directly, or, if syncSecret: true is enabled, the driver will also create a standard Kubernetes Secret with the same data.

Conclusion

By combining AWS Secrets Manager, the Secrets Store CSI driver, and IRSA, you can deliver secrets to Amazon EKS workloads securely and with fine‑grained access control. Secrets never reside in container images or manifest files, and the workflow scales across clusters while adhering to the principle of least privilege.