Amira Abidi

Posted on Sep 15

Kubernetes Configuration Management with Kustomize: Dev vs Prod

#aws #cicd #devops #kubernetes

Managing multiple Kubernetes environments looks simple at first.

You create a few YAML manifests, deploy them to development, then adapt them for production.

Then reality happens.

Development and production need different replica counts, resource limits, image tags, hostnames, configuration values, and sometimes completely different infrastructure settings.

Copying the manifests for each environment may work for a small project, but it quickly becomes difficult to maintain.

This is where Kustomize becomes useful.

In my LibraryCorner project, I used Kustomize to manage Kubernetes configurations across separate development and production environments while keeping a shared configuration base.

1. The Problem: Kubernetes Manifests Across Environments

Imagine that we have a simple application deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: librarycorner-api
spec:
  replicas: 2
  template:
    spec:
      containers:
        - name: api
          image: librarycorner-api:latest
          ports:
            - containerPort: 8000

This might be perfectly fine for development.

But production may require:

more replicas
different resource requests and limits
a production image tag
different configuration
different ingress/hostname settings
stricter security settings

One tempting approach is to create:

deployment-dev.yaml
deployment-prod.yaml

and maintain both independently.

The problem is duplication.

If the base Deployment changes, we now have to remember to update multiple copies.

That's where Kustomize provides a cleaner approach.

2. What Is Kustomize?

Kustomize is a Kubernetes-native configuration management tool.

Instead of using templates, Kustomize lets us define a common base configuration and then apply environment-specific customizations through overlays.

The general structure looks like this:

k8s/
├── base/
│   ├── deployment.yaml
│   ├── service.yaml
│   └── kustomization.yaml
│
└── overlays/
    ├── dev/
    │   └── kustomization.yaml
    │
    └── prod/
        └── kustomization.yaml

The important concept is:

                 Base
                  │
          ┌───────┴───────┐
          │               │
         Dev             Prod
       Overlay           Overlay

The base contains what is common.

The overlays contain what changes.

3. Building the Base

Let's start with the common configuration.

`base/deployment.yaml`

apiVersion: apps/v1
kind: Deployment
metadata:
  name: librarycorner-api
spec:
  replicas: 2
  selector:
    matchLabels:
      app: librarycorner-api
  template:
    metadata:
      labels:
        app: librarycorner-api
    spec:
      containers:
        - name: api
          image: librarycorner-api
          ports:
            - containerPort: 8000

`base/service.yaml`

apiVersion: v1
kind: Service
metadata:
  name: librarycorner-api
spec:
  selector:
    app: librarycorner-api
  ports:
    - port: 80
      targetPort: 8000

Then we define the resources in base/kustomization.yaml:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
  - deployment.yaml
  - service.yaml

At this point, the base describes the application without knowing whether it will run in development or production.

4. Creating the Development Overlay

Now let's create:

overlays/dev/kustomization.yaml

We reference the base:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
  - ../../base

We can then apply development-specific changes.

For example, we might want only one replica:

replicas:
  - name: librarycorner-api
    count: 1

We can also modify the container image:

images:
  - name: librarycorner-api
    newName: librarycorner-api
    newTag: dev

The resulting development configuration is generated from the base plus these customizations.

5. Creating the Production Overlay

The production overlay follows the same principle.

overlays/prod/kustomization.yaml

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
  - ../../base

replicas:
  - name: librarycorner-api
    count: 3

images:
  - name: librarycorner-api
    newName: librarycorner-api
    newTag: prod

Now we have:

Base
 │
 ├── Dev → 1 replica + dev image
 │
 └── Prod → 3 replicas + prod image

The common Kubernetes configuration remains in one place.

6. Patches for More Specific Changes

Sometimes changing replicas or images isn't enough.

For example, production may require different resource limits.

We can use a patch.

`overlays/prod/resources-patch.yaml`

apiVersion: apps/v1
kind: Deployment
metadata:
  name: librarycorner-api
spec:
  template:
    spec:
      containers:
        - name: api
          resources:
            requests:
              cpu: "250m"
              memory: "256Mi"
            limits:
              cpu: "500m"
              memory: "512Mi"

Then reference the patch from the production overlay:

patches:
  - path: resources-patch.yaml

This keeps the production-specific configuration isolated from the shared base.

7. Environment-Specific Configuration

Another common requirement is changing configuration values between environments.

For example:

ENVIRONMENT=dev

versus:

ENVIRONMENT=prod

Kustomize can generate ConfigMaps from literals or files.

For example:

configMapGenerator:
  - name: librarycorner-config
    literals:
      - ENVIRONMENT=dev

The production overlay can define:

configMapGenerator:
  - name: librarycorner-config
    literals:
      - ENVIRONMENT=prod

This allows the same application manifest to be reused while keeping environment-specific configuration separate.

For sensitive information, however, I would not put credentials directly into Git-managed manifests. Secrets should be handled through an appropriate secret-management solution.

8. Previewing the Result

One of my favorite aspects of Kustomize is that we can render the final Kubernetes manifests before applying them.

For development:

kubectl kustomize overlays/dev

For production:

kubectl kustomize overlays/prod

This makes it possible to inspect exactly what Kubernetes will receive.

We can also apply the configuration directly:

kubectl apply -k overlays/dev

or:

kubectl apply -k overlays/prod

The -k option tells kubectl to use Kustomize.

9. Kustomize in a CI/CD Pipeline

Kustomize becomes particularly useful when combined with CI/CD.

For example, a simplified deployment pipeline can look like:

Git push
   │
   ▼
Build application image
   │
   ▼
Run tests
   │
   ▼
Push image
   │
   ▼
Update Kubernetes configuration
   │
   ▼
Kustomize overlay
   │
   ▼
Deploy to EKS

For example, a pipeline could deploy development using:

kubectl apply -k k8s/overlays/dev

while production uses:

kubectl apply -k k8s/overlays/prod

This makes the Kubernetes configuration part of the application delivery process instead of maintaining deployment settings manually.

10. How I Used It in LibraryCorner

In LibraryCorner, I used separate Kubernetes configurations for development and production.

The goal was not to create two independent Kubernetes applications.

Instead, I wanted:

One common application configuration

plus

Environment-specific differences.

The structure was conceptually:

k8s/
├── base/
│   ├── deployment
│   ├── service
│   ├── config
│   └── kustomization.yaml
│
└── overlays/
    ├── dev/
    │   └── kustomization.yaml
    │
    └── prod/
        ├── kustomization.yaml
        └── patches/

This approach helped keep the environments consistent while still allowing production to have different scaling and resource requirements.

It also fits naturally with the infrastructure architecture of the project:

AWS
│
├── Development
│   └── EKS
│       └── Kustomize → Dev configuration
│
└── Production
    └── EKS
        └── Kustomize → Prod configuration

11. Kustomize vs Helm

Kustomize and Helm are often compared because both can be used to manage Kubernetes configurations.

The difference is largely in their approach.

Kustomize

works directly with Kubernetes YAML
uses bases and overlays
doesn't require a templating language
is integrated into kubectl

Helm

uses charts and templates
provides packaging and versioning
has a larger ecosystem for distributing applications
is particularly useful when deploying reusable third-party applications

For my LibraryCorner project, Kustomize was a good fit because I wanted to maintain a relatively small number of environment-specific configurations without introducing a templating layer.

12. What I Learned

The biggest lesson for me was that environment separation is not only an infrastructure problem.

It also exists at the Kubernetes configuration level.

A clean structure such as:

Base + Dev Overlay + Prod Overlay

helps avoid configuration drift and duplicated manifests.

It also makes the deployment process easier to reason about:

Same application
      +
Different environment configuration
      =
Predictable deployments

Kustomize is therefore a relatively small tool, but it solves a very practical problem when Kubernetes configurations start becoming more complex.

Conclusion

Kubernetes manifests are easy to write.

Keeping them maintainable across multiple environments is the harder part.

Kustomize provides a simple Kubernetes-native approach:

Base → Overlays → Environment-specific configuration

For projects running across development and production environments, this can significantly reduce duplication while keeping the differences explicit.

In my case, it became an important part of the LibraryCorner deployment architecture on AWS EKS.

And the next question naturally follows:

How do we make sure these Kubernetes workloads are actually healthy and observable once they're running?

That's where monitoring and observability come in — and that's a topic I've already explored with Prometheus and Grafana in LibraryCorner.

DEV Community