Kustomize is used for Kubernetes native configuration management. It introduces a template-free way to customize application configuration that simplifies the use of off-the-shelf applications. Kustomize traverses a Kubernetes manifest to add, remove or update configuration options without forking. It is available both as a standalone binary and as a native feature of
Kustomize simplifies deployments by allowing you to create an entire Kubernetes application out of individual pieces — without touching the YAML configuration files for the individual components.
Kustimize leverages layering to preserve the base settings of your applications and components by overlaying declarative yaml artifacts (called patches) that selectively override default settings without actually changing the original files.
Kustomize relies on the following system of configuration management layering to achieve reusability:
- Base Layer – Specifies the most common resources
- Patch Layers – Specifies use case specific resources
In this tutorial, we’ll set up kustomize and explore how it works with a sample application deployment.
- Getting started with Kubernetes – Kubernetes Components
- How to set up Kubernetes Cluster on Ubuntu 20.04 with kubeadm and CRI-O
Features of Kustomize:
- Purely declarative approach to configuration customization
- Natively built into
kubectlfrom version 1.14
- Manage an arbitrary number of distinctly customized Kubernetes configurations
- Available as a standalone binary for extension and integration into other services
- Every artifact that kustomize uses is plain YAML and can be validated and processed as such
- Kustomize encourages a fork/modify/rebase workflow
Kustomize can be installed by downloading precompiled binaries.
Binaries at various versions for linux, MacOs and Windows are published on the releases page.
The following script detects your OS and downloads the appropriate kustomize binary to your current working directory.
curl -s "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh" | bash
This script doesn’t work for ARM architecture
Let’s step through how Kustomize works using a deployment scenario involving 2 different environments:
prod. In this example we’ll use
horizontal pod autoscaler resources. For the
uat environment, there won’t be any HPA involved. All of the environments will use different types of services:
They each will have different HPA settings. This is how directory structure looks:
➜ tree . ├── base │ ├── deployment.yaml │ ├── hpa.yaml │ ├── kustomization.yaml │ └── service.yaml └── overlays ├── prod │ ├── hpa.yaml │ ├── kustomization.yaml │ ├── rollout-replicas.yaml │ └── service-loadbalancer.yaml └── uat ├── hpa.yaml └── kustomization.yaml 4 directories, 10 files
The base folder holds the common resources, such as the standard
hpa.yaml resource configuration files. We’ll explore each of their contents in the following sections.
apiVersion: apps/v1 kind: Deployment metadata: name: citiapp spec: selector: matchLabels: app: citiapp template: metadata: labels: app: citiapp spec: containers: - name: app image: foo/citiapp:latest ports: - name: http containerPort: 8080 protocol: TCP
apiVersion: v1 kind: Service metadata: name: citiapp-service spec: ports: - name: http port: 8080 selector: app: citiapp
apiVersion: autoscaling/v2beta2 kind: HorizontalPodAutoscaler metadata: name: citiapp-hpa spec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: citiapp minReplicas: 1 maxReplicas: 5 metrics: - type: Resource resource: name: cpu target: type: Utilization averageUtilization: 50
The kustmization.yaml file is the most important file in the base folder and it describes what resources you use.
apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization resources: - service.yaml - deployment.yaml - hpa.yaml
Uat Overlay Files
The overlays folder houses environment-specific overlays. It has 2 sub-folders (one for each environment).
This file defines which base configuration to reference and patch using patchesStrategicMerge, which allows partial YAML files to be defined and overlaid on top of the base.
apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization bases: - ../../base patchesStrategicMerge: - hpa.yaml
This file has the same resource name as the one located in the base file. This helps in matching the file for patching. This file also contains important values, such as min/max replicas, for the dev environment.
apiVersion: autoscaling/v2beta2 kind: HorizontalPodAutoscaler metadata: name: citiapp-hpa spec: minReplicas: 1 maxReplicas: 2 metrics: - type: Resource resource: name: cpu target: type: Utilization averageUtilization: 90
If you compare the previous
hpa.yaml file with
base/hpa.yaml, you’ll notice differences in
To confirm that your patch config file changes are correct before applying to the cluster, you can run
kustomize build overlays/uat:
apiVersion: v1 kind: Service metadata: name: citiapp-service spec: ports: - name: http port: 8080 selector: app: citiapp --- apiVersion: apps/v1 kind: Deployment metadata: name: citiapp spec: selector: matchLabels: app: citiapp spec: containers: - image: foo/citiapp:latest name: app ports: - containerPort: 8080 name: http protocol: TCP template: metadata: labels: app: citiapp --- apiVersion: autoscaling/v2beta2 kind: HorizontalPodAutoscaler metadata: name: citiapp-hpa spec: apiVersion: apps/v1 kind: Deployment maxReplicas: 2 metrics: - type: Resource minReplicas: 1 name: citiapp resource: name: cpu target: averageUtilization: 90 type: Utilization
Once you have confirmed that your overlays are correct, use the
kubectl apply -k overlays/uat command to apply the the settings to your cluster:
$ kubectl apply -k overlays/dev service/citiapp-service created deployment.apps/citiapp created horizontalpodautoscaler.autoscaling/citiapp-hpa created
After handling the
uat environment, let us also demo the production environment.
Define Prod Overlay Files
In our production
hpa.yaml, let’s say we want to allow up to 10 replicas, with new replicas triggered by a resource utilization threshold of 70% avg CPU usage. This is how that would look:
apiVersion: autoscaling/v2beta2 kind: HorizontalPodAutoscaler metadata: name: citiapp-hpa spec: minReplicas: 1 maxReplicas: 10 metrics: - type: Resource resource: name: cpu target: type: Utilization averageUtilization: 70
There’s also a
rollout-replicas.yaml file in our production directory which specifies our rolling strategy:
apiVersion: apps/v1 kind: Deployment metadata: name: citiapp spec: replicas: 10 strategy: rollingUpdate: maxSurge: 1 maxUnavailable: 1 type: RollingUpdate
We use this file to change the service type to LoadBalancer.
apiVersion: v1 kind: Service metadata: name: citiapp-service spec: type: LoadBalancer
This file operates the same way in the production folder as it does in your base folder: it defines which base file to reference and which patches to apply for your production environment. In this case, it includes two more files:
apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization bases: - ../../base patchesStrategicMerge: - rollout-replica.yaml - hpa.yaml - service-loadbalancer.yaml
Review Prod Patches
Lets see if production values are being applied by running
kustomize build overlays/prod
Once you have reviewed, apply your overlays to the cluster with:
$ kubectl apply -k overlays/prod service/citiapp-service created deployment.apps/citiapp created horizontalpodautoscaler.autoscaling/frontend-deployment-hpa created
Bases and overlays
Kustomize’s configuration transformation approach leverages the use of kustomization layers so that the same base configuration files can be reused across multiple kustomization configurations. It achieves this with the concepts of bases and overlays.
- A base is a directory containing a file called
kustomization.yaml, which can enumerate some set of resources with some customizations that will be applied to them. A base should be declared in the
resourcesfield of a kustomization file.
- An overlay is a directory that refers to another kustomization directory as its, or one of its, bases.
A base can be thought of as a preliminary step in a pipeline, having no knowledge of the overlays that it is referenced by. After a base is finished processing, it sends its resources as input to the overlay to transform according to the overlay’s specification.
The following is an example of a kustomization base:
- image: nginx
This base could be reused by multiple kustomization overlays. The following is an example of an overlay that could refer to this base:
Running the command
kustomize build overlay produces the following output:
- image: nginx
The Deployment received the name prefix
bar-from the base kustomization, and then another name prefix
foo- from the overlay kustomization. The ConfigMap only received the name prefix
foo- because it was declared in the overlay, and thus was processed only by the overlay.
Generate Secrets and ConfigMaps
You can generate Secrets and ConfigMaps from a file by using the
configMapGenerator fields in your kustomization file. For example:
- name: my-app
generates the following YAML output:
Edit the kustomization file
Kustomize provides several imperative commands that help you manage your kustomization file.
- To add all the YAML files in your current directory to the kustomization’s
resourcesfield, run the following command:
kustomize edit add resource *.yaml
- To view the kustomize edit help page and see all the subcommands it offers, run the following command:
kustomize edit -h
- To get specific help for subcommands, add the subcommand as an argument. For example:
kustomize edit add -h