021: Features Kubernetes resource manifests with Kustomize Skip to main content

021: Features Kubernetes resource manifests with Kustomize

Kustomize is a native, declarative, template-free and comprehensive approach to composition, customisation and patch of any Kubernetes resource.

The daily mood

We had another meeting with developers on Friday afternoon about the migration project I had been involved in a week ago. We talked about the use of a Kubernetes DEV cluster offering a Hybrid deployment mode (as discussed in a previous post). I learnt about significant limitations due to natural differences in the architecture and components between DEV and PROD, which confused me a bit. At the end unfortunately, I will not be able to support this initiative actively, and rather let the experts proceed.
Regarding my work, I realised that it was not convenient to share and read larger pieces of code within my posts so I created a GitHub account dedicated to this blog, and will try to catch up with all previous assets soon. Since I was already using a personal GitHub account and want this blog to stay anonymous for now, I found it tricky to configure my project working copy so that the author is setup correctly (this article definitely helped).

I also realized that I had skipped some fundamental tools for application configuration and packaging within Kubernetes. I need to have a look at them since they might help understand, categorize and evaluate what we already know or use. I am maintaining a list of tools, today I picked up Kustomize.
Waht is Kustomize

As already explained in our previous post about Helm, Kustomize is template-free and therefore purely declarative. It started as a dedicated tool (kustomize) and was then additionally embedded as a native plugin of kubectl. At its core, a resource dictionary file called kustomization.yaml. Depending on how you structure your project, resource definitions may inherit from (bases) and overwrite (patches) each over.

So again I am working on the basic nginx application from my previous post on Kubernetes. Let's put our deployment.yaml in a folder named full and create our first kustomization.yaml next to it. 
resources:
- deployment.yaml
Now we can just print current configuration using one of the following commands, depending on wether you use the embedded or dedicated tool
kubectl kustomize                    # native plugin
kustomize build                      # dedicated command
For now the result is exactly the same as
cat deployment.yaml
In case you prefer a base folder where you have all your resources defined as separate files.
resources:
- nginx-deployment.yaml
- volume-configmap.yaml
- container-configmap.yaml
- nginx-service.yaml
You will find that kustomize is concatenating/managing all your resources into a single manifest. Now you may want to deploy this configuration through one or the other way
kubectl apply -k .                   # native plugin
kustomize build | kubectl apply -f - # dedicated command
Delete obviously works as usual
kubectl delete -k .                   # native plugin
kustomize build | kubectl delete -f - # dedicated command
Staging configuration
# BASE
mkdir base && mv full/*.yaml base/
kubectl kustomize base | grep -E 'name|replicas' # default resource names and 2 replicas
# INT
mkdir int
head -6 base/nginx-deployment.yaml > int/replica_count.yaml
cat > int/kustomization.yaml <<- "EOF"
namePrefix: int-
bases:
- ../base
patches:
- replica_count.yaml
EOF
sed -i 's/replicas: .*/replicas: 3/' replica_count.yaml
kubectl kustomize int | grep -E 'name|replicas'  # prefixed resource names and 3 replicas
We end up with the following folder structure
$ tree
.
├── base
│   ├── container-configmap.yaml
│   ├── kustomization.yaml
│   ├── nginx-deployment.yaml
│   ├── nginx-service.yaml
│   └── volume-configmap.yaml
├── full
│   ├── deployment.yaml
│   └── kustomization.yaml        # identical to base
└── int
    ├── kustomization.yaml
    └── replica_count.yaml        # deployment patch
Conclusion
  • Nice features
    • Resource name prefix (see above)
    • Common labels and annotations
    • ConfigMap and secrets generator
    • Configuration diff and substitute
    • Multi-repo support via relative (used for bespoke workflow) or absolute paths (used for off-the-shelf workflow)
  • Pro (over Helm)
    • Less infrastructure-as-a-code
    • No new abstraction layer or DSL to learn
    • No more need for branching/forking when a parameter is missing
    • Less replication of configuration sources thanks to generators
    • Support for GitOps rollback via git checkout and kubectl apply -k
  • Contra (over Helm)
    • Helm better supports immutability via packaging/repository
    • Helm allows logical grouping of applications
    • Helm GitOps rollback is just one step instead of two
UPDATE: Later on I found that I missed something interesting: Having both Helm and Kustomize completary instead of replacing each other.

References
Source code

Posted by tncad
Labels:

Comments

Post a Comment