The newbie cloud architect diary

FluxCD GitOps operator supports Kustomize for dynamically featuring Kubernetes resources includind HelmRelease. This post is about solution analysis. The daily mood My manager looked at my current activity on Helm deployment toolchain, and latest findings as described in my  previous post on Flux Helm Operator + Kustomize .  He seems to generally support my directions and approach, but also raised some concerns and questions that I shall answer in this solution draft. Context We currently operate hundreds of services in Kubernetes clusters . Without claiming to have a Microservice architecture, we generally try to design small-sized standalone components that communicate through HTTP. We also try to develop and test with agility ( Scrum ), create immutable deliveries ( Maven artifacts , Docker images , Helm charts ), use resilient infrastructure, operationalise for reliability. Our services are actually shared by a dozen of different teams, combined via logical stacks on...

FluxCD GitOps operator supports Kustomize for dynamically featuring Kubernetes resources including HelmRelease. This post is about parametrization. The daily mood I got in touch with another team mate, also well aware of, and willing to help on Helm deployments. He gave me some precious pointers on our current solution (based on Ansible) which allowed me some additional level of understanding. He also pointed out that following to the past PoC on Flux, HelmReleases were introduced as the future solution, but had the negative effect to make life even more complicated for developers, and configurations even more replicated by SRE. He suggested myself to study the possibility of using Kustomize to feature HelmReleases, so that replications can be avoided. In fact I had not yet considered Helm and Kustomize as a potential fit so far, except if we wanted to render Kubernetes resources first, then configure them as described in this article . I realised about how many different tools we were...

Skaffold Helm supports automatical deployment of composed and configured Helm charts to a local or remote Kubernetes cluster environment. The daily mood I had the opportunity to assist a status meeting from colleagues working on a new Data Lake project on AWS. The team basically consists in an engineering manager, a software engineer, 2 data scientists and a SRE. The first use case is a REST endpoint able to predict the duration of a given process. They struggled a bit with inter-VPC-security as the data to analyse was actually ingested on a different account than the data access and analysis layer. In parallel I am still focused on our software delivery process for Kubernetes, with in mind the goal to rampup on related tools, and potentially, give an improvement proposal back to our team and organization. In a previous post on Skaffold, we have seen how easy it is to setup a project and automatically build/deploy applications to Kubernetes. Our example was based on a kubectl pipeline...

Helmfile is a declarative spec and client tool that enables better composition, configuration and deployment of multiple Helm charts, than Umbrellas. The daily mood If find it a great practice that my peers often use our team call to share what they are about to present to our internal customers, that is to say our Product Management (PM) team or Chief Technology Officer (CTO). They usually show only 2-3 slides from a larger presentation for which they are unsure or required feedback. The occasion for me to get familiar with diverse topics e.g. API versioning lifecycle, Security, Observability. I am also impressed how fast the other members of the team can connect with a core problem or solution without a reminder on the context, and then ask important questions. In my experience of customer-facing, presentations always started with a hook and reference, in order to get attention and credibility on what you are going to present next (cf. Great Demo from Peter Cohan, Talk like Ted fro...

Flux Helm Operator supports the automated deployment of Helm charts to Kubernetes while maintaining compliance with the GitOps operational model. The daily mood I watched a bit the  DockerCon 2020 . Although most talks are addressed to a C-Level or beginner audience, I found some interesting pointers around application packaging, integration and delivery. Among others, I will definitely have a look at KubeStack  which seems to be able to solve some problems around GitOps.  My today's goal is to have a look at Flux for automating Helm chart deployments. Source: Revelry.co / Bitnami Requirements Same as in this  previous post about Flux without Helm. Installation steps Delete previous objects kubectl delete namespace flux kubectl delete clusterrole flux Add new Helm repository helm repo add fluxcd https://charts.fluxcd.io Apply HelmRelease CRD to the cluster kubectl apply -f https://raw.githubusercontent.com/fluxcd/helm-operator/master/deploy/crds.yaml Create Flux na...

This article discusses different proxy types and introduces Envoy, a high performance distributed proxy designed for Microservices architectures. The daily mood In the past I intensively worked on our Platform-as-a-Service (PaaS) as a user. Although I was interested in it, I didn't have access to architecture details, developer repositories, operational processes and environments. Now that I have all this, it feels a bit stange that I am not yet able to bring them to life, in a way that I could immediately run and use our solutions as needed... Not that easy! Indeed, an application that basically looks like a monolith web server and database on the frontend side, actually appears to be a big and resource-intensive collection of  Microservices . Those need to be configured, secured and provisioned individually so that the application can be operated in Kubernetes.  This finding "positively" explains how we can scale accross a large number of tenants. It also "negati...

Traefik is an open-source HTTP reverse proxy and load balancer that is easy to use.  Let Traefik point at your orchestrator and you are ready to go. The daily mood A fellow peer explained me our approach to Software deployment . Instead of getting lost in tons of infromation sources, it is such a high value to get told about the history, as well as to get idiot questions answered. In this case, that opportunity definitely raised my degree of understanding, comfort and motivation. Project history Following to our adoption of Helm for immutable Kubernetes packaging, our SRE team leveraged known tools for configuration management ( Ansible ) and pipeline automation ( Jenkins ). In a few years our platform grew to a hundred of services developped by 12 teams, staging accross 4 environments and different Cloud imfrastructure providers. At some point, we not only needed more DevOps automation, but also process standardization and parametrization (e.g. via templating). Ansible ...

Helm is a template-based spec format for configuring, packaging and deploying  applications that consist in multiple resource objects to Kubernetes. The daily mood My last posts were about setting up a Kubernetes environment. We'll now learn the tooling. In general i am not yet much familiar with common developer productivity tools like  Z-Shell ,  Gists  and  Lints , but i assume this will come with the time. Now we are going to look into the packaging of Kubernetes applications. What is Helm Helm is a commonly used package manager for Kubernetes. It is not the only one option for managing your Kubernetes applications but the one which happens to reach critical mass at enterprise grade, just like Docker did for Containerization and Kubernetes for Orchestration. One of the reason for its great popularity is the Helm Hub , the official Helm public repository which is hosting tons of standard packages. At its core, Helm generates Kubernetes manifests out of...