The newbie cloud architect diary

Amazon SageMaker is a Machine-Learning-as a-Service (MLaaS) framework that focuses on ML model development and automation, for example model serving. The daily mood I realize that I am looking much at Cloud Native technology but not using much of Cloud provider solutions, which is probably as much important if you do not want to re-invent the wheel at application level.  As already discussed in my  previous post  about "ML model deployment", we've been using both AWS Databricks and Amazon SageMaker as part of our internal Data lake project. In my last post , I looked at MLflow which is actually the relevant part of Databricks for deployment. Today I am looking at Amazon SageMaker which is indeed integrated by MLFlow for model deployment via the SageMaker SDK, but also offers a slightly different tooling and approach for development and operations. Why AWS for ML AWS currently offers one of the largest set of managed capabilities for Machine Learning . The offer consists ...

Because a Machine Learning model is a living asset, it is best-practice to automate its development  lifecycle to ensure repeatability. The daily mood Today was my weekly office day and I had the opportunity to meet a colleague from the SRE team, that I had actually not seen for a long time anymore. He told me about the SRE organisation, pains and achievements, which I found very insightfull. We also joined 3 other colleagues for quite a long lunch break at the terrasse of a restaurant. None of us had much social contact during the last months of lock-down, and it looks like we all enjoyed meeting again. Beside that, I am now looking at some ML tooling for enforcing best-practices and operationalization, that we are using in a project (see my previous post on ML model deployment) and which I was not yet much familiar with. First candidate MLflow is actually pretty straight forward, at least in theory. What is MLflow MLflow is a ML lifecycle management platform written in Python. ...

Knative is a serverless framework for Service-Mesh and Event-Mesh architectures that make it easier to run elastic applications in Kubernetes. The daily mood I am just starting to realise how much my work life has changed with my "new" job. No more technical marketing, only questionable facts. No more Business trips, only Home-office. No more telephone calls, only Slack chats. No more productivity tools like Trello  and Todoist , only "old-school"  Jira projects. There are also things that didn't change, like daily concerns about wether I am doing the right thing at the right time in order to perform as an employee, while protecting my work-life-balance. They are actually days in and days out, which people might count so or so... Provided that my organisation is currently pivoting its business and technology, automation is the key. Today is our 2nd post around Cloud native and we are looking at Serverless in the context of Microservices and Kubernetes, or the...

A Data lake is a corporate repository that bridges the gap between data storage and analytics via the support of multiple formats and query engines. The daily mood I had the opportunity to present GitHub Actions (see my previous post ) during our team call, with report on our evaluation activities and findings. This actually rose good questions and feedback. At the same time I've been concerned, potentially complaining, about not having a clear role and acceptance when entering projects. On one hand, I still have a lot to learn before becoming fully operational. On the other hand, I had not real engagement since I started, and cannot wait for it. There is one program which I can definitely contribute too, based on my available skills and experience: Building our internal Data lake. For some reason, this topic is pretty must resting in the background of other initiatives although it has actually been initiated 2 years ago. So that I fell empowered to make it great again ;-) Source:...

A Service Mesh is a managed network of Microservices. Its model is probably one of the most powerful enablers for a dynamic Cloud Native architecture. The daily mood After 2 months part-time on-ramping, I am now set at full-time architect (as originally agreed). I just had the opportunity to present my work on "Flux featuring with kustomize" (see my previous post ) to the team, and didn't hear any negative feedback although I had not even prepared a Powerpoint. It's always good to get an assessment from peers.  I may not have so much time for this blog in the future, but I definitely expect to follow-up on a number of topics that I feel right with. I am now starting a new series of posts around Cloud Native , or the high-level abstraction of Kubernetes architectures through components helping at "solving the boring but difficult". What is Cloud Native The term was born when Kubernetes v1.0 was officially annouced at  CNCF  2015 conference. At its core, the f...