Skip to main content

Posts

Showing posts with the label aws s3

038: Infrastructure provisioning with Terraform

Terraform is an Infrastructure-as-Code framework that is used to provision network, storage and computing resources on (nearly) any Cloud environment. The weekly mood I am back from vacation while many colleagues are still on leave. It has the advantage for me to review my past achievements, potentially go back to some topics I had not yet well understood or remembered, and think about further plans. I realize how much difficult it is to build my network and catch-up with projects when working from home. Naturally, I spend more time reading than chatting, which has the consequence than I don't learn anybody or anything by hazard, only on purpose. Also, I wonder a bit about my goals and areas of focus which are still disparate between concrete use-cases and general practices.  I still have to get more confident with DevOps therefore I am now looking at Terraform. What is Terraform Terraform is an open-source infrastructure-as-code (IaC aka. "InfraCode") software tool cre...

036: ML model serving with SageMaker

Amazon SageMaker is a Machine-Learning-as a-Service (MLaaS) framework that focuses on ML model development and automation, for example model serving. The daily mood I realize that I am looking much at Cloud Native technology but not using much of Cloud provider solutions, which is probably as much important if you do not want to re-invent the wheel at application level.  As already discussed in my  previous post  about "ML model deployment", we've been using both AWS Databricks and Amazon SageMaker as part of our internal Data lake project. In my last post , I looked at MLflow which is actually the relevant part of Databricks for deployment. Today I am looking at Amazon SageMaker which is indeed integrated by MLFlow for model deployment via the SageMaker SDK, but also offers a slightly different tooling and approach for development and operations. Why AWS for ML AWS currently offers one of the largest set of managed capabilities for Machine Learning . The offer consists ...

033: Data lake - Part 1 - Architecture

A Data lake is a corporate repository that bridges the gap between data storage and analytics via the support of multiple formats and query engines. The daily mood I had the opportunity to present GitHub Actions (see my previous post ) during our team call, with report on our evaluation activities and findings. This actually rose good questions and feedback. At the same time I've been concerned, potentially complaining, about not having a clear role and acceptance when entering projects. On one hand, I still have a lot to learn before becoming fully operational. On the other hand, I had not real engagement since I started, and cannot wait for it. There is one program which I can definitely contribute too, based on my available skills and experience: Building our internal Data lake. For some reason, this topic is pretty must resting in the background of other initiatives although it has actually been initiated 2 years ago. So that I fell empowered to make it great again ;-) Source:...