The newbie cloud architect diary

Amazon SageMaker is a Machine-Learning-as a-Service (MLaaS) framework that focuses on ML model development and automation, for example model serving. The daily mood I realize that I am looking much at Cloud Native technology but not using much of Cloud provider solutions, which is probably as much important if you do not want to re-invent the wheel at application level.  As already discussed in my  previous post  about "ML model deployment", we've been using both AWS Databricks and Amazon SageMaker as part of our internal Data lake project. In my last post , I looked at MLflow which is actually the relevant part of Databricks for deployment. Today I am looking at Amazon SageMaker which is indeed integrated by MLFlow for model deployment via the SageMaker SDK, but also offers a slightly different tooling and approach for development and operations. Why AWS for ML AWS currently offers one of the largest set of managed capabilities for Machine Learning . The offer consists ...

Machine Learning (ML) Deployment is one of the dark sides of both Data Science and Data Engineering. Mmanaged services like Databricks might help. The daily mood As already mentioned in a previous post , I have the privilege to shadow a starting Data lake project . The team already prepared data and built a first Machine Learning (ML) model for a specific use-case.  They are currently in the process of deploying the  scoring application and find this quite challenging. We are going to discuss why it is difficult, and of course how technology and automation may help. Big Data & ML adoption In 2009, the Knowledge Discovery in Data Minining (KDD) conference became a competition (KDD Cup) which reached the IT world with a  disrupting report  of lessons learnt in large scale ML projects. The industry just started to realize the rize of Big Data (ex. IoT) and the potential of ML for Business. In the following years,  Business Intelligence  (BI) organizati...