The newbie cloud architect diary

MongoDB is a popular document store for backing Web applications, but a challenge for the Data Engineer to ingest its content into a Data Warehouse. The weekly mood We worked in team around the functional specification of our API Gateway as well as our Data Lake project, which I enjoyed very much.  As part of the Data Lake, we currently need to source and analyse information from a specific Web application that uses MongoDB as operational database. Aggregations should be ran inside our corporate Cloud  Data Warehouse  (DW) based on  Snowflake , BI-reports in Tableau .  While this post sets the focus on MongoDB, I do plan a series of articles around Data Engineering and Architecture, including our finding based end-to-end transformation pipeline from source to destination. What is MongoDB MongoDB  is an open-source NoSQL database that stores its data in JSON-documents. The project started in 2007 at a company called "10gen" which was renamed Mongo...

Blogging is synonym for sharing something with the world,  building a digital footprint and finally growing a network. Here are  my tips about Blogger.    The weekly mood I started this blog along with my new position. I've now been in for 60 working days already, among those 35 in part-time and 25 in full-time. During that time I wrote 40 blog posts. Since a couple of people were curious how about how that works for me, I thought it would be a good idea to write about it. My experience with the Web I belong to that generation born before the (global) Internet . I had the chance to put my hands on a computer at primary school but my first contact with the web was in the early 1990s at secondary school. To be honest back then, it wasn't a great experience to search for academic content through AltaVista using a  Netscape navigator. The broadband was really slow, the capabilities very limited and the content extremely poor. I actually preferred playing Arcade G...

With the proliferation of Microservices, a C ontract-1st approach keeps API changes consistent while an API Gateway keeps API operations under control. The weekly mood This is just another office day without too much project activity, so that i have enough time to review pre-defined team tracks, as well as my past achievements and future directions.  I am now looking at our  API Architecture  Re-design. While our Platform based on Microservices architecture grew with the time, we happen to have quite a lot of heterogeneous APIs. This characteristic tends to become hard to manage and to maintain. Therefore we started a dedicated initiative. Why is API a hot topic? In a nutshell, an  Application Programming Interface  (API) is a generic facade that enables modularity and integration between software applications and components, independently from their technologies.  In an API driven economy (aka. API Economy ), APIs are actually the main point of i...