The newbie cloud architect diary

Data Build Tool (DBT) allows you to easily create and manage ELT-pipelines to automatically build and populate a target analytical model. The weekly mood Thanksgiving is over and I wish I could integrate an Advent calendar into my blog, but in this case I should have had the idea before. Therefore, let us put it to next year... perhaps. Talking about ideas, I am still puzzled on how to help my organisation becoming data-driven. Is it ok to struggle with some basic Data Warehouse implementation, or is it a shame? As a matter of fact, we need to do something, and we need to do it now. See also: Building An Analytics-Centric Organisation . Today we'll have a look at a tool that is quite popular for populating dimensional data models: DBT. What is DBT Data Build Tool (DBT) is an Open-Source CLI written in Python that is used to manage, document, test, compile, package and run compute queries on a  Data Warehouse  (DW) or a Data Lake (DL). DBT is developed by a Philadelphia-...

Tableau Desktop is  a powerful BI tool for the Data Warehouse. Tableau Prep is a promising but less mature product for  cleansing  new Datasets. The weekly mood This the end of my first quarter working as a full-time Architect. During this period, I had the opportunity to deep-dive into different topics such as DevOps, API and Data Management. It feels like a year from a content perspective, but actually not more than a month from a collaborative standpoint. Indeed, it has been and it is still a challenge for me to connect with other people within a distributed organisation and a world of pandemic. And as I am currently gaining visibility through a Data Lake project, I remind that as for my previous position (I've been working for years as a Solution Engineer), this Architect role might be sometimes very unclear, difficult to endorse and align with other people's expectation. Architecture is a double-edge. It brings people together around  Software design principles ...

Snowflake is a Cloud Data Warehouse that is comprehensive to use and probably belongs to the most successful cloud platforms over the last few years. The weekly mood This week slightly challenged my sense of responsibility and communication . It started with some (expected) sign of non-satisfaction from my manager, and ended with some (unexpected) sign of Leadership from me. In fact there are enough areas where I usually put enough of pressure on myself, like for example "delivering what I promised". However, I still definitely need a manager to motivate me for other areas like for example "making things happen". As an outcome, I am currently enjoying a much better team work around our Data Lake project and API Gateway evaluation. It's a shame that this would probably not happen without an external trigger, but that's also the essence of Lifelong learning . This experience reflects the common mistake that people make when focusing on results rather than on o...