Skip to main content

Posts

Showing posts with the label data lake

049: Data Lake - Part 2 - Governance

Data Governance is such a wide topic in theory, that most people are confused about which practical approach to choose regarding their Data Lake. The weekly mood I finally have the warm feeling that I well integrated the department which I joined nearly half a year ago. Why such a long time for such a simple goal? Well, if I had to retrospect on this development, I would split it into the following under-estimated phases: Observation Fail-fast  Exploration Deep-dive Immersion Go Mainstream I was definitely not as easy as I expected given my background and motivation. I was glad to have the chance to touch-point on various topics from DevOps to API and Big Data. It was much less pleasant to partly have to wait for months until I got access to the systems and services I needed. Obviously there were some social restrictions related to COVID-19 and time-zones. To finish with, I not only needed to get used to our workflows, but confident with our goals, which also asked for some normal ...

033: Data lake - Part 1 - Architecture

A Data lake is a corporate repository that bridges the gap between data storage and analytics via the support of multiple formats and query engines. The daily mood I had the opportunity to present GitHub Actions (see my previous post ) during our team call, with report on our evaluation activities and findings. This actually rose good questions and feedback. At the same time I've been concerned, potentially complaining, about not having a clear role and acceptance when entering projects. On one hand, I still have a lot to learn before becoming fully operational. On the other hand, I had not real engagement since I started, and cannot wait for it. There is one program which I can definitely contribute too, based on my available skills and experience: Building our internal Data lake. For some reason, this topic is pretty must resting in the background of other initiatives although it has actually been initiated 2 years ago. So that I fell empowered to make it great again ;-) Source:...