🧠Second Brain
Search
Data Warehouse Automation (DWA)
A very long time ago, 1986, 31 years ago to be precise, IBM in Europe created the very first architecture of a data warehouse. And it seems to be a masterpiece as it hasn’t changed much since. How can we improve or bring some innovation into the Data Warehouse business in times when everyone is talking about big data, Data Lakes, the Internet of Things (IoT), predictive analytics, Data Vault, etc.?
No matter how we want to improve the architecture, it has to be automated as much as possible. Nowadays, it has become too slow to serve the business needs by doing it the traditional way. However, I don’t think DWH will go away anytime soon (see more DWH vs Data Lake). I strongly believe that DWA tools are the future and will boost the Data Warehousing reputation back to earlier years.
But why is Data Warehouse Automation not used more often and more popular? I’m asking that myself more and more. That’s why I’m writing a series of blog posts all about DWA. In this first blog, I’m trying to find possible reasons behind and also argue for DWA, and why we should use it more often.
Everyone needs to make data-driven decisions faster, so why not use a generator that gives you answers in days instead of months..?
More content on my DWA articles:
- Data Warehouse Automation (DWA) – Series
- What Data Warehouse Automation tools are on the market
- Why automate? What does Data Warehouse Automation for us?
- Why Data Warehouse Automation is not more popular
# Other Data Warehouse Automation Tools (DWA)
# Orchestration
A DWA is essentially an orchestrator (keyword automation) with adding data modeling capabilities an orchestrator typically doesn’t have but without DAGs.
A difference between model DAGs and traditional orchestrators is DWA’s model dimensions and DAGs’ model data flow.
Origin:
Data Warehouse Automation (DWA) – Series | ssp.sh
References:
Created 2023-11-16