🧠Second Brain
Search
Data Engineering Architecture
A good data engineering architecture is hard.
But the best one is the overview from Emerging Architectures for Modern Data Infrastructure - a16z (or Zotero: The Guide to Modern Data Architecture | Future). Check also Perfect DWH architecture.
# Common Architectures
- Examples and Types of Data Architecture:
- Data Warehouse
- Cloud Data Warehouses
- Data Marts
- Data Lake
- Modern Data Stack vs Open Data Stack
- Lambda Architecture vs Kappa Architecture, Medallion Architecture
- Metrics Layer vs Semantic Warehouse vs Data Virtualization
- Metrics, Key Performance Indicator (KPI)
- Push-Downs vs rollup
- Data Modeling vs Dimensional Modeling
- Data Contracts
- Delta Lake architecture where they unified batch processing and Streaming
- Architecture for IoT
- Data Mesh
- Data architectures have countless other variations
- Data Fabric, data hub, scaled architecture, metadata-first architecture, event-driven architecture, live data stack, and many more. Reis and Housley - Fundamentals of Data Engineering.pdf
Many overlap and more I have listed in Data Modeling.
# Data Architectures Images
Updated:
by
Emerging Architectures for Modern Data Infrastructure | Andreessen Horowitz
# People (on top of the illustration)
RW The Modern Data Graph - By Stephen Bailey
See a collection of many more on Data Engineering Architectures Overview or 4 Data Architectures.pdf, or Perfect DWH architecture.
Origin:
References:
Created 2022-04-05