Search
Search Icon Icon to open search
Data Engineering Concepts
Last updated
Nov 11, 2024
There are many data engineering concepts. Most of them are described in this second brain. But here’s a table of the most important in data engineering:
Category
Data Engineering Topics
Storage Solutions
Data Warehouse , Data Lake , Data Lakehouse , OLAP , OLTP , NoSQL , Storage Layer , Data Lake File Formats , Data Lake Table Format
Data Processing
ELT , ETL , EtLT , Reverse ETL , ELTP , Notebooks , MapReduce , Hadoop , Streaming , batch processing , Data Orchestration
Data Architecture
Modern Data Stack , Open Data Stack , Functional Data Engineering , Software-Defined Asset , Declarative vs Imperative , Data Engineering Lifecycle , Data Lifecycle
Data Management
Data Catalog , Data Contracts , Data Governance , Data Quality , Data Security , Slowly Changing Dimension , Data Modeling , Schema Drift , Logging Pipelines , idempotency
Data Modeling & Warehousing
Metrics Layer , Semantic SQL , Semantic Warehouse , Data Virtualization , Data Modeling , Dimensional Modeling , Conceptual, Logical to physical Data Models , Data Vault , Relational Model , Normalization , Denormalization , Granularity , Cardinality
Performance & Optimization
Push-downs , Metrics , KPI , CAP Theorem
Cloud & Big Data Technologies
Cloud Data Warehouses , Cloud Data Provider , Data Fabric , Data Mesh , MLOps
Data Integration & Federation
Data Orchestrators , Data Lineage , Graph Databases , GraphQL , Analytics API , Data Integration
See also Data Engineering Approaches or Data Engineering Vault .
Origin:2023-08-21