šŸ§  Second Brain


Search IconIcon to open search

Open Standards

Last updated Jul 16, 2023

With the explosion of tools in the open-source data ecosystem, there is the need to couple them, maximize the tools into a single data stack, and simplify along the way. To achieve this, we need open standards that everyone is implementing.

There are already standards such as S3 for theĀ  storageĀ interface (many other storage providers implemented the S3 API as a default) andĀ  Apache ParquetĀ asĀ  File FormatĀ in data lakes. Or standards such asĀ  Table Formats, which bundle distributed files into one database-like table. It is an abstraction layer between your physical data files and how they are structured to form a table.

Open standards are also vital for the Open Data Stack and Modern Data Stack’s success in integrating the various tools into a powerful data stack. With more existing tools maturing, more will transform them into open data standards. Time will tell which tools it will be.

Open standards build on the foundation of Openness.

Created 2023-01-25