While Iceberg solves this problem, open standards are needed in other areas as well. We’re now seeing a new battlefield emerging in the area of data catalogs, which play a critical role in a multi-engine architecture. Catalogs make operations on tables reliable by supporting atomic transactions. This means that data engineers and the pipelines they build can modify tables concurrently, and queries on these tables produce accurate results. To accomplish this, all Iceberg-table read and write operations, even from different engines, are routed through a catalog.
SaaS providers and hyperscalers can use the catalog as a way to create customer stickiness, but enterprises are getting wise to this. They understand that, just as Iceberg provides a common format for tables, an open catalog standard will let them choose the best tool for the job and maximize the value of their data.
Open standards are good for business, good for customers, and good for the wider ecosystem. Enterprises have complex data architectures, and open standards allow them to use data across these platforms without incurring additional cost and governance challenges. Open standards also promote innovation, because they force companies to compete on an implementation and allow customers to choose between them.