Domain ownership, information as an asset, data catalogues and context-aware governance should all inform how architecture is modelled and used
First introduced in 2019, data mesh is a decentralised, platform-agnostic approach to managing analytical data that involves organising information by business domain. The concept has quickly grown in popularity, with 60% of companies with over 1,000 employees having adopted data mesh in some form within three years of its inception.
According to Krzysztof Sazon, Senior Data Engineer at STX Next, businesses seeking to implement a data mesh strategy must ensure they adhere to four vital principles.
Take ownership of domains
Teams managing data must ensure it is structured and aligns with business needs.
Sazon said: “Ultimately, an organisation is responsible for its data. Therefore, specialist teams should be able to quantify the reliability of the information they store, understand what is missing and explain how information was generated.
“Centralised teams, on the other hand, are strangers to the organisation’s overarching data infrastructure and lack the understanding of where data is stored and how it is accessed. Giving specialist teams the chance to implement ideas and populate data warehouses while stored information is recent allows businesses to unlock the potential of the data at their disposal.”
Treating data as a product
Businesses must ensure their data upholds a specific set of standards if it is to become an asset they can leverage to drive growth.
“The data mesh approach advocates treating data like a product, where domains take ownership of the data they generate and grant access to internal users. This shifts the perception of data as a byproduct of business activity, elevating its status to a primary output and asset that companies actively curate and share.
“Data must adhere to specific standards if it is to become an asset: it should be easy to navigate, trustworthy, align with internal processes and comply with external regulations. Central data teams must build the infrastructure to support these principles, making data accessible and discoverable in the process.”
Implementing self-serve architecture
Users should be able to navigate stores of information on their own, without consulting a middleman.
Sazon continued: “It’s vital employees have the ability to autonomously navigate internal data product stores relevant to their business sector. To facilitate this, a catalogue of all data products, with a search engine that provides detailed and up-to-date information about the datasets, is a key requirement.
“There should be no need to ask external teams to set up data sharing and updating. Ideally, this is automated as much as possible and provides useful features, such as data lineage, instantly.”
Federated computational governance
Governance of data mesh architectures should embrace decentralisation and domain self-sovereignty.
Sazon concluded: “Governance is more of an ongoing process than a fixed set of policies – rules, categorisations and other properties evolve as needs change. Typically, there is a central data standards committee and local data stewards responsible for compliance in their domains, allowing for consistency, oversight and context-aware governance.
“Federated computational governance enables decentralised data ownership and flexibility in local data management, while maintaining standards throughout the organisation.”
