Data Mesh Architecture
Data mesh architecture empowers business domains to create, access, and control their own data products.
Data mesh architecture benefits
Why use a data mesh architecture?
Domain
autonomy
Enable business domains to create, consume, and share data products.
Speed and
agility
Define and adapt data products
at the speed of business.
Federated governance
Control data quality, privacy and access, at any level of federation.
“Data mesh architecture allows for decentralized data management. It promises to provide the flexibility domains need to build the data products they require"
Roxane Edjlali, Sr. Director Analyst, Gartner
Modernized data management
Data mesh architecture
Domain-owned data products
Data products are created and owned by SMEs in the domains where data is produced
Distributed data management
Decentralized data management accelerates time to data and improves business agility
Shared services and best practices
Global data quality, privacy, and security policies are enforced, and best practices are shared
Common data language
Collaboration between data product producers and consumers is improved, and trust in data elevated
4 data mesh principles
How K2view supports data mesh architecture
1. DOMAIN-DRIVEN OWNERSHIP OF DATA
Share trusted data across domains
Data teams in a domain create, publish, and adapt data products for use by authorized data consumers in any domain.
- Data products are created by data SMEs in the domains in which the data is produced
- Data is externalized via data products through any combination of: APIs, streaming, messaging, CDC, and SQL
- Data products are discoverable and accessible by authorized data consumers
- Centralizing multi-domain master data for cross-company consumption
2. Data-as-a-product approach
Package data with everything needed to increase usage and trust
A data product bundles data together with its metadata and business logic to make it independently usable by data consumers.
-
The metadata includes the data product schema, its access controls, governance policies, SLAs, and more
-
The business logic for ingesting, cleansing, transforming, enriching, synthesizing, and delivering data is translated to code
- The data product platform executes the data product, based on its metadata and code
- A K2view data product packages its data in Micro-Databases, with the data for each business entity stored in its own Micro-Database
3. Self-serve data platform
Empower domain teams by abstracting complexity
Core to the data mesh architecture is a self-serve data product platform that enables business domains users, data product engineers, and platform admins to create, share, and consume data products that deliver business value.
- A semantic data layer, together with no-code tooling, enable data product engineers to quickly and autonomously create and share data products
- A data product catalog enables data users in the domains to discover and consume data products based on their roles and privileges
- Admins monitor data product performance and usage, integrate with CI/CD platforms, set access controls, and more
4. Federated governance
Embed data quality and privacy policies into data products
Data mesh architecture employs centralized data governance to defines the global data quality and data compliance policies.
- Global data governance is implemented in the data mesh hub
- Data domains implement global data governance policies in their respective data products
- Data domains employ domain-specific data governance as required
See the K2view platform in action
Experience a free interactive demo, or book a demo with our experts
k2view data product platform as a data mesh
Key components of data mesh architecture
Data Product Management
Create, test, deploy, monitor, and adapt data products
Data Integration and Delivery
Integrate and deliver data in any method, in bulk or real time
Data Catalog
Discover, profile, and classify data assets for use in the data products
Federated Data Governance
Employ data quality, governance, and security policies globally and locally
Self-Serve Tooling
Enable domain SMEs with the no-code/low-code tools they need
Decentralized DataOps
Monitor, control, and adapt data product usage, performance, and value
Data mesh architecture
frequently asked questions
What is data mesh?
Data mesh is a decentralized data management approach enabling business domains to define, deliver, maintain, and govern data products.
Data products integrate, process, and deliver data for use by authorized data consumers (business users, data analysts, data engineers or other systems).
Data products are easily discovered and consumed, and deliver value according to SLAs that are agreed to between data product creators and consumers.
What is the difference between data mesh architecture and data fabric?
Data fabric architecture is an emerging data management design pattern that focuses on the technology components required to achieve scalable, centralized data management.
On the other hand, data mesh architecture focuses on the decentralized operating model and tooling needed to implement distributed data management.
What are the 4 principles of data mesh?
Data mesh architecture and operating model are grounded on four principles, which are supported by the K2view Data Product Platform:
- Domain-driven data ownership: Data mesh decentralizes and distributes data management responsibility to SMEs who are closest to the data in order to support continuous change and scalability.
- Data as-a-product: Data that's delivered by domains must be treated as a product and the consumers of that data should be treated as customers - happy and delighted customers. Data should be built, packaged, released, and monitored to deliver value according to agreed-upon SLAs.
- Self-serve data platform: This is required to enable the data domains to have the autonomy for creating and consuming data products they need, without requiring specialized technical skills.
- Federated computational governance: Enabling and empowering domain data product owners with domain-local data governance abilities, while adhering to a set of global policies that are applied to all data products and their interfaces.
When should data mesh NOT be used?
Data mesh architecture and its operating model are emerging data management practices that support distributed data management in decentralized enterprises with a high-level of data complexity. As such, data mesh architecture is less applicable for organizations that:
- Are small
- Have a low level of data complexity
- Are not domain-centric
- Have a low level of data management maturity
- Have insufficient data management expertise in the domains
- Are not ready to embrace data product lifecycle management
How is data mesh different vs data lake?
While data mesh is a distributed data management architecture, data lake is a centralized repository of enterprise data in its raw format. As such, data lake can be the end-point of a data mesh.
Read more about data mesh vs. data lake
What use cases does data mesh support?
Data mesh architecture supports operational and analytical use cases, such as:
- Data lake pipelining: preparing and delivering data for analytics
- Cloud migration
- Customer 360: Real-time customer data integration
- Multi-domain master data management
