Taking a "Data as a Product" approach to democratize data access

Oren Ezra

Oren Ezra

CMO, K2View

For data-driven enterprises, data is an incredibly valuable commodity. But in order to take a "data as a product"  approach, its overall quality must be assured. An operational data fabric does just that.

Table of Contents


The Proliferation of Data
What is a Data-Driven Enterprise?
Data as a Product and the Data Delivery Lifecycle
A New Role: Data Product Manager
Best Practices for Data as a Product
The Business Entity – the Logic Behind Data as a Product
Data as a Fabric Facilitates Data as a Product
K2View Data Fabric: Data as a Product Inside

The Proliferation of Data

As digitization grows, so does the amount of data that’s available to an enterprise. The sheer volume of digital products, services, and business models, combined with greater connectivity to devices, has led data to proliferate exponentially. With 90% of the world’s data created in the past 2 years, enterprises are becoming more and more data-driven.

According to McKinsey, data-driven companies are 23 times more likely to acquire customers, and 19 times more likely to be profitable.

What is a Data-Driven Enterprise?

A data-driven enterprise maximizes the value of its data by treating its data as a product, and differentiating data based on its overall quality (e.g., completeness, availability, accessibility, and general fitness for use). It treats data as a product in order to drive business outcomes, for example:

  • A telco predicting likelihood to churn in real time, during a customer interaction

  • A media company serving personalized content to its subscribers

  • A bank promoting a new financial product to a targeted client segment

Data as a Product and the Data Delivery Lifecycle 

To take a "Data as a Product" approach, data teams must adopt a cross-functional product lifecycle approach to data. The data product delivery lifecycle should follow agile principles, by being short and iterative - to deliver quick, incremental value to consumers of the data.

Define and Design the Data Product
Define the data requirements, within the context of the business objectives, the constraints of data privacy and governance, and the inventory of existing data assets. Design how the data will be structured and how it will be componentized as a product, to be consumed via services.

Engineer the Data Product
Build the data product according to the requirements by identifying, integrating, and collating the data from its sources, and then masking it as needed. Create web services APIs to enable consuming applications with the right credentials to access the data product, and devise pipelines to securely publish the data to subscribers. 

QA the Data Product
Test and validate the data to ensure it is complete, compliant, and timely, and that it can be securely consumed by applications at high scale. 

Support and Maintain the Data Product
Continually monitor data usage, pipeline performance, and reliability, and work closely with data engineering to address issues per defined SLAs.

A New Role: Data Product Manager

Much like software product development, where the software product manager is responsible for gathering user needs, prioritizing them, and working with software development and QA to ensure the right product is delivered at the right time, we believe that there is a place for a similar role in the data team. The data product manager will be responsible to collect data needs from data consumers (data scientists, data analysts, application owners), prioritize them, and work closely with data engineering to deliver the data product on time and on budget.

The data product must deliver business value, and realize ROI, such as more informed decision making, quicker application development, and more. For this to happen effectively, the data delivery must have a definitive timeline – a kind of service level agreement between IT and business.

27 image1

In the Data as a Product approach, data engineers, data testers, and data product managers collaborate
to deliver the right data to the right users at the right time

Best Practices for Data as a Product

Close collaboration
Data collectors and custodians should work closely with their consumers. This calls for experimentation and product evolution, and the ability to develop new features, or rollback changes, as needed.

Agile development
Data products must be developed quickly and reliably, meaning that data assets should be decoupled as much as possible. A good data catalog would be a good first step.

Comprehensive QA
By definition, building data products is a process. Data teams should always have a good CI/CD setup in place, and do their best to identify issues through automatic testing and data quality checks. And when things go wrong (which they inevitably will), be sure to learn from mistakes and improve the product.

High-speed availability
Data products have to be used by consumers in order to judge their value, so data engineers need to make them available quickly and easily. Standard interfaces should be used to accommodate the needs of diverse teams.

27 image2

Cross-functional collaboration leads to more flexible development, better QA, and quicker availability.

The Business Entity – the Logic Behind Data as a Product

The most obvious way to engineer a data product is to model it around the business entity that it supports, such as a customer, employee, credit card, product, or anything else that is important to the business.

Each business entity (customer John Smith) should be complete in all its attributes, enriched via analytics (propensity to churn), and easily accessible to any data consumer (person or application) that has access rights to that entity.

Usage of the business entity should be measurable. How is the data accessed, and how long does it take to get to it (response time)? How often is it accessed, and by whom? Who tried to access it, but didn’t have the right credentials? Which insights did it drive? The list goes on and on.

The overall quality of the data product must be assured, in terms of completeness, integrity, and freshness, in the sense that it’s always up-to-date.

Data as a Fabric Facilitates Data as a Product

A data fabric, which manages, prepares, and delivers data in the form of business entities, is the perfect platform for data as a product, because it inherently defines and manages the entire lifecycle of the data, and productizes all the data flows used to create, maintain and deliver the data to its consumers. A data fabric defines an intermediary data schema aggregating all the attributes of a business entity (such as a customer, product, location or order) across all systems, in order to prepare and deliver the data as an integrated “digital entity”.

Such a data fabric is key to supporting the data as a product methodology. It essentially integrates data, from all sources, into digital entities – cleansing, validating, enriching, masking, and transforming it, every step of the way.

K2View Data Fabric: Data as a Product Inside

K2View Data Fabric, with its patented approach to organizing data as digital entities, effectively turns enterprise data into data products – enabling companies to proactively adopt the data as a product mindset necessary to sustain data-driven leadership.

Learn how the concept of "data as a product" can work for you by reading the Gartner report on data fabrics.