K2VIEW ebook

What is Data Masking?

The complete data masking guide

Data masking protects sensitive data while retaining the data's integrity and usability. It helps organizations comply with privacy regulations and reduce the risk of unauthorized access to confidential information.

Data Masking_Upper Banner

INTRO

Data masking: An imperative for today’s enterprises

With the proliferation of personal data – collected by enterprises across all industries – the need for protecting individual privacy is paramount. One way to protect Personally Identifiable Information (PII) is by masking data (i.e., consistently changing names, or including only the last 4 digits in a credit card or Social Security Number).

This reference guide explores today’s data masking techniques, the challenges they pose for enterprises, and a novel approach, based on data products, that addresses these challenges in the most comprehensive manner.

Chapter 01

What is data masking?

Data masking software protects sensitive data by creating a version of the data that can’t be identified or reverse-engineered. It should assure data consistency, and usability, across multiple databases.

Data masking process
Data masking substitutes real information with random characters.

The most common types of data masking include:

  • PII: Personally Identifiable Information, in response to privacy regulations, such as GDPR and CPRA
  • PCI-DSS: Payment Card Industry Data Security Standard (payment card information)
  • PHI: Protected Health Information
  • IP: Intellectual Property

Data masking best practices call for its use in non-production environments – such as software development, data science, and testing – that don’t require the original production data.

Simply defined, data masking combines the processes and tools for making sensitive data unrecognizable, but usable, by software or authorized personnel.

Chapter 02

Data masking vs other data obfuscation methods

Data obfuscation refers to a variety of processes that transform data into another form, in order to secure and protect it. The 3 most common data obfuscation methods are data masking, data encryption, and data tokenization. While data masking is irreversible, encryption and tokenization are both reversible in the sense that the the original values can be derived from the obscured data. Here’s a brief explanation of the 3 methods:

Data masking

Data masking tools substitute realistic, but fake, data for the original values, to ensure data privacy. Development, support, data science, business intelligence, testing, and training teams use masked data in order to make use of a dataset without exposing real data to any risk.

There are many techniques for masking data, such as data scrambling, data blinding, or data shuffling, which will be explained in greater detail later on. The process of permanently removing all Personally Identifiable Information (PII) from sensitive data is also known as data anonymization, or data sanitization. There is no algorithm to recover the original values of masked data.

Data encryption

While data encryption is very secure, data teams can’t analyze or work with encrypted data. The more complex the encryption algorithm, the safer the data will be from unauthorized access. Encryption is ideal for storing or transfering sensitive data securely.

Data tokenization

Data tokenization, which substitutes a sensitive data element with random data (token), is a reversible process. The token can be mapped back to the original data, which is stored in a secure “data vault”.

In a data masking vs tokenization comparison, tokenization supports operations like processing a credit card payment without revealing the credit card number. The real data never leaves the organization, and can’t be seen or decrypted by a third-party processor.

Data masking vs data tokenizationData tokenization supports the Payment Card Industry Data Security.

So, what is data masking? It's the most common form of data obfuscation. The fact that data masking is not reversible makes it more secure, and less costly, than encryption.

Another big plus is that data masking maintains data integrity across systems and data bases, which is critical in software testing and data analysis. Minimizing the use of actual data protects an enterprise from unnecessary risk.

In the case of obfuscated data, integrity means that the dataset maintains its validity and consistency, despite undergoing data anonymization. For example, a real credit card number can be replaced by any 16-digit value that is validated by the “CheckSum” function. Once anonymized by a new value, the same (new) value must be used consistently across all systems.

In short, there are 2 major differences between data masking and other data obfuscation methods like encryption or tokenization:

  1. Masked data remains usable in its obfuscated form.
  2. Once data is masked, the original value can’t be recovered.

Chapter 03

Why data masking?

Data masking solutions are important to enterprises because they enables them to:

  • Maintain compliance with privacy laws, like GDPR and CCPA, by eliminating the risk of sensitive data exposure.
  • Protect data from cyberattacks, while preserving its usability and consistency.
  • Reduce the risk of data sharing, e.g., in the case of cloud migrations, or when integrating with third-party apps.

While data masking methods have been around for decades, they're now needed more than ever to effectively protect sensitive data, and to address the following challenges:

Regulatory compliance
Highly regulated industries, like financial services and healthcare, already operate under strict privacy regulations, including the Payment Card Industry Data Security Standard (PCI DSS), and the Health Insurance Portability and Accountability Act (HIPAA). Since the introduction of Europe’s GDPR in 2018, there has been a proliferation of privacy laws across the globe including CCPA and CCPR in California, LGPD in Brazil, and PDPA in the Philippines and Singapore. Such privacy laws seek to protect Personally Identifiable Information (PII) by, and restrict access to it whenever possible.

Insider threats
Many employees and third-party contractors access enterprise systems on a regular basis. Production systems are particularly vulnerable, because sensitive information is often used in development, testing, and other pre-production environments. With insider threats rising 47% since 2018, according to the Ponemon Institute report, containing sensitive data costs companies an average of more than $200,000 per year.

External threats
In 2020, personal data was compromised in 58% of the data breaches, states a Verizon report. The study further indicates that in 72% of the cases, the victims were large enterprises. With the vast volume, variety and velocity of enterprise data, it is no wonder that breaches proliferate. Taking measures to protect sensitive data in non-production environments will significantly reduce the risk.

Data governance
Data masking is commonly used to control data access. While static data masking obscures a single dataset, dynamic data masking provides more granular controls. With dynamic data masking, permissions can be granted or denied at many different levels. Only those with the appropriate access rights can access the real data. Others will see only the parts that they have to see.

Flexibility
Data masking is highly customizable. Data teams can choose which data fields get masked, and how to select and format each substitute value. For example, every Social Security Number (SSN) has the format xxx-xx-xxxx, where “x” is a number from 0 to 9. They can substitute the first five digits with the letter x, or all 9 numbers with other random numbers, according to their needs.

Free Gartner® Report:
Market Guide for Data Anonymization & Masking

Learn from industry firm Gartner about data data anonymization and data masking, including:

  • Market description: including dynamic and static data masking techniques
  • Critical capabilities: PII discovery, rule management, operations, and reporting
  • Data anonymization vendors: broken down by category

Chapter 04

Data masking approaches

Over time, a variety of data masking techniques have been devised. Selecting the right approach is dependent on the intended data use. The goal is to maximize data protection, while minimizing data exposure.

Static data masking
Non-production environments, such as those used for analytics, testing, training, and development purposes, often source data from production systems. In such cases, private data is protected with static data masking, a one-way transformation ensuring that the masking process cannot be undone. When it comes to testing and analytics, repeatability is a key concept because using the same input data delivers the same results. This requires the masked data values to persist, over time, and through multiple extractions.

Static data masking is usually employed on a copy of a production database. It makes data look real enough to permit accurate development, testing, and training, without exposing the original data.

Dynamic data masking
Dynamic data masking is used to protect, obscure, or block access to, sensitive data. While prevalent in production systems, it is also used when testers or data scientists require real data. Dynamic data masking is performed in real time, in response to a data request. When the data is located in multiple source systems, masking consistency is difficult, especially when dealing with disparate environments, and a wide variety of technologies. Dynamic data masking protects sensitive data on demand.

Dynamic data masking automatically streams data from a production environment, to avoid storing the masked data in a separate database. As a rule, it’s used for role-based security for applications – such as handling customer queries, or processing sensitive data, like health records – and in read-only scenarios, so that the masked data doesn’t get written back to the production system.

On-the-fly data masking
When analytics or test data is extracted from production systems, staging sites are often used to integrate, cleanse, and transform the data, before masking it. The masked data is then delivered to the analytics or testing environment. This multi-stage process is slow, cumbersome, and risky due to the possible exposure of private data.

On-the-fly data masking is performed on data as it moves from one environment to another, such as from production, to development or test. It’s ideal for enterprises engaging in continuous software development and large-scale data integrations. A subset of the masked data is generally delivered to authorized users upon request, because keeping a backup of all the masked data is inefficient and impractical.

Statistical data masking
Production data can hold different statistical information, which statistical data obscuration techniques can masquerade. Differential privacy is one technique where you can share information about patterns in a data set without revealing information about the actual individuals in the data set.

Test data masking
Applications, of any kind, require extensive testing before they can be released into production. Test data management tools that provision production data for testing must mask the test data to protect sensitive information. For example, in a legacy modernization program, the modernized software components must be tested continuously, making test data masking a key component in the testing process. Masking data with referential integrity – from production systems, to the test environments – is critical.

Unstructured data masking
Scanned documents and image files, such as insurance claims, bank checks, and medical records, contain sensitive data stored as images. Many different formats (e.g., pdf, png, csv, email, and Office docs) are used daily by enterprises in their regular interactions with individuals. With the potential for so much sensitive data to be exposed in unstructured files, the need for unstructured data masking is obvious.

Unstructured data masking

Masking of unstructured data is particularly important in financial services and healthcare industries

Enjoying the article? Save it as a PDF

Get the whitepaper

Chapter 05

Data masking techniques

There are several techniques associated with data masking, including:

Scrambling
Scrambling randomly orders characters and/or numbers to obscure the original content. For example, when a shipment with tracking number 572918 in a production environment undergoes character scrambling, it might read 125879 in a different environment. Although easy to implement, scrambling can only be used on certain data types, and is not as secure as other techniques.

Data Masking Diagram pillar page@100x-100
Data scrambling assures that the data can’t be easily traced back to its source.

Nullifying
Nullifying applies a null value to a data column so that unauthorized users won’t be able to see the actual data in it. Despite its ease of implementation, nullifying results in data with less integrity, which is often problematic in development and testing environments.

Substitution
Substitution, which replaces the original data with another value, is one of the most effective data masking techniques because it preserves the original nature of the data. Although difficult to execute, substitution can be applied to several types of data, and is excellent protection against data breaches.

Shuffling
Like substitution, shuffling uses the same individual masking data column for randomly ordering characters or numbers. For example, when patient name columns are shuffled across multiple patient records, the results look accurate but don’t reveal any personal medical information. However, anyone with access to the shuffling algorithm can reverse-engineer the process.

Date/number variance
Data/number variance is used for masking important financial and transaction date information. For example, masking the employee salaries column with the employee salary variance, displays the salaries between the highest- and lowest-paid employees. Data integrity can be assured by applying a variance of, say, +/- 5% to all salaries in the dataset.

Date aging
Date aging increases or decreases a date field based on a pre-defined data masking policy, within a specific date range. For example, decreasing the date of birth field by 1,000 days would change the date 1-January-2023 to 7-April-2020.

Chapter 06

Data masking challenges

To effectively answer the question "What is data masking?" the following challenge must be addressed: Not only must the altered data retain the basic characteristics of the original data, it must also be transformed enough to eliminate the risk of exposure, while retaining data integrity

Enterprise IT landscapes typically have many production systems, that are deployed on premises and in the cloud, across a wide variety of technologies. To mask data effectively, an organization needs to:

  1. Identify the sensitive data and PII that require protection
  2. Resolve identities to ensure the data integrity across systems. For example, If Rick Smith is masked as Sam Jones, that identity must be consistent wherever it is used
  3. Comply with company governance policies for role, location, and permissions-based data access
  4. Scale for real-time access and mass-batch data extraction
  5. Manage growing volumes of unstructured data

Chapter 07

Data masking with business entities

A business entity approach to data masking challenges ingests, organizes, processes, and delivers data from disparate systems by business entity (customer, order, device, or anything else that’s important to the business).

By masking the data for a particular business entity as a singular unit, regardless of the underlying source systems and their technologies, relational integrity of the masked data is maintained – assuring that the masked data for that entity is always consistent, and complete.

The data for each specific business entity is managed in its own Micro-Database™, which is encrypted by its own 256-bit encryption key. The PII in the Micro-Database is masked in-flight, according to predefined business rules.

Entity-based data masking supports dynamic data masking for operational use cases, like customer 360, and static data masking for test data management and legacy application modernization.

new UI screens-Data MaskingIn-flight screenshots-1
Benefits of business entities

Entity-based data masking eliminates the need for slow, cumbersome, and risk-prone staging areas, where unmasked data is exposed to potential breaches.

Using no-code data orchestration tools, data from multiple production systems is integrated, cleansed, and masked on the fly.

A business entity approach to data masking simplifies complexity, ensuring that an individual's customer data, which is fragmented across multiple sources is:

  • Consistent, across multiple sources
  • Persistent, over time and multiple extractions
  • Preserved, with referential integrity and formatting

Dynamic data masking

Dynamic data masking transforms, obscures, or blocks access to sensitive information fields based on user roles and testing environment privileges.

web_Dynamic Data Masking
And with data orchestration tools, a wide variety of in-line masking functions can be invoked to protect the data.

Unstructured data masking

Protect unstructured data including images, PDFs, XML, CSV, text-based files, and more, with static and dynamic masking capabilities. Unstructured data masking lets you:

  • Replace sensitive photos with fake alternatives
  • Use OCR to detect content and enable intelligent masking
  • Employ synthetic data generation, to create digital versions of receipts, checks, contracts and other items for testing purposes.

new-UI-screensDV-copy-4-3By managing unstructured data within a data product schema,
referential integrity and consistency are ensured.

Extensive and extendible masking functions

Entity-based data masking comes with a comprehensive library of prebuilt masking functions, designed to provide realistic, but fake, data.

The table below highlights a few examples, including masking that creates a valid social security number (SSN), selecting (masked) names from name directories, as well as generating random numbers, and address-based zip codes. The library can be easily extended by custom Java functions that implement additional masking functions.

Field

Masking function

SSN/National ID

Generate valid SSN

Credit card

Generate valid number based on card type

First name/Last name/Zip code

Select from collection

DOB

Shuffle (preserve statistical diversity)

Any String/number

Random String/number

Email

Concatenation based on new first and last name

Const

Static masking based on a pre provided value

Address

Based on the provided Zip

 

Chapter 08

Summary

Data masking has become a pillar technology that global enterprises use to comply with privacy protection regulations.

Although the practice of masking data has been around for years, the sheer volume of data – structured and unstructured – and the ever-changing regulatory environment, have increased the complexity of data masking at enterprise scale.

The offerings of the current data masking vendors are proving to be insufficient. However, a new approach, based on data products, is setting the data masking standard at some of the world’s largest organizations.

See in-flight data masking in action

Book a Demo

What is data masking?

Data masking protects sensitive information in a database by replacing it with a disguised version of the data. Data masking ensures that any Personally Identifiable Information (PII) remains secure and confidential, but permits the data to be used for development, testing, and other legitimate purposes. By masking data, enterprises assure data privacy and security, reduce the risk of mass data breaches, and comply with privacy regulations such as the GDPR, CCPA, and HIPAA.

Data masking is important to organizations for the following reasons:

  • Data security: If masked data is breached, the original information it replaced is kept safe and protected.

  • Test data management: Masking data is a safe alternative to using real production data needed for test data management tools.

  • CI/CD: Continuous Integration / Continuous Delivery (CI/CD) in DevOps requires clean and usable data on demand. Dynamic data masking allows DevOps teams to quickly provision, use, and test new applications. 

  • Compliance: As the amount and strictness of data privacy regulations grow, data masking and synthetic data generation play a critical role in data-intensive companies.

  • Customer 360: By masking data associated with Customer 360 use cases, companies gain access to representative and insightful data to enhance customer experiences, while protecting PII.

  • Third-party protection: Enterprises must mask any data that’s processed by, or that integrates with, third-party vendors to preempt any breaches in the supply chain

The different types of data masking include:

  • Data anonymization, which removes or encrypts the personal data found in a dataset.

  • Data pseudonymization, which substitutes PII, such as a name or Social Security Number, with a fake name or figures.

  • Encrypted lookup substitution, which provides a table indicating realistic alternative values to personal data. 

  • Redaction, which replaces personal data with generic values in testing and development environments.

  • Shuffling, which randomly inserts other masked data instead of substituting data with generic values. 

  • Data aging, which applies policies to each data field to conceal the true date. For example, you can set back the dates by 150 or 1,700 days, to maximize concealment.

  • Nulling out, which gives a null value to a data column, making it invisible to unauthorized users.

Data masking best practices call for the substitution of real, sensitive data with fake, yet lifelike, data, in order to maintain its ability to carry out business processes. Masked data can’t be reverse-engineered. Data masking replaces personal or sensitive information with random values, without any way to reveal the original ones. 

Data tokenization obfuscates sensitive data by replacing it with a meaningless token, for use in databases or internal systems. The tokenization of data process secures data at rest, and in motion. If somebody needs the real data value, the token can be “detokenized” back to its original state.

Data masking tools protect personal information by replacing real-life, sensitive data with jumbled, yet statistically equivalent, data. Although information that has undergone data masking can’t be reidentified, it remains functional for non-production environments. By using masked, instead of real-life, data, personal information is protected in the case of a breach.

Synthetic data generation does not obscure sensitive data. Instead, it builds artificial, yet lifelike, datasets, enabling development and testing teams to test new software quickly, while virtually eliminating any risks of non-compliance.

Data encryption and data masking techniques are two separate approaches to data privacy management. 

Data masking replaces real, sensitive data with fake, yet realistic, data. Although masked data can’t be reverse-engineered or identified, it’s still functional for software testing and data science.

Data encryption, which converts plaintext into incomprehensible ciphertext, employs a mathematical algorithm that acts as a cryptographic key. Those with access to the key can view the original, plaintext data.

Unlike masked data, encrypted data is vulnerable to data breaches via hacking or social engineering.