After managing many complex digital transformations and migrations, for some of the world’s largest companies, I’d love to share some key cloud migration considerations with you – as I see them.
Table of Contents
Credentials of a Data Pro
I’ve been working in the data management business for more than 30 years, primarily with large enterprises in the United States. During that time, I’ve participated in, and/or managed, 16 different data migrations in massive-scale enterprise operations, in a number of industries.
Insights from the Field
I’ve learned a lot from every enterprise implementation I’ve been involved with. Below is a list of some of the most important cloud migration considerations, along with my recommendations. I hope you’ll find them useful!
1. Accounting for multiple source systems without all the knowledge
In large enterprises, customer data is fragmented, and sourced from many different big data stores. Some systems are so outdated, that no one in the company really knows how their data sources are structured and organized.
Make sure your cloud migration toolkit includes the following:
Data catalog, to collect the metadata related to data formats, semantics, and quality across all systems (including legacy), so you can make informed decision-making related to mapping and transformation logic.
Auto-discovery, to automatically discover and visualize the meta data structure, and the relationships between business entities.
Ready-made connectors to the most common source and target technologies.
A data catalog automatically discovers and visualizes the meta data structure, and the relationships between its entities, for all of data assets.
2. Adopting a phased approach to assure business continuity
Dividing the migration into a series of phases can reduce complexity and risk. However, a phased migration also might increase cost and complexity because both legacy and cloud environments must run in parallel, until all phases are complete.
Migrate only the data subsets you need, when you need them. For example, an enterprise with a million customers would be able to migrate the data of one customer segment at a time. Often these are segmented into customers with specific products, in a particular geographical area, consumer, SMB, or enterprise.
If retiring legacy systems to cut costs is your priority, migration can also be performed by customers in a particular system. These customers can then be subjected to one or more of the segmentations within a particular system – by product, geographical area, or market segment.
Select a system that enables you to change your migration strategy – all the way from flash card, to phase, to on-demand – without delay.
3. Focusing on data quality
Bad data causes migration projects to take up to twice as long as originally planned. That’s because there are often serious quality issues in legacy data, and extra time and resources aren’t allocated to identify and address them. Imperfect migrations may result in incorrect bills, or irrelevant correspondence being sent to end-users, impacting customer satisfaction.
Build a data model centered around the business entities you wish to migrate, and then migrate that data according to those entities. This modeling ensures the completeness and integrity of the migrated data.
In addition, automatically test converted data for integrity and make sure that only 100% fully-tested data is migrated. Ensure your migration solution is geared towards identifying and handling fallout from data mismatches.
Choose a tool that can not only identify imperfect data, but also be able to correct it, in-flight.
4. Minimizing the scope to cut costs
Not all legacy data needs to be migrated. By reducing the migration’s scope – whether by deleting redundant data sources, updating outdated business processes, or archiving data that no longer has value – the scale and complexity of the project can be significantly scaled down, reducing risk, time, and cost.
Identify only the relevant data, while discarding redundant, old, and “look-alike” information. Don’t migrate everything, just the data required to meet business goals and legal requirements. The rest can be archived. Evaluate if you still need the application, even if you want to keep the data. Turning off applications and keeping the data helps the business case. In short, liberate the data.
5. Defining and tracking metrics
The ability to measure and track the progress of data migration efforts is crucial for informed decisions and process improvement. This requires metrics of the quality of the data being migrated, as well as updates on the pace, progress, and performance of the overall cloud migration effort.
Deliver metrics on data quality aspects, determine whether quality-level expectations are being met, and show the status of the data workstream (e.g., # data instances migrated, # iterations, time and resources spent on execution and testing, etc.).
A dashboard showing cloud migration results allows for informed decision-making.
6. Exceeding parity
It is amazing how many digital transformations boil down to doing the same thing in the cloud. If we have learned anything over the pandemic, it is that we can change rapidly if we want to. Why move all the problems to the cloud? Sometimes, the functionality of the new cloud system is initially less than that of the old one. New functionality is incrementally added over time to bridge the gap.
Rethink the “that’s the way we do things” mindset:
Plan that the new system and approaches to development be centered on, and integrated with, real-time data pipelines. This will completely change the way the organization operates.
Uncouple the data that does not conform to the new system’s data model – and store it separately. Don’t overextend the new system’s data model to reflect the legacy model.
Account for “bridging”, which allows for legacy systems to serve the missing functionality, alongside the new cloud environment.
Cloud Migration Considerations Summary
Although my 32-year career in data migration spanned a number of employers, products, countries and services, I’ve had the privilege of working at K2View, a trail-blazing company for the past decade. Where traditional data migration strategies called for a few days of downtime (usually planned for long weekends), K2View has an “on-the-fly” approach that does not affect the business in any way. Today, K2View is leading the way to smooth cloud migrations for the world’s largest enterprises.