Storage
Mobility
Protection
Analysis and control
TOPICShare
More about Data Management
Subscribe to our blog
Thanks for subscribing to the blog.
July 30, 2021
Topics: Cloud Data Sense Data MigrationElementary6 minute read
What is a Data Migration Strategy?
Data migration is the process of copying or moving data from one device or system to another. Migration must be designed in such a way that it doesn’t interrupt or disable business operations. After the data is transferred, the business process transitions to the new device or system.
For any company, data migration is essential to making better business decisions. It allows organizations to expand their data storage, data management and analytics capabilities, by introducing new systems and processes. Data migration is highly strategic and a large part of digital transformation efforts—according to an IDC study, 60% of the workload in large enterprise projects can be attributed to data migration.
In this article, you will learn:
Data Migration Goals
The goals of an effective data migration strategy include:
- Ensuring data is completely and accurately migrated from the source platform to the target platform in accordance with company policies and relevant compliance standards. This means there are no records in the target environment that are missing, incomplete, or failed some form of validation.
- Get the target system running as quickly as possible with a minimal amount of downtime and disruption to business operations.
- Minimize costs of migration in terms of technical and manpower requirements.
Types of Data Migration Strategy
There are three main types of data migration: “big bang” in which systems are migrated in one go, “trickle” in which data is transferred gradually, and synchronization, in which the source and destination systems continue to live side by side.
Big Bang Migration
In this type of migration, a complete data transfer is performed within a limited time. As soon as data is transferred to the new database or storage system, the old system goes down. This process is a one-time event and can be completed relatively quickly, if properly planned.
Big bang migration has an obvious advantage—it is completed in the shortest possible time—but it carries significant risk.
When migrating from one system to another, critical business functions are inevitably interrupted. Few companies can last for a long time when core systems are not running. This means the migration process happens under tremendous pressure, with very little room for error.
To succeed in a big bang migration, it is recommended that companies perform at least one pilot implementation of the migration process, and develop a detailed emergency response plan prior to field activities.
Trickle Migration
This type of migration can be completed in smaller steps, and take a longer period of time. Old and new systems run in parallel during data transfer, eliminating downtime and reducing the risk of the big bang approach.
However, there are still risks—being able to track the migrated data is a complex process. Trickle migration also means that the users have to switch between the two systems, which may lead to inconsistencies.
Synchronization Strategy
In many modern migration projects, organizations use a synchronization architecture to ensure source and target systems contain the same data. This can mitigate most of the risks of big bang or trickle migration.
There are two types of synchronization:
- One-way—synchronizes new changes received on the target system with the old system, or alternatively, synchronizes from the old to the new system. This makes it possible to decide on one of the systems—old or new—as the new target for all business processes, and keep the other system active in the background. The organization can switch users to the new system while keeping the old one as fallback, or keep users on the old system, while developing and testing the new one with fresh data.
- Two-way—any change in the old system is reflected to the new system, and vice versa. This solution is usually implemented if you need to do incremental migration, while running both systems in parallel. For example, the legacy system may continue to perform certain activities with integrated systems. Records created by users in the new system are synchronized back to the old system to enable these legacy operations. This provides the most flexibility, but is much more complex to achieve.
4 Key Phases in a Data Migration Strategy
Most data migration projects will include the following steps: assessing source data, designing the migration, building a migration solution, and monitoring the migration once in progress.
Assessing the Source
Before starting the migration project, it is important to understand:
- How much data will be moved.
- Which data is being migrated and which data will be left behind or moved to data archiving
- Data formats, types, fields and structure.
- Field mapping—how many of the source data fields will be mapped to the target system.
- Missing data fields that need to be computed or filled from another data source.
- Data quality—run an audit on the data, and if you find inaccuracies, data corruption, or other issues, migration should be reconsidered.
Designing the Migration
In the design phase, you choose the type of migration (learn about migration types above ), and define the exact migration process. Think about how data will be pulled from the source and transferred to the target system, define timelines, risks and dependencies. Clearly document your migration plan.
It is important to consider your data security plan as part of migration design. Identify data that needs to be protected, and ensure you adhere to security policies through the migration process.
Build the Migration Solution
Because migration is typically a one-time project, and is critical to the business, it is important to implement it correctly. A common strategy is to subdivide the data, create a technical process to transfer one category at a time, and then test it. Building the migration solution incrementally can dramatically reduce risks and help you catch problems early on.
To reduce risk, use the trickle approach—deploy the migration solution for each category of data, as opposed to deploying the entire solution at the end.
Migration Monitoring
Both source and destination environments must be carefully monitored before, during and after migration. The following are key considerations for migration monitoring:
- Monitoring tools—what technology will you use to monitor the data source, the destination environment, and the interim data flows?
- Monitoring procedure—how often will monitoring take place, how will you track status, and what will reporting look like?
- Troubleshooting—what is the process when problems are discovered in migration? Who is responsible for investigating and fixing technical issues?
- Performance—what is the desired performance in the target environment? How do you know if the new system is performing under par?
A solid monitoring strategy will ensure you can catch any migration problems early and remediate them before they cause damage to critical business processes.
Hybrid Data Discovery, Mapping and Classification with NetApp Cloud Data Sense
NetApp Cloud Data Sense automatically discovers, maps, and classifies your data wherever it may be. Data availability, ownership and quality are crucial for business efficiency and cost optimization. With Cloud Data Sense, you can automatically label and act on information stored in files and database entries on premise and in the cloud. Make smart data decisions and automate your data optimization and compliance plans.