Storage
Mobility
Protection
Analysis and control
TOPICShare
Data migration is an inherent part of virtually any digital transformation project. But it's also a highly risky process that involves an almost overwhelming amount of work.
Given the sheer scale and complexity of enterprise IT systems, you'll have a whole host of different things to consider. You’ll also have any number of different things that can go wrong.
However, you can do much to mitigate the risks involved in data migration by putting robust planning, preparation, and testing measures in place. This post will take a look at the key steps you need to take in preparing a data migration plan which will be crucial for a successful migration and serve as a roadmap to smooth transition of your data to its new location.
This is part of an extensive series of guides about data security.
Jump down using the links below to read the sections on:
- Create a Data Migration Plan of Action
- Understand Your Data
- Clean Up Your Data
- Test the Data Migration Process
- Take Backups
- Perform Post-Migration Checks
- Data Migration Tools for the Planning Stage
Create a Data Migration Plan of Action
Just as with any other complex undertaking, planning a data migration is essential. While each organization’s migration project is unique, there is a lot of common ground. Much of this depends on the type of migration you’re looking to accomplish.
You may be moving data to the cloud or to new on-premises servers. You may be transferring blocks and files or performing a database migration. Or you might be moving your data and rearchitecting your applications at the same time.
Whatever your migration use case, you'll need to consider the following:
- Which data needs to move to the cloud?
- What applications process or depend on that data?
- Where is it located and how can you integrate data from multiple sources?
People will also be an important part of the planning process. For example, different departments often use different applications to access the same data. So it's important to involve those affected by the migration and keep them informed of progress throughout the project. What's more, you'll need to select people with the right expertise to oversee different aspects of the operation.
And, at some stage, you'll also need to draw up a checklist of technical provisions, such as:
- Specifications for any data conversion
- End-to-end encryption of data in transit over open networks
- Resource requirements, such as storage capacity
It's also worth evaluating your choice of data transfer technology, taking into account features such as error-checking capabilities to help prevent data loss during the migration. This plan will be essential as you progress through all the data migration steps.
Understand Your Data
A data-profiling exercise will help you understand your source data so you can make informed decisions on what files and records to move.
With the right tools, you'll be able to quickly determine data ownership. You'll be able to ensure your data is in good shape before you move it to its new location by identifying any consistency, security, or compliance issues ahead of time. You may also potentially discover data you didn't even know you were storing.
In addition, data profiling is important to understanding the structure of your data and whether it's currently being mapped to the right places. It can also provide insights that serve as a reference point for your post-migration checks.
Clean Up Your Data
Once you have a clear overview of what's in your data, you should prepare a clean copy that's purged of anything unsuitable for migration. This is especially important if you're migrating to the public cloud.
For example, you may need to keep personal data on premises in order to meet the data residency requirements of applicable privacy regulations. This is most likely to affect you if you operate in a highly regulated sector, such as finance or healthcare, or, depending on the regulations, if your data will reside in a cloud region outside your own country.
What's more, the public cloud is on-demand pay-as-you-go (PAYGO) infrastructure. So you'll need to make efficient use of storage to avoid racking up unnecessary charges on your monthly cloud bill.
First, you should identify and remove duplicate data. This will not only help keep costs down but also improve data quality, leading to better analytics and decision making accordingly.
Secondly, subject to any business or regulatory requirements, you should look to remove stale data that hasn't been accessed for a specific period of time.
In addition to cost savings and better data governance, good data hygiene will reduce the amount of data you need to migrate. This will speed up the entire process of transferring the data and reduce the scope for any potential issues.
Read more about creating clean data copies for migration here.
Test the Data Migration Process
Start by making sure you have a comprehensive and well-defined testing strategy in place. That way, you'll know what to look for when checking the results of your test migrations. Moreover, it will help significantly reduce the potential for trouble further down the line.
Testing can uncover a wide range of problems you may need to address—from incorrect data mapping and data loss to cross-object dependency issues.
Data migration undertakings that form part of a wider application modernization project will be particularly complex and require far more rigorous testing. But you can still run into problems with a relatively simple lift-and-shift migration. For example, you can encounter conflicts between operating systems, which could render data unreadable or inaccessible or result in access control issues.
The larger the dataset you can use for your tests, the better. And, where possible, involve end users in your testing regime. They're the ones who work with your applications and data. So they're better equipped than your technical teams to spot certain types of problems.
Take Backups
Always take a full backup immediately before you start the migration. In addition to your source systems, you should back up any other data that could be potentially affected by the switchover.
This is because, even if the data in your source environment remains unaltered, the migration process may need to modify information in other systems. So, if something goes wrong during the migration, you may not be able to put things right by simply falling back on your original system. Your backup copy will be a crucial fallback here.
Perform Post-Migration Checks
The period immediately after you've completed the switchover is the most critical in the entire migration process. You'll need to perform follow-up checks as quickly as possible to minimize downtime or resolve any unforeseen problems that could have an adverse impact on your business.
So be prepared by making sure you have people at the ready to check your new system is in full working order. Just as with the pre-migration phase, you should follow a robust testing strategy. This should also include measures you can take when things go wrong.
And don't forget that post-migration checks shouldn't just cover data integrity and application functionality, but also security. For example, you should ensure that:
- File permissions have been copied over correctly to your new location
- All sensitive data has been encrypted
- Appropriate security tools are up and running in your new environment
Data Migration Tools for the Planning Stage
As you can tell from the list above, data management tools will play a key role in the success of your data migration project. They can help you understand your data. They can provide an essential layer of backup protection when something unexpected happens. And they can help you identify and remove unnecessary or problematic data so you can keep your storage costs down.
NetApp has a number of migration solutions, but for your pre-migration planning and data mapping stage, the most critical one to have at your disposal will be Cloud Data Sense.
Data Sense can scan, locate, map, and report on all the data types that you have stored so you can determine which data needs to move to the cloud, which data is redundant, and which needs a higher level of protection.
FAQs
What does “migration plan” mean?
A migration plan is a document that includes all of the pre- and post-migration tasks that you should undertake in order to carry out a migration to the cloud. This document should be created with the input of your entire IT team as well as business leaders.
What should a data migration plan include?
A data migration plan should include all of the steps that you need to take prior to the migration, including mapping all of the data that you own to decide what belongs in the cloud and what doesn’t, creating backups to ensure data and application integrity if anything goes wrong, testing the new environment post-migration, and more.
How do you plan data for a migration project?
Creating a clean copy of your data is the best way to plan around which data needs to move to the cloud and which data doesn’t. Creating such a clean copy entails doing a full audit of all the data that you plan to migrate, across any of the systems where that data resides. Since this can be a considerable task, automation will be a key factor here.
See Additional Guides on Key Data Security Topics
Together with our content partners, we have authored in-depth guides on several other topics that can also be useful as you explore the world of data security.
AWS SQL
Authored by NetApp
- SQL Server in AWS: Two Deployment Options
- AWS PostgreSQL: Managed or Self-Managed?
- AWS NoSQL: Choosing the Best Option for You
Ransomware Protection
Authored by Cynet
- 6 Ransomware Protection Strategies You Must Know
- Ransomware Removal: Recovering Your Files and System Cleanup
- Ransomware Detection: Common Signs and 3 Detection Techniques
Incident Response
Authored by cynet
- What Is Incident Response?
- What Is a SOC? 10 Core Functions and 6 Key Challenges
- Incident Response SANS: The 6 Steps in Depth
