Storage
Mobility
Protection
Analysis and control
TOPICShare
Data migration is a key part of embarking on a digital transformation journey. The data migration process, however, is not without its own set of difficulties.
In a previous post, we looked at some of the major challenges associated with data migration to the cloud and saw how Cloud Data Sense can help by mapping, classifying, and simplifying data for a clean migration.
In this post we'll continue that discussion by looking at how Data Sense can help you better govern your data and accelerate your cloud migration projects.
Use the links below to jump down the sections on:
What Makes Data Migration Difficult
Data migration is a complex, costly, and lengthy process. It starts with identifying the right data sets for migration, determining their business value, the benefits of moving specific data to the cloud, avoiding duplication, etc. Compliance with data regulations also has to be considered and planned around accordingly. We explored the challenges of data migration in this blog. If you haven't read it yet, please do check it out for better context.
Before the migration, it’s important to analyze your data estate for a number of pain points:
- Sensitive private data. This data should either be omitted from migration based on compliance requirements, or additional security measures should be included for an end-to-end secure migration procedure.
- Regular data access patterns. The data that is frequently used should be segregated from infrequently used, or even obsolete data, and targeted for the migration.
- Duplicate data. Data that has been duplicated and holds no business value should be segregated as well, to avoid adding to migration costs.
The common approach adopted by organizations to address these challenges is to use a data management platform to orchestrate the migration process while relying on replication engines for the data migration itself. However, the question of which files to migrate always looms large ahead of the operation and might slow it down considerably.
Imagine manually sorting through all of the files in the organization to assess the migration potential of the full data set. It could take years! But skipping this planning stage can be disastrous.
The Solution
Getting over these migration hurdles requires a way to parse the data on hand before the migration takes place. That needs to happen in a way that’s timely, cost-effective, and easily managed.
NetApp Cloud Data Sense can be used to analyze, map, and classify data so that you can identify exactly which data needs to move to the cloud.
However, while Data Sense is primarily known for its data governance capabilities, it can also be used as a migration tool. Primarily, this is in the search functions that make it possible to locate irrelevant and sensitive data so you can isolate a clean copy of the data that is safe and optimized for the cloud.
Beyond creating clean copies, Data Sense also has built-in capabilities that can be used to initiate the actual migration process and make the entire project much easier.
Data Migration with Data Sense: Tools and Options
In addition to data governance and analysis, Data Sense also assists with the final leg of the process—the actual migration of data from source to destination. This can be done with the help of two proprietary NetApp technologies: FlexClone® data cloning and Cloud Sync® data synchronization. Data Sense also has some native functionality of its own to move data.
Let’s take a look at each of these features below.
Data Migration with FlexClone
For ONTAP-based environments NetApp FlexClone can be used to migrate data to a new volume in the same aggregate. Using Data Sense’s capabilities, you can restrict the migration to include just the relevant data—i.e., cloning the entire volume instantly and then deleting the inverted query using batch delete.
The cloned volume can either be based on an on-premises ONTAP system or Cloud Volumes ONTAP. In the backend, FlexClone uses NetApp Snapshot™ technology to create a point-in-time copy of the original volume that is writable and fully functional. These copies are highly space-efficient: a FlexClone copy shares the same disk space as its parent volume, with additional storage space only consumed for delta data. That makes the process instantaneous, and space-efficient.
Using Data Sense, this FlexClone operation can be triggered from the data investigation results pane. Any items not included in the query used for data analysis will be deleted from the clone volume without impacting the original volume. This cleaned-up data volume can then be used as a data set for migration.
SnapMirror™ data replication technology can be used to move this clone data set to a target volume. The whole process can be initiated from Cloud Manager using the GUI’s drag-and-drop functionality. A FlexClone volume or its parent can be used as the source volume for creating this SnapMirror relationship.
Once the data is migrated to the new volume, this data can also either be automatically or manually included in the scope of further Data Sense scanning.
Moving Data with Cloud Sync
NetApp Cloud Sync can be used in the migration process by users with non-ONTAP environments that are scanned by Data Sense. Cloud Sync can be used for the initial copy of data followed by continuous sync of any data changed in the source environment. Using Cloud Sync, you can copy a maximum of 200,000 files at a time this way.
All files selected for migration should be part of the same data source. Data filters can be used to select only specific types of files for synchronization. The sync process can be initiated from the Data Sense’s Data Investigation pane, which will open up the Cloud Sync UI. The source data will be what you have selected in Data Sense with the applied filters. The destination for data migration includes cloud storage services such as Amazon S3, Azure Blob storage, SMB/NFS servers, Cloud Volumes Service, Cloud Volumes ONTAP, Amazon EFS, and more.
For data migration purposes, a one-time data sync can be configured in Cloud Sync, meaning the selected files with the appropriate filters are copied over to the destination. Periodic data syncs between the source and destination can also be scheduled.
It's worth noting that if the source system adds additional files matching the query used for creating the filters, those files will also be automatically copied and synchronized. This helps ensure that only the required data is migrated to the target environment adhering to the controls that you define.
Watch a demo by Yossi Ashkenazi, Cloud Solution Architect, presenting how you can move data with Data Sense and CloudSync.
Moving and Copying Data with Data Sense
Data Sense has some other capabilities to move data as well. The source files Data Sense scans can be moved to a target NFS file share directly from the Data Investigations results tab in Data Sense. These migrations can be carried out only by users with an account admin or workspace admin role in Data Sense.
Data Sense can also copy data sources it scans to destination NFS shares, either from the same or different scanned volume/file share.
Conclusion
It's critical for every organization moving to the cloud to analyze and segregate data before the migration process. Cloud Data Sense helps every step of the way—starting with mapping, classifying, and governing data both before and after the migration.
You can leverage the power of Cloud Data sense for the clean migration of data. Prior to migration, the data can be filtered using a wide variety of parameters so that only required and compliant data is included in the migration data set.
To read more about creating clean data copies, check out this clean data migration checklist.
To get started with Data Sense today, sign up for a free 1TB trial of Cloud Data Sense here.
