More about Cloud Migration
- Azure StorSimple EOL: Using Cloud Volumes ONTAP for Hybrid Cloud Management
- Refactoring Applications to Kubernetes in Cloud Migrations
- The War Over the Cloud Has Ended (and the Winner Might Surprise You)
- Modern Data Estate: What IT Leaders Need To Know
- Strategies for AWS Migration: The New 7th R Explained
- Cloud Computing Deployment Models and Architectures
- Cloud Adoption Strategy: What’s the Best Approach for Your Organization?
- What is Cloud Migration? Strategy, Process and Tools
- Hybrid Cloud Strategy: A Winning Model for Enterprises
- Google Cloud Architecture and Building Your Own Solution Architecture
- SnapMirror in the Cloud: New Use Cases for NetApp’s Data Replication Technology
- 8 Digital Transformation Technologies and Their Business Impact
- What Is Digital Transformation in Banking?
- Digital Transformation in Healthcare: 4 Key Trends
- Digital Transformation: Examples from 5 Industries
- The Future of Cloud Computing: 5 Trends You Must Know About
- 5 Types of Digital Transformation and the Technologies that Power Them
- Digital Transformation Strategy: 6 Tips for Success
- Kubernetes Data Mobility with Cloud Volumes ONTAP
- Application Migration to Azure: 4 Approaches and One Migration Tool
- Cloud First Strategy: Challenges, Considerations, and Best Practices
- Application Migration to AWS: Free Tools to Ease Your Migration
- Why Cloud Adoption Fails and 6 Tips for Success
- Cloud Application Migration: A Practical Guide
- AWS Snowball vs Snowmobile: Data -Migration Options Compared
- AWS Snowball Edge: Data Shipping and Compute at the Edge
- AWS Snowmobile: Migrate Data to the Cloud With the World’s Biggest Hard Disk
- AWS Snowball Family: Options, Process, and Best Practices
- AWS Cloud Adoption Framework (CAF): 6 Migration Perspectives
- Top 3 Cloud Adoption Frameworks: Your Path To The Cloud
- AWS Snowball Pricing Simplified
- Azure Cloud Adoption Framework: The 9 Methodologies Explained
- Cloud Roadmap: Mapping Out Your Path To The Cloud
- Hybrid Deployment on Google Cloud: Meet Google Anthos
- 3 Ways to Create an Azure Migrate Project
- Azure Migration Step by Step: Discover, Migrate, Optimize, and Monitor
- Migrate from VMware to Azure: The Basics and a Quick Tutorial
- Migrate SQL Server to Azure: Options, Tools, and a Quick Tutorial
- Migrate Databases to Azure: 3 Quick Tutorials
- 4 Ways to Migrate SQL to Azure
- Azure Migration Program: 4 Key Elements
- Azure Migrate: Key Components and a 4-Step Migration Plan
- Cloud Journey: 6 Stages of Cloud Adoption
- 5 Azure Data Migration Tools You Should Be Using
- Azure Migration Tools: One-Click Migration for VMs and Data
- Typical Mistakes and Misconceptions Business Leaders Have About Hybrid and Multicloud
- Google Cloud PostgreSQL: Managed or Self-Managed?
- AWS Storage Gateway: Connecting Your On-Premise Storage to the Amazon Cloud
- Azure PostgreSQL: Managed or Self-Managed?
- Cloud Scalability: How Cloud Volumes ONTAP Stores Petabytes of Data
- AWS Migration: Understanding the Process and Solving 5 Key Challenges
- Google Cloud Pricing vs AWS: A Fair Comparison?
- VMware Cloud Services: A New Option for Hybrid Cloud Management
- AWS PostgreSQL: Managed or Self-Managed?
- Google Cloud MySQL: MySQL as a Service vs. Self Managed in the Cloud
- Azure MySQL: MySQL as a Service vs. Self-Managed in the Cloud
- AWS MySQL: MySQL as a Service vs. Self Managed in the Cloud
- Cloud Adoption for Financial Companies
- VMware on AWS: Architecture and Service Options
- VMware on Google Cloud: A Deployment Roadmap
- VMware on Azure: One-Step Migration to the Cloud
- Better in the Cloud: Workloads Gartner Says You Should Move to the Cloud Now
- 3 Cloud Migration Approaches and Their Pros and Cons
- Oracle on Google Cloud: Two Deployment Options
- How to Upload Files to Azure Blob Storage with AzCopy, PowerShell, and More
- Oracle on Azure: Fully Managed vs Self Managed
- Azure SQL Server: Managed Service vs Self Managed
- Google Partners with NetApp on New Bare Metal Solution in the Cloud
- AWS Database Migration Service: Copy-Paste Your Database to Amazon
- SQL Server on Google Cloud: Two Deployment Options
- GCP Migration with Cloud Volumes ONTAP
- SQL Server in AWS: Two Deployment Options
- Azure Database Migration Service: Automate Your Cloud DB Migration
- Azure Managed Service Provider: How to Save Time and Reduce Cloud Overhead
- Hybrid Cloud Storage: The Best of Both Worlds
- Hybrid Cloud Storage Solutions for AWS Compared: Storage Gateway vs Cloud Volumes ONTAP
- AWS Oracle: Two Options for Migrating Your Oracle DB to Amazon
- Migrating Physical Server to AWS: Now Free with AWS CloudEndure
- Azure Hybrid Cloud:
Azure in Your Local Data Center - What Is a Lift and Shift Cloud Migration?
- Google Cloud Migration Tools: Copying 1GB or 500TB? Learn How
- VMC on AWS Vs. Cloud Volumes ONTAP
- AWS Cloud Migration Services: Don’t Migrate Alone
- 5 Steps to the Cloud: AWS Migration Checklist
- Setting up Storage Gateway with Amazon EC2
- VMware Cloud on AWS: How Fujitsu Saves Millions Using Cloud Volumes ONTAP
- SQL Server End of Life Challenges and How Cloud Volumes ONTAP Can Help
- The Cloud Tiering Service Architecture: How We Get Cold Data from Your Data Center to the Cloud
- Cloud Data Integration 101: Benefits, Challenges, and Tools
- Google Cloud Migration: Why and How to Make the Move
- What’s the Easiest Way to Start Using the Cloud? Three Cloud Onboarding Approaches
- AWS vs Azure vs Google Cloud: Choosing the Best Cloud Provider for You
- Cloud Migration Tools: Transferring Your Data with Ease
- Azure Migration Strategy: Four Steps to the Cloud
- 11-Step Azure Migration Checklist
- AWS Migration Strategy: The 6 Rs in Depth
- Officeworks Adopts a Cloud-First Strategy with Cloud Volumes ONTAP
- Azure Migration: The Keys to a Successful Enterprise Migration to Azure
- Cloud Volumes ONTAP: Cloud Migration Case Studies
- One Cloud Out of Many: Why Enterprises Are Turning to Multicloud and Hybrid Cloud Architectures
July 8, 2020
Topics: Cloud Volumes ONTAP Data MigrationAWSDatabaseElementary8 minute read
AWS Database Migration Service (DMS) automates the migration process to AWS database services. You can use DMS to migrate a wide range of data types, from any data source, to the AWS cloud. The process works by first connecting DMS to your source database. DMS then reads the data, prepares it for compatibility with the target database, and then transfers the data according to predefined migration tasks.
AWS DMS offers many more automated features; however, it is not a fully-automated service. You do need to set up the process, and that requires an understanding of DMS components and processes. In this post, we’ll explain AWS DMS benefits, pricing, and components. We’ll also show how NetApp Cloud Volumes ONTAP can help simplify the migration process and address database workload challenges in the cloud.
In this article, you will learn:
- What is AWS Database Migration Service
- AWS DMS benefits
- AWS DMS pricing
- How AWS Database Migration Service works
- Components used in an AWS migration with DMS
- AWS Database Migration Service with Cloud Volumes ONTAP
What Is AWS DMS?
AWS Database Migration Service (DMS) is a service you can use to migrate from any database to AWS. You can use this service for both homogeneous (x to x) or heterogeneous (x to y) and on or offline migrations. DMS supports migration to Amazon RDS, Aurora, RedShift, DynamoDB, and DocumentDB.
To use the Database Migration Service you do not need to install agents or alter your source database (in most cases). Instead the migration process is controlled from the AWS Management Console. Once started, migration is a managed service that is resilient and self-healing.
Learn more about AWS migration in our guide: 5 Steps to the Cloud: AWS Migration Checklist. Or read about other AWS Cloud Migration Services.
AWS DMS Benefits
There are several benefits of using AWS Database Migration Service over traditional migration methods. These include:
- Minimal downtime—with DMS changes to your source database are continuously replicated. This occurs while your source database remains operational. Then once AWS migration is complete, you can switch over at any time without shutting down either database.
- Low cost—DMS is offered as a free service for migrations to Aurora, Redshift, DynamoDB, or DocumentDB. For other database services, AWS DMS pricing is based on the amount of log storage you require, and the computational power needed to transfer. Bandwidth to AWS is free.
- Reliability—DMS includes continuous monitoring of your target and source databases, replication instances, and network connectivity. The service is also self-healing and will automatically restart if an interruption occurs. You also have the option of setting up a Multi-AZ (availability zone) replication for disaster recovery.
AWS DMS Pricing
As mentioned, one of the benefits of AWS Database Migration Service is that it is available for free for specific services. This access is provided for the first six months which should be more than enough to get you through a standard migration. After that, you are charged the same as when using DMS with one of the not listed services.
If you are past the six-month period, or are using a different service pricing is as follows:
|
Single AZ |
Multi AZ |
Instance costs |
From $0.018 — $3.30 per hour |
From $0.036 — $6.60 per hour |
Storage costs |
$0.115 per GB per mth |
$0.23 per GB per mth |
Data transfer |
Free for all data transferred into DMS and any data transferred between DMS and RDS or EC2 instances in the same AZ.
If you migrate data to a different AZ or region, or migrate outside of AWS pricing starts at $0.0047 per hour. |
How the AWS Database Migration Service Works
When using AWS Database Migration Service, you connect the service to your source database. The service then reads the data, formats it to match the requirements of your target database, and transfers the data according to the migration task you have defined. These processes occur primarily in-memory for greater performance. However, large transfers, logs, and cached transactions require disk storage or buffering.
Once replication from source to target begins, DMS begins monitoring your database for changes. This is done with Change Data Capture processes, known as AWS DMS CDC. CDC ensures that your data remains consistent between databases during the transfer process. It does this by caching changes made during migration and processing the changes once the primary migration task is over.
After the initial transfer is complete, Database Migration Service works through any cached items, applying changes to the target as needed. The service continues to monitor for and cache changes during this time and until you transfer your workloads and shut down your source database.
Components Used in an AWS Migration With DMS
When operating Database Migration Service there are several components that you should be familiar with.
Amazon DMS Components
Replication instance
A replication instance is a managed EC2 instance that is used to host replication tasks. There are multiple replication instance types you can choose from, including:
- T2/T3—designed for developing, configuring, and testing database migration processes. These instances are also suitable for periodic migration tasks.
- C4—performance-optimized instances designed for compute-intensive workloads. These instances are well suited for heterogeneous
- R4/R5—memory-optimized instances. These instances are suitable for ongoing migrations and high-throughput transactions.
Endpoints
To transfer data, AWS Database Migration Service uses an endpoint to connect to your target and source databases. The type of endpoint varies according to your database type, but all endpoints generally require the same information. This includes endpoint type, engine type, server name, port number, encryption protocols, and credentials.
Database replication tasks
Replication tasks define what data is moved between your target and source database and when. When creating a replication task, you need to define which replication instance to use, your target and source endpoints, and your migration type option.
The table below explains the migration options available to you.
Full load |
This option migrates the data that currently exists in your database when the task is started. It does not account for changes made after data has been replicated.
This is a good option when you can pause workloads for migration or if you are okay with not capturing changes. For example, if you are creating a test migration. |
Full load + CDC |
This option migrates data at the time of task start and caches data changes for later replication. Once changes are replicated, this option continues to monitor your database until the task is ended.
This is a good option when you have large databases to transfer and cannot afford to or simply don’t want to pause workloads. Using this option you can migrate your database well in advance of when you shift your workloads over, giving you greater flexibility in your migration. |
CDC only |
This option only migrates changes made to the database since the task was started. It does not perform any initial transfer of data.
This option is useful when you are using another method to transfer databases but still need to capture changes to keep databases synced. For example, if you can start a CDC only task immediately after exporting a database copy. You can then import your copy into your target database while collecting changes with the DMS task. |
Schema migration and conversion
When performing a homogenous migration, Database Migration Service attempts to create a target schema. However, this isn’t always possible. For example, if you are migrating an Oracle database, a target schema isn’t created to ensure security. In these cases, you’ll need to use tools such as Oracle SQL Developer, pgAdmin III, or MySQL Workbench.
For heterogeneous migrations, you also need to take additional steps because AWS Database Migration Service is not able to perform schema conversions. For this conversion, you can use a third-party tool, or you can use AWS Schema Conversion Tool (SCT).
SCT enables you to automatically convert your source schema to a compatible format for your target database. This includes the conversion of most code objects, including stored procedures, functions, and views. It is also able to scan your database applications for legacy SQL Server or Oracle functions and update functions to an equivalent AWS service. If the tool is unable to convert objects, it marks each for you to manually handle.
AWS Database Migration with Cloud Volumes ONTAP
NetApp Cloud Volumes ONTAP, the leading enterprise-grade storage management solution, delivers secure, proven storage management services on AWS, Azure and Google Cloud. Cloud Volumes ONTAP supports up to a capacity of 368TB, and supports various use cases such as file services, databases, DevOps or any other enterprise workload, with a strong set of features including high availability, data protection, storage efficiencies, Kubernetes integration, and more.
In particular, Cloud Volumes ONTAP helps in addressing database workloads challenges in the cloud, and filling the gap between your cloud-based database capabilities and the public cloud resources it runs on. You can leverage Cloud Volumes ONTAP features to ensure your data remains available and secure as you migrate it to AWS.
