Of Amazon’s 212 cloud computing services, fifteen are purpose-built database engines. In this post, we focus on eight AWS database services: Amazon RDS, Amazon Aurora, Amazon DynamoDB, Amazon DocumentDB, Amazon ElastiCache, Amazon Neptune, Amazon Timestream, and Amazon Quantum Ledger Database (QLDB).
We offer a review of the functions, feature, capabilities and use cases of each of these services, to help you choose the database that is a perfect match for your use case and project. In addition, we’ll show how NetApp Cloud Volumes ONTAP can help solve AWS database migration and management challenges.
This is part of an extensive series of guides about managed services.
In this article, you will learn:
AWS offers a wide range of database services for you to choose from. The service fall into two groups: relational and non-relational (NoSQL).
In this section we provide a breakdown of Amazon database services into relational and non-relational, in the AWS Database Services section below you can learn about selected services in more detail.
Relational databases store data in tabular form with columns and rows, and can be queried using the SQL query language. In these databases, columns represent attributes and rows represent records. Each field in the table is a data value.
Use cases for relational databases on Amazon include:
The primary Amazon services providing relational databases are:
Relational databases are not suitable for many use cases, especially those requiring very high performance or dynamic scalability. NoSQL, or non-relational databases, break the paradigm of storing data in tables with columns and rows, allowing them to distribute and process data more efficiently. NoSQL is commonly used to handle big data - large volumes of unstructured or semi-structured data.
The table below shows the main NoSQL databases services offered by AWS.
|
Type of Database |
Use Cases |
Amazon Services |
|
Key-value Key-value databases store data as a collection of key-value pairs with the key as an ID. These databases can store various types of data, including simple and compound objects. |
● Real-time bidding ● eCommerce shopping carts ● Product catalogs ● Customer preferences |
● Amazon DynamoDB |
|
Document Document databases store data in JSON or JSON-like documents. You can query data using the same document-model format used in programming applications. |
● Cataloging ● Content management systems ● Customer profiles and personalization ● Mobile apps |
● Amazon DocumentDB |
|
In-memory In-memory databases store data in-memory for low-latency access. You can use these stores as a database, cache, message broker, or queue. |
● Caching ● Session stores ● Gaming ● Leaderboards ● Geospatial services ● Pub/sub messaging ● Real-time streaming |
● Amazon ElastiCache for Memcached ● Amazon ElastiCache for Redis |
|
Graph Graph databases are a type of NoSQL (non-relational) database. This database type represents relationships directly. You can query data with specific graph languages. |
● Fraud detection ● Social networking ● Recommendation engines ● Knowledge graphs ● Data lineage |
● Amazon Neptune |
|
Time-series Time-series databases store data in time-order and as append-only. You can query data over various time intervals. |
● DevOps ● Application monitoring ● Industrial telemetry ● IoT applications |
● Amazon Timestream |
|
Ledger Ledger databases store data in an immutable, transparent, and cryptographically verifiable log. This log is owned by a trusted central authority to ensure provenance. |
● Finance ● Manufacturing ● Insurance claims ● HR and payroll ● Retail inventories |
●Amazon Quantum Ledger Database |
Note: While we listed in-memory databases under the non-relational category, there are in-memory databases that are relational, SQL databases. A notable example is SAP HANA. You can run SAP HANA on AWS using the managed SAP HANA instance.
Once you understand what options are available to you for databases in AWS, you can begin narrowing them down. The following services are some of the most commonly used. Keep in mind when reviewing these descriptions that frequently, AWS customers implement multiple database types to meet their needs. You too should consider multiple options if one doesn’t meet all of your needs.
Amazon RDS is a managed, relational database service that includes six different database options. These include AWS Oracle, PostgreSQL, AWS MySQL, MariaDB, SQL Server, and Amazon Aurora. You can manage these database engines from a centralized management console, a command-line interface, or via API calls. When using this service, many administrative tasks are automated, including database setup, hardware provisioning, backup, and updating.
Use cases of Amazon RDS include:
You can learn more in our articles about RDS instance size and AWS Oracle RDS.
Amazon Aurora is a fully managed relational database engine designed specifically for AWS. It is MySQL and PostgreSQL compatible with minor changes to your source database. Aurora includes features for self-healing, fault tolerance, point-in-time recovery, and continuous backup.
Use cases for Amazon Aurora include:
Amazon DynamoDB is a fully managed, document and key-value database. It includes features for multi-master, multi-region used along with built-in security, automated backup and restoration, and in-memory caching. DynamoDB can provide support for serverless web apps, microservices, and mobile backends.
Use cases of Amazon DynamoDB include:
Amazon DocumentDB is a fully managed document database service. It is scalable, highly-available, and compatible with MongoDB. With it, you can store, index, and query JSON files. With DocumentDB, you can scale your compute and storage resources separately for maximum flexibility.
Use cases of Amazon DocumentDB include:
Amazon ElastiCache is a fully managed, in-memory data store service. It is compatible with both Redis and Memcached. ElastiCache automates setup, hardware provisioning, configuration, monitoring, updates, and backup and recovery processes. With ElastiCache you can scale both write and memory processes through sharding and data replication.
Use cases of Amazon ElastiCache include:
Amazon Neptune is a fully managed graph database service. It enables you to create and run applications using highly-connected data sets. It supports the storage of massive relationship data sets with low-latency access. Neptune supports a variety of graph models and languages, including RDF, SPARQL, and Gremlin. It includes features for point-in-time recovery, read replicas, and continuous backup.
Use cases for Amazon Neptune include:
Amazon Timestream is a fully managed, time-series database service. It enables you to store, process, and analyze up to 1,000X better query performance at 90% lower cost, compared to relational databases offered on AWS. Timestream provides automatic hardware provisioning, updates, setup and configuration, and data tiering.
Use cases for Amazon Timestream include:
Amazon (QLDB) is a fully managed, serverless ledger database service. You can use it to track application data changes with a verifiable history. With QLDB, you can avoid the need to build custom ledger applications and associated verification tools. You can query data in QLDB using a SQL-like API.
Use cases for Amazon QLDB include:
Once you’ve decided which services are best for your needs, you need to decide what data you want to migrate and how. Depending on the databases you are currently using, this can be as simple as creating a backup and importing that backup to the new database service. Or, it could require reformatting or re-scheming data.
To make migration easier, Amazon offers the AWS Database Migration Service. This service is free for the first six months of many AWS database services, including Aurora, DynamoDB, and Redshift. With this service, you can generally keep your databases operational during migration. This means minimal downtime and enables you to reduce revenue loss and disruptions to productivity.
Database Migration Service supports most commercial and open-source databases for easy transfer. This includes both homogeneous migrations, for example, SQL Server to SQL Server, or mixed migrations, for example, Oracle to Amazon Aurora. You can also use it to continuously replicate and consolidate databases to a data warehouse via Amazon S3 and RedShift.
Using AWS Database Migration Service can provide several distinct benefits. The most notable benefits include:
Simple to use
Using this service doesn’t require the installation of applications or drivers. It also generally doesn’t require you to make changes to your existing databases. The service is easily launched and managed from the AWS Management Console and once started is fully managed.
To get started from the console, you only need to define your migration parameters in a new migration task. This involves setting up connections between databases and choosing replication instances. Then, you can use these parameters as a template to perform migration tests or multiple live migrations using the same settings.
Minimal downtime
Any changes you make during the migration process are automatically, and continuously replicated to your target service. This enables you to continue using existing databases up until you are ready to go live with your replacement services. You also have the option of leaving the sync operational indefinitely. This is particularly useful for mission-critical applications that you wish to create a failover for.
Low cost
AWS Database Migration Service is free for the first six months (depending on the service you’re migrating to). You are only responsible for the compute resources and log storage used during migration. All inbound data transfers are free. Even compute and log storage costs are very low, however. For example, a terabyte-sized migration can generally be done for around USD $3.
For a custom analysis of what migration with this tool might cost, you can check out Amazon’s TCO calculator.
Supports widely used databases
AWS Database Migration Service can migrate data between most commonly used databases. You can use it to easily move databases between platforms or to migrate data between services. For example, you can move data from EC2 to Amazon RDS or from RDS to EC2. You can also use it to transfer data between database types, including NoSQL, SQL, and text-based stores.
Reliable
When you use the Database Migration Service, your migration is performed in a highly resilient and self–healing way. The service achieves this by continually monitoring your source and target databases, network connectivity, and replication instances. This ensures everything runs smoothly.
However, if issues do arise, the service automatically detects the problem and takes action to correct it or notify you. Typically, this means automatically restarting the migration process from where the issue occurred.
NetApp Cloud Volumes ONTAP, the leading enterprise-grade storage management solution, delivers secure, proven storage management services on AWS, Azure and Google Cloud. Cloud Volumes ONTAP supports up to a capacity of 368TB, and supports various use cases such as file services, databases, DevOps or any other enterprise workload, with a strong set of features including high availability, data protection, storage efficiencies, Kubernetes integration, and more.
In particular, Cloud Volumes ONTAP helps in addressing database workloads challenges in the cloud, and filling the gap between your cloud-based database capabilities and the public cloud resources it runs on.
In this article, we offered an overview of eight AWS databases, including features, capabilities, and use cases. We also provided a brief guide about AWS migration services, and how you can leverage Cloud Volumes ONTAP to address the challenges of managing databases in the AWS cloud.
There’s a lot more to learn about AWS databases. To continue your research, take a look at the rest of our blogs on this topic:
AWS MySQL: Two Ways to Enjoy MySQL as a Service
You can set up MySQL on AWS on your own, or you can use MySQL databases as a service. The latter is offered as a service called AWS RDS, which enables you to run a managed database instance. Another option is to use Amazon Aurora, an elastic database service which is fully compatible with MySQL.
If you’re looking for information about RDS and Aurora for MySQL, check out our article: AWS MySQL: Two Ways to Enjoy MySQL as a Service.
AWS Oracle: How to Lift and Shift Your Oracle DB to Amazon
Many Oracle databases are still located on-premises, despite the difficulties of on-prem data management. If you are looking to migrate your Oracle database to AWS, there are two main AWS database services that can run Oracle databases: The Relational Database Service (RDS) and Amazon EC2.
Our article: AWS Oracle: How to Lift and Shift Your Oracle DB to Amazon, can help walk you through the process of migrating on-prem Oracle workloads to the cloud.
AWS Oracle RDS: Running Your First Oracle Database on Amazon
Amazon Relational Database Service (RDS) is a managed service for relational databases such as Oracle. You can use the AWS platform to backup, monitor, secure, configure, and scale your Oracle workloads.
This post reviews the features and licenses offered for relational Oracle databases on AWS RDS, and walks you through the process of creating an RDS DB instance.
Read more: AWS Oracle RDS: Running Your First Oracle Database on Amazon
DynamoDB Pricing: How to Optimize Usage and Reduce Costs
DynamoDB is an AWS database for NoSQL high-performance applications. DynamoDB is a great service for serverless web apps, mobile backends, microservices, and operations that need low-latency data access. Learn how DynamoDB Pricing works, including basic DynamoDB storage and architecture concepts. Discover tips for optimizing your DynamoDB costs.
Read: DynamoDB Pricing: How to Optimize Usage and Reduce Costs
AWS Database as a Service: 8 Ways to Manage Databases in AWS
AWS Database as a Service (DBaaS) offerings enable cloud users to leverage cloud-based databases without doing administrative infrastructure work. Each AWS DBaaS offers different types of uses, functionalities, and management levels. Learn about the eight types of managed databases offered on AWS, what is different about the AWS database offering, and read real life case studies.
Read: AWS Database as a Service: 8 Ways to Manage Databases in AWS
SQL Server in AWS: Two Deployment Options
There are two primary ways to run SQL server in the Amazon cloud, each with its pros and cons. Managed database: deploying SQL server using Amazon RDS and self-managed: deploying SQL server on EC2 with EBS. Learn the details and pros and cons of each of these deployment options.
Read: SQL Server in AWS: Two Deployment Options
Amazon DocumentDB: Basics and Best Practices
DocumentDB is a managed service by Amazon Web Services (AWS) that provides access to the popular NoSQL database engine, MongoDB. Learn about the key features of DocumentDB, its architecture, and best practices for successfully deploying MongoDB on AWS.
Read: Amazon DocumentDB: Basics and Best Practices
AWS PostgreSQL: Managed or Self-Managed?
PostgreSQL is a high-performance, enterprise-class open-source relational database that supports both SQL (relational) and JSON (non-relational) queries. Learn about running AWS PostgreSQL using RDS, Aurora, or self-managing PostgreSQL in an EC2 compute instance.
Read: AWS PostgreSQL: Managed or Self-Managed?
AWS NoSQL: Choosing the Best Option for You
NoSQL databases enable you to store data with flexible schema and a variety of data models. These databases are relatively easy for developers to use, and have the high performance and functionality needed for modern applications. NoSQL AWS databases can hold large volumes of data while still providing low latency. Learn about 6 types of NoSQL Databases Offered on AWS, and leading AWS NoSQL database services, including DynamoDB, DocumentDB, ElastiCache and Neptune.
Read: AWS NoSQL: Choosing the Best Option for You
AWS Database Migration Service: Copy-Paste Your Database to Amazon
AWS Database Migration Service (DMS) is a service for migrating any data type from any data source to the AWS cloud. DMS supports migration to RedShift, Amazon RDS, Aurora, DocumentDB, and DynamoDB. You do not need to install agents or alter your source database to use the Database Migration Service. Instead, you can control the migration process from the AWS Management Console. Once started, migration is a managed service that is resilient and self-healing.
Read more: AWS Database Migration Service: Copy-Paste Your Database to Amazon
See Our Additional Guides on Key Cloud Storage Topics
Together with our content partners, we have authored in-depth guides on several other topics that can also be useful as you explore the world of managed services.
Authored by NetApp
Authored by Cynet
Authored by BlueVoyant