Amazon FSx for NetApp ONTAP was recently released, allowing users to leverage ONTAP’s enterprise-grade capabilities as a fully managed storage service native to the AWS cloud. To give users more flexibility, AWS FSx for ONTAP operations can be fully automated programmatically and used in conjunction with various infrastructure-as-code (IaC) tools, including Ansible and Terraform.
In this post we’ll take a closer look at the various options available for you to automate FSx for ONTAP operations. Use the links below to jump right down to the sections on:
For organizations that use FSx for ONTAP extensively and at scale, manually provisioning and managing file systems can be a bottleneck. There are some risks involved with keeping up with the demands of a fast-growing line-of-business application. To reduce that risk and make operations smoother, there is an option to leverage automation and manage FSx for ONTAP file systems as code.
With FSx for ONTAP, there are multiple ways to automate file system creation and data management operations, which this blog will cover at a high level.
An API, or application programming interface, is a defined syntax and protocol to call a function. It does not describe how to perform the request itself, but only describes the structure of the request and what format the response will be. The API could be within the same program or a service running on another system on the internet. The network endpoint paradigm is generally what most people think an API is.
The three APIs described in this article are various services developers can use when automating the provisioning, configuration, and administration of FSx for ONTAP file systems and resources: the NetApp BlueXP Console API, the Amazon FSx API, and the ONTAP API.
These APIs each have different forms and capabilities. However, the request parameters for these APIs are all JSON formatted and sent with the API request, which is very typical of modern APIs.
The NetApp BlueXP Console API, just like the BlueXP Console UI, is the central control point for all your ONTAP-based storage infrastructure and services. BlueXP Console can be used to set up Amazon FSx for ONTAP, and the platform also offers developers a single API endpoint from which they can directly control BlueXP Console functions, including any FSx for ONTAP storage operations.
Users can leverage the API to build and manage FSx for ONTAP file systems, which are referred to as working environments in BlueXP Console. Once an FSx for ONTAP file system is created, there is a wide range of possible operations that can be carried out with the API. Users can leverage the API set to create and extensively manage SMB, NFS, and iSCSI volumes, including volume tiering and reviewing storage tiers usage, snapshot management, cloning, and more.
The BlueXP Console APIs can also be used to carry out key operations with other NetApp technologies and services that can be seamlessly integrated with FSx for ONTAP, including the Replication service (which simplifies SnapMirror® operations), Cloud Sync, and Cloud Data Sense.
To learn more about the features and capabilities provided by this API visit the BlueXP Console Platform API documentation.
Like any AWS service, FSx for ONTAP and its resources can be managed through a native API. With the Amazon FSx API—which is also available in the AWS SDK—you can perform operations on file systems, SVMs, and volumes and control your FSx for ONTAP backups.
The API endpoint URLs for FSx for ONTAP are region-based and allow you to form, configure, and erase FSx for ONTAP file systems within that region. Other functions let you set up and remove SVMs, compose volumes, and share them via SMB or NFS. Using the Amazon FSx API, you could also manage the volume tiering policy, volume storage efficiency, volume size, and volume security style.
With the Amazon FSx API, you can create and manage backups of FSx for ONTAP volumes. The FSx for ONTAP backup capability is part of the FSx service that takes advantage of ONTAP’s highly efficient backup architecture. By leveraging this architecture, users can enjoy all the efficiency features that come along with NetApp Snapshot copies and SnapMirror Cloud (SnapMirror replication to object storage), making backups extremely fast and highly space efficient.
To learn more about this API available actions and data types, visit the Amazon FSx API Reference and see Tools to Build on AWS for more information on AWS SDKs.
Once the FSx for ONTAP file system is created, you can leverage the ONTAP REST API to perform more extensive operations with ONTAP resources that are not available in the APIs described above. To access the ONTAP API you can use the file system’s management endpoint or the management endpoint of an existing SVM.
With this API, you can manage any supported SVM service, volume, policy or parameter in an FSx for ONTAP environment. This lets you change details that range from data containers (FlexVol, FlexGroup, LUN and qtree) and data access protocols (NFS, CIFS, and iSCSI) to services such as DNS, NIS, LDAP/AD mapping and data protection jobs and scheduling.
To get started with ONTAP REST API in FSx for ONTAP, visit the Managing FSx for ONTAP resources with NetApp applications page in the FSX for ONTAP user guide and the ONTAP Automation docs. For a complete reference, check out the latest REST APIs in the resources section of the NetApp Developer Network.
Both the Amazon FSx API and the BlueXP Console API allow you to programmatically perform operations on the three fundamental resources of FSx for ONTAP: file systems, SVMs, and volumes. That may fulfill the requirements for simple usage and relatively straightforward use cases, however, it may not be enough for more complex FSx for ONTAP setups.
There are certain scenarios where the BlueXP Console API can be more useful. These include:
Additionally, in case you need to use any other resources available within FSx for ONTAP, the ONTAP REST API can be used.
The APIs discussed above are the building blocks for many Infrastructure-as-Code (IaC) tools to provision and manage FSx for ONTAP file systems. IaC tools abstract the code requirements, leaving the setup parameters specified in friendlier configuration files.
In the following sections we’ll discuss the three IaC tools you can use with FSx for ONTAP: Terraform, Ansible, and AWS CloudFormation.
Terraform gives users an IaC tool that makes it possible to safely and efficiently build, version, control, and change infrastructure, including that for FSx for ONTAP file systems. Terraform uses plugin modules called providers which create and manage infrastructure and work with any platform or service that has an accessible API. A provider composes resources from the parameters of each resource that can be declared and managed in the Terraform configuration files.
The resources usually describe one or more infrastructure components, such as an FSx for ONTAP Storage Virtual Machine or an FSx for ONTAP volume for an FSx for ONTAP provider. The parameters are the components’ specifications.
There are two providers that Terraform could use to build and manage FSx for ONTAP: the HashiCorp AWS provider and the NetApp BlueXP Console provider.
The AWS Provider is maintained by Hashicorp, the company that developed Terraform, and as such this provider is generally very stable. This provider offers lifecycle management for AWS resources, including FSx for ONTAP. The resources available for FSx for ONTAP are file systems, SVMs, and volumes. Another resource that is available is “FSx backup,” through which volume backups can be managed.
The NetApp BlueXP Console Provider is actively developed by NetApp to ensure compatibility and feature parity. Currently, this provider has two resources: one that can be used to create a complete FSx for ONTAP environment, including an SVM, and another one that can create volumes configured for CIFS or NFS sharing. In addition, the “snapmirror” resource can be used to create new replication relationships between on-premises ONTAP or Cloud Volumes ONTAP to FSX for ONTAP, which can be used for disaster recovery, backup/archive, or migration.
Both providers can create fully functional FSx for ONTAP file systems, however it’s important to note that there are differences in their capabilities. For example, the AWS Provider allows managing more than a single SVM within the same file system, making it possible to simplify operations in a multi-tenancy configuration where multiple SVMs are required.
On the other hand, using the NetApp BlueXP Console Provider gives users the ability to create volumes, configure them for NFS or SMB access with the proper sharing settings, and set up replication relationships. This can be a big-time saver and eliminate the need to perform additional configuration steps using another IaC tool, API call, or manual step to complete the setup.
Ansible is another open-source IaC tool for infrastructure management. Though owned by Red Hat, many of the Ansible modules are written and supported by other companies and by communities of users.
Ansible groups related modules into collections within namespaces. Many technology vendors maintain the collections within a namespace, but there are also community-maintained collections within each community’s namespaces.
There are two AWS module namespaces, one maintained by Amazon and one maintained by a community, but neither currently support FSx. NetApp maintains an extensive collection of modules for Ansible under the NetApp namespace, with collections grouped by use.
For example, the ONTAP collection can already fully administer any ONTAP storage, so the available modules enable managing existing FSx for ONTAP file systems, its SVMs, and volumes. NetApp recently added to the BlueXP Console collection the ability to provision a complete FSx for ONTAP environment, and establish SnapMirror relationships with FSx for ONTAP configured as a destination.
AWS CloudFormation is somewhat similar to Terraform in that resources are managed as parameterized "Templates" in configuration files. For example, it can be used to specify a new resource or update an existing one in a file and CloudFormation will provision the new resource or modify an existing resource, recreating anew if needed.
In addition, CloudFormation offers similar resources for FSx for ONTAP as the AWS provider for Terraform. Using CloudFormation, FSx for ONTAP file systems, SVMs, and volumes and their configuration parameters can be declared in CloudFormation templates, and then managed through CloudFormation Stacks. For more information see the Amazon FSx resource type reference in the AWS CloudFormation documentation.
In this article we have shown several tools that can be leveraged to simplify the creation and operation of Amazon FSx for NetApp ONTAP, reducing the complexity and time to production.
Each of the tools discussed has a different use case within the realm of FSx for ONTAP management and while a single option may be perfect for your use case, it's not uncommon to use two different methods to build your environment.
With these various DevOps API functionalities and IaC management options to decrease the risk of configuration error and provide considerable improvements in provisioning times, imagine what you could do.