The latest release of Astra Control version (21.12) introduces Execution hook feature to enable application consistent snapshot, backup, and clones for your custom Kubernetes applications.
An execution hook is a custom script that you can run before and after creating a snapshot of a managed app. For example, if you have a database app, you can use execution hooks to pause all database transactions before a snapshot, and resume transactions after the snapshot is complete. This ensures application-consistent snapshots.
Application consistent recovery plans require a solution that understands the application metadata, its state and the persistent data that resides in its PVCs. Snapshots or backups need to ensure an application consistent view is committed across all these resources to the persistent volumes. To achieve this state, we require an orchestration to execute a user provided script or set of application specific tools before and after creating an application snapshot or backup.
Astra Control comes with default execution hooks, provided by NetApp, that handle freeze and thaw operations before and after snapshots for MariaDB, MySQL and PostgreSQL.
If you use other apps, Astra Control provides an option to use execution hooks with pre- and post- snapshot scripts. Pre and post execution hooks in Astra Control provide the ability to perform application consistent snapshots and backups by running scripts to quiesce applications before taking a snapshot and unquiesce after taking a snapshot. You can create an execution hook after the application is managed by Astra Control.
The pre- and post- execution hooks can be set up to run in all containers in the managed application or specific containers by providing the image name for the container. You can add more than one pre- and post- snapshot execution hook scripts for a managed application.
NetApp Astra Control creates snapshots and backups that preserve the application state, Kubernetes resources, and its volumes in one easily manageable unit. Application snapshots reside in the PVs (Persistent Volumes) and all application backups are stored in an object store. When using execution hooks, the pre-snapshot script will run first before taking any snapshot or backup.
Refer to Astra Control documentation for additional information execution hooks. Example execution hooks for commonly used cloud-native apps, like MariaDB and Cassandra, are available in the NetApp-sponsored open-source project Verda.
Databases like Apache Cassandra and other stateful applications benefit from application consistent snapshots and backups. A write operation in Apache Casandra can come through any node (called a coordinator node) in the Apache Cassandra Cluster. When the write operation hits an individual node, it will immediately write into the commit log and then gets merged to memory (memtable). Now the client will get the acknowledgement back for the write operation. However, the memtable only gets flushed to the disk (sstable) periodically. To create an application consistent state, Apache Cassandra must flush the data written to memory (memtable) to the underlying PVC for each node in the cluster. Apache Cassandra provides the nodetool utility to flush the data from memtable before taking a snapshot or backup. Running this tool on every node in an Apache Cassandra Cluster before taking a snapshot or backup creates a consistent state across the Cassandra cluster.
In the screenshots below, execution hooks in Astra Control facilitate running nodetool flush --<keyspace_name> on each Apache Cassandra node (on Cassandra container with image name cass-management-api) at the same time to flush the memtable to PVC (Persistent Volume Claims) before taking a snapshot and backup.
And after completing the snapshot, Astra Control will run the post snapshot script nodetool verify --<keyspace_name> to check the data checksum of all tables in the keyspace. When the keyspace is not specified the node tool flush and verify will run for all keyspaces in your Apache Cassandra Cluster.
Below screenshot shows the final view after adding pre and post snapshot execution hook scripts.
Now the pre and post execution hooks are in place and let us take an application consistent snapshot for Apache Cassandra cluster.
Once a snapshot operation is initiated, you can see the details of running execution hook scripts from Astra Control in the Activity logs
You can see the Pre and Post execution hooks run three times. Yes, the pre and post scripts ran three times as the Cassandra cluster here has three nodes. The actions can be verified through the debug log file entry from a Cassandra Cluster node shown below.
With Astra Control, you can take application consistent backup and snapshot for your entire Cassandra cluster that includes the Cassandra operator, its Kubernetes objects, the state of your Cassandra cluster and its volumes (PVCs) as one easily manageable unit.
Download the latest version of the self-managed Astra Control Center free trial, or sign up for the fully managed Astra Control Service free plan. Install it in your on-premises Kubernetes cluster or try it out in the cloud to provide application data management for all your Kubernetes applications. Please let us know your experience and feedback. You can reach astra.feedback@netapp.com. Looking forward to hearing from you!!