Storage
Mobility
Protection
Analysis and control
TOPICShare
When it comes to cloud storage, knowing how to properly leverage and configure the different services you use can make a huge difference, both in your day-to-day operations and in the bill you get at the end of each month. With different and often unpredictable amounts of data to deal with, how can you best manage your data lifecycle? Google Cloud Storage provides you with a handy solution.
In Google Cloud Storage (GCS) the Object Lifecycle Management feature makes this job much easier. This article will introduce you to the basics of GCS and practical steps on how to create lifecycle rules to automate actions on your data using the built-in capabilities of GCS without the need to develop a custom service or operate servers.
What Is Google Cloud Storage?
Google Cloud Storage is the object storage service provided by Google Cloud Platform. It offers multiple out-of-the-box features such as high availability, built-in server side encryption, different storage classes, and lifecycle management. Google Cloud Storage pricing is also quite low, making this a very popular service for enterprise storage deployments that have vast amounts of data on hand.
Like any cloud-based object storage service, Google Cloud Storage enables close to unlimited storage capacity and out-of-the-box high availability. Put and get operations for users or applications can be carried out via a REST API. The files (also known as objects) are logically stored in a bucket. While you can have many buckets and objects, a single object can only belong to one bucket.
Object Storage Vs. Block Storage: Understanding the Differences
The traditional storage that can be found in both on-premises and cloud computing is block storage. This type of storage is often associated with the physical hard disks of servers or a network drive. Block storage can be directly used and accessed as a mounted volume by the operating system; in contrast, object storage is only accessible via a REST API (except in some rare cases, such as trying to mount S3 as drive).
Lifecycle Rules in Google Cloud Storage
Lifecycle rules provide the ability to manage different aspects of the objects’ lifecycle inside a given bucket. Rules can be triggered based on certain conditions such as age, storage class, date, state, and version. If the defined conditions are met, objects can be either deleted or moved to another of the Google Cloud Storage classes.
Lifecycle management for object storage isn’t exclusive to Google Cloud Storage. Amazon S3 and Microsoft Azure Blob Storage both offer services that are similar to Object Lifecycle Management. The main differences are usually associated with the different types of storage class and how to leverage the lifecycle rule capabilities.
Lifecycle Rules are a powerful tool that is built-in Google Cloud Storage and it is important to know how to leverage them. While the Google Cloud Storage pricing (per GB) is low, it can ramp up quickly when used for big data volumes or when used with multiple API operations. This is often due to improper configurations or not knowing how to take advantage of the built-in features.
Lifecycle Rules in Practice
Creating a Google Cloud Storage bucket
Since Object Lifecycle Management is directly related to a given bucket, the creation of a Google Cloud Storage bucket is the first step required to be able to explore lifecycle rules. The creation process is quite easy and straightforward. You can find out how to create a Google Cloud Storage bucket in this post.
Once your bucket is created and the files are uploaded you can proceed with setting up lifecycle rules to manage them, as described below.
How to Set Up a Lifecycle Rule to Automatically Delete Data
As mentioned above, the built-in lifecycle management feature enables you to automate different aspects of the data lifecycle in a bucket. One possible real-world scenario is to automatically delete data older than a specific number of days. The steps below will show you how to set this lifecycle rule up.
1. Head to the bucket browser by selecting “Browser” from the left-hand menu. Under the “Lifecycle” column, click “None” to go to the lifecycle management screen.
The bucket lifecycle management screen
2. The screen will list all the rules enabled for the bucket. Also, it provides a way to add and delete rules. Click on the “Add rule” button to initiate the rule creation process.
Bucket object lifecycle rules screen
3. Select the conditions the object must meet to trigger a specific action. In this case, our rule will delete any object that is older than 90 days. Enter that number in the “Age” field and then click “Save” to keep these changes.
Create a new lifecycle rule
4. Once saved, the rule will immediately become active. Your rule to delete data after a certain period of time is now set. This is a powerful option that enables the automation of the object lifecycle without the need for custom code or servers.
How to Set Up a Lifecycle Rule to Automatically Change the Storage Class
A different real-world scenario would be to automatically change the storage class for data inside the bucket that was created before a certain date. This could be done, for example, by changing Standard storage to Coldline storage.
1. Head to the bucket browser by selecting “Browser” from the left-hand menu and from there go to the lifecycle management screen. Note that this time, under the “Lifecycle” column, the option will be called “Enabled” instead of “None” because of our previous created rule.
The bucket lifecycle management screen
2. The existing lifecycle rules enabled for the bucket will be listed below. From this screen you can also add the new rule by using the “Add rule” button.
Viewing the object lifecycle rules for the bucket.
3. Here you can select the conditions under which the storage class change will be triggered. As stated above, there are several options that you can choose from to define this rule: the object’s age, it’s live state, storage class, creation date, etc. For this example, we will use the object’s creation date; the rule will transition the object to the Coldline storage class when the creation date for the object took place on or before June 1, 2019.
Setting the creation date for which all data will have its storage class changed.
4. As soon as you click “Save”, the rule will become active and visible alongside any previously created rules on the object lifecycle rules dashboard.
Conclusion
The Object Lifecycle Management feature is a powerful tool that, when understood and used properly, can bring down the overall cost of your object storage usage on Google Cloud Platform.
This article demonstrated step by step how to leverage the lifecycle management capabilities in Google Cloud Storage. While Google storage services offers interesting features such as the Object Lifecycle Management feature we saw above, and you can get other interesting capabilities for storage services in Google Cloud using Cloud Volumes ONTAP, the premier cloud data management platform from NetApp.
Features that have been popular in Cloud Volumes ONTAP for AWS and Azure such as storage efficiency, data protection and cloning are now available for Google Cloud.
