Storage
Mobility
Protection
Analysis and control
TOPICShare
Not all data are created equal. Some are more important than others, and others are so cold they’d never be in use. When it comes to data that is stored as objects in Google Cloud Storage, there is a storage class to fit the data’s usage pattern. But what can you do if those usage patterns change over time?
In Google Cloud—for which NetApp® Cloud Volumes ONTAP is now available—the object storage service is aptly named Google Cloud Storage (GCS). This article will introduce you to the basics of GCS, such as the different storage classes, and practical steps on how to add/remove data and switch data between different storage classes so you can find the repository that best fits your data’s usage pattern at a given time.
What Is Object Storage?
An object storage enables users and applications to put and get files (also known as objects) via a REST API. Multiple files are stored in a bucket, i.e., a logical group of objects. You can have multiple buckets and objects; however, an object can only belong to one bucket. Google Cloud provides built-in features for managing lifecycle rules in its storage buckets. Object storage services provide near-unlimited growth capacity and high availability.
The concept of object storage became popular with the rise of cloud computing. While it can exist in more traditional data center infrastructures, due to its unique value proposition it is often only associated with cloud computing. Abstracting the underlying operational details to the customers is an incredible selling point that makes object storage very attractive to organizations and is today the most widely used type of cloud storage service.
Storage Classes in Google Cloud Storage
Storage Class is a feature that enables the Google Cloud Storage customer to have some control over the type of availability they require from their data. A storage class can be set to either an entire Google cloud storage bucket or individual objects. Understanding these storage classes is crucial to cost-efficient usage of GCP storage.
In this object-based Google storage service, there are three different storage classes:
- Standard: This storage class is well suited to frequently access data, also called “hot data.” It offers great performance for data-intensive computation when placed in the same location as the processing applications. It’s also a great option for data that is stored for only short periods of time.
- Nearline: This storage class is best suited for infrequently accessed data. At a reduced cost compared with the Standard storage class, it is a great option when slightly lower availability is acceptable (from a typical availability of 99.99% in Standard to 99.95% in Nearline). The minimum storage duration is 30 days and is ideal for data that only needs to be accessed once per month (or less). Typical uses cases are data backups, archiving, and multimedia content.
- Coldline: This storage class is better suited for data that you plan to access once a year or less. The minimum storage duration is 90 days and while the storage cost is incredibly low ($0.007 per GB-month), the cost of access per-operation is higher than the other storage classes. This storage class is ideal for data that you need to store for legal or regulatory reasons, or high volumes of data required for disaster recovery.
Availability of data can be defined for each storage class by choosing the location type: multi-region, dual-region or region. This enables the customer to choose a certain availability SLA. Which, in practice, means that if Google fails to meet that uptime expectation, the customers are eligible to receive a credit.
Choosing the right storage class and availability for a Google storage bucket is extremely important, yet often underestimated. While the right combination can allow massive cost savings and efficient operations, the wrong combination may result in severe performance and availability issues.
Google Storage Service in Practice
Creating a Google Cloud Storage Bucket
A Google storage bucket is the starting point to start using GCP storage. A bucket is a logical namespace that contains one or more files (known as objects). Creating a bucket is a very straightforward process.
1. In the Google Cloud Platform Console UI, head to the Storage section and initiate the process by clicking the “Create Bucket” button.
2. Some required information will need to be filled out to complete the bucket creation process (Fig.1). You will need to choose a unique global name, i.e., a name that is unique not just to you but to all Google Cloud customers. Also, you will need to specify in which location the bucket will reside in and what type of availability you want (region, dual-region, or multi-region).
It is worth noting that on the right-side panel you will be able to estimate the monthly cost for your Google storage service usage based on your inputs.
Fig.1 Bucket creation screen (Name and Location type)
3. Complete the creation process by choosing one out of three default storage classes (Standard, Nearline, or Coldline) for the objects in the bucket (Fig.2) and click “Create.”
Later on in this article we’ll show you how this setting can be changed manually at any point after bucket creation.
Fig.2 Bucket creation screen (Storage class)
Your bucket should now be set up and ready to use, which we will show you how to do in the next section.
Adding and Removing Data in Google Cloud Storage
Now that you have created a Google Cloud Storage bucket, you can easily use the GCP Console UI to add and remove data.
1. In the Bucket details screen, you can simply click the “Upload files” button and select the files you wish to put in the bucket (Fig. 3). Another valid option would be to drag & drop your files into the browser window.
Fig.3 Bucket details screen (Upload files).
2. The status of the upload for each file can be followed in the bottom-right corner (Fig.4.). Once the transfer is completed you see the files listed on the screen.
Fig.4 Bucket details screen (Upload status and Delete files)
3. If you wish to delete the files, you can simply select the chosen objects and use the “Delete” button (also shown above in Fig. 4).
Manually Set the Storage Class
As mentioned in the section on the bucket creation process, the default storage class can be changed at any time after the bucket is created. To do so, simply click the “Edit bucket” button (Fig. 5) and select the new storage class.
Fig.5 Bucket edit screen (Default storage class)
It’s worth mentioning that it is also possible to set a storage class for a specific object that is different from the default one. However, this option is only available via the Google command line tool and therefore is not present in the GCP Console UI.
How to Set Up Lifecycle Policies to Automatically Switch Storage ClassesSince changing the default storage class only applies the setting to new objects (Fig. 5), the command line tool will also be the only method available to change existing objects.
Sometimes you’ll have requirements that dictate certain objects should transition between different storage classes. While you could write a custom code to perform this task, a different and simpler alternative is available to you: Lifecycle Policies.
1. To create a Lifecycle policy rule, go to to the bucket browser screen (Fig. 6) and click on the bucket lifecycle rule list (by default, “None”).
Fig.6 Bucket browser screen (Lifecycle)
2. The screen will allow you to add different rules according to your own business requirements (Fig. 7).
Fig.7 Bucket lifecycle rules screen (Rules).
An object lifecycle rule has a great potential, enabling you to transition objects, if certain conditions are met, to a different storage class or mark for eventual deletion.
3. A quite straightforward and typical example of using Lifecycle policies is to transition objects to a storage class type with less availability after a certain amount of time (Fig. 8). If this supports your business requirements, it will enable to automatically save you on the GCP storage costs.
Fig.8 Bucket lifecycle rules screen (Add object lifecycle rule)
Conclusion
This article introduced the concept of object storage in cloud computing and, provided an overview of this crucial Google storage service. One of the most important features of buckets in GCP storage are the different storage classes and their respective availability types.
In addition, the article demonstrated step by step how to create a Google Cloud Storage bucket, how to add and remove data, and detailed a number of different methods to change the storage class. As mentioned above, and worth highlighting once more, the advantages of understanding in depth and leveraging the different storage classes and availability types can, when done right, lead to massive cost savings. However, when misconfigured, the exact opposite can take place, leading to a potential negative cost and performance impact.
There are even more benefits that Google storage services can get from Cloud Volumes ONTAP, the premier cloud data management platform from NetApp. Cloud Volumes ONTAP provides Google Cloud users with all of the storage efficiencies, data protection, and cloning features that have made it essential to deployment in data centers and AWS and Azure for years.
