Azure Storage Account allows the users to store all kinds of data, including structured, semi-structured, and unstructured data, through tables, queues, blobs, and files. Storage blobs are used to store all forms of unstructured data. In this blog, we will learn about how Cerebrata can be used to amplify your Azure Blobs Storage.
Azure Blob Storage
Azure Blob Storage is a feature of Microsoft Azure. It allows users to store large amounts of unstructured data on Microsoft’s data storage platform. In this case, Blob stands for Binary Large Object, which includes objects such as images and multimedia files. These are known as unstructured data because they don’t follow any data model.
Blob is the storage arena for text, audio, images, and video. Every Blob resides inside a container. Blob storage act as the persistent store, where the data is stored for a long time. Three options include Private, Public Container, and Public Blob.
One of the advantages of Blob storage is that it offers zone-wise redundant processes. Different copies of data are created in the same zone or across two separate zones. So, making copies in two distinct zones helps in data recovery. If one of the zones faces any failure, then the Azure redundant process can still help retain data. It even allows data read options from another location.
Problem
Blob storage is optimized for storing massive amounts of unstructured data. A storage account container can hold an unlimited number of blobs. There are times when we want to find out details about all the blobs in a blob container to analyze the blobs’ properties further. So here are the five recommendations to amplify your Azure Storage Blobs management using Cerebrata.
Cerebrata Quick intro
Cerebrata is a cross-platform desktop tool for managing Azure Resources under one roof. It facilitates working on multiple operating systems like Windows, Linux, and Mac. Over the last few years, Microsoft has built excellent tooling around different technologies. Today, everything is available in one place – the Azure Portal. With Cerebrata, these Azure technologies can be efficiently managed and maintained across various platforms.
Search blob by tags
Blob index tags categorize data in your storage account using key-value tag attributes. These tags are automatically indexed and exposed as a searchable multi-dimensional index to find data quickly. So, blob tags can be used to group and search-related blobs efficiently. We can create tags with the same key and value for related tags. Creating more than one tag for a single blob is also possible. So, we can filter the blobs in a container that contains the required tags.
Since Azure Storage blob is designed to hold an enormous amount of data, it should be straightforward for us to filter the required blobs while listing the blobs in a container. Azure Storage allows us to filter the blobs in a container based on the prefix. So, it is possible to list the blobs in a container whose name starts with the provided text. But it is not always possible to create related blobs with the same prefix. Another disadvantage here is that we can list the blobs starting with only one prefix at a time
Azure solves this problem by listing the blobs based on the tags created on them. It solves the pain caused by listing the blobs based on prefixes. We can create tags with the same key and value for related tags. Creating more than one tag for a single blob is also possible. So, we can filter the blobs in a container that contains the required tags. It is also possible to apply the numeric operations like >, <, >=, and <= for the tag values holding numeric values.
Bulk Operations
Perform bulk operations on blobs/files to change properties, take snapshots, change access tier, etc. Cerebrata provides a convenient option to select any number of blobs inside a container and change their access tier along with the snapshot and its version.
Properties of blobs can be updated in bulk. It is also possible to set the content type of the blobs based on their extensions. These extensions help remove the need to set the Blob’s content-type property manually.
It is possible in Cerebrata to change the access tier for multiple Storage blobs along with the snapshot and its version based on the last modified date/time. Various time interval options are available, or you can enter your custom date and time as per your requirements. For example, if you select the option 30 days, all files modified before 30 days will be automatically charged to the given access tier.
Here is an example of changing the Access Tier using bulk operations.
Uploads/Downloads
Manage uploads/downloads on block size, overwrite behavior, compression settings, etc. In addition to Blobs creation, Cerebrata also allows users to upload the files and folders from the local storage. Consider if the need is to back up the blobs into a local machine. Cerebrata will enable users to download the files/folders into the required folders.
It is another powerful feature in Cerebrata where you can customize how the blobs are downloaded to your local file system. Another powerful feature in Cerebrata is the way you are enabled to customize the upload operation.
When it comes to uploading, you can customize
Chunk Size: Chunk size determines how a large file is split into smaller chunks (blocks), and then these chunks are uploaded in parallel. This setting becomes useful when you do not have a good Internet connection.
Overwrite Behavior: Overwrite behavior controls if the Blob should be overwritten or not. It also gives you the capability to create a snapshot of an existing blob before overwriting the Blob.
Page Blob File Extensions: You can specify a list of file extensions, and any matching files with those extensions are uploaded as page blobs (default will be block blobs).
Compress File Settings: At times, you may want to compress a file before uploading. Typical examples are compressing images, javascript, and CSS files. You can specify a list of file extensions, and any matching files with those extensions are first compressed and then uploaded.
Compression Algorithm: Using this setting, you can specify the compression algorithm (GZIP or DEFLATE).
Block Blob Access Tier: For storage accounts supporting access tiers (Hot, Cool, or Archive), you can instruct Cerebrata to upload a blob in a specific access tier directly.
MD5 Hash: By default, Azure Storage SDKs only calculate the MD5 hash of uploaded files without breaking them into chunks. Using this setting, you can instruct Cerebrata to calculate the MD5 hash of any file regardless of its size.
When it comes to downloading, you can customize
Overwrite Behavior: Overwrite behavior controls if the Blob should be downloaded or not if a file by the same name already exists on the local computer.
Preserve Blob’s Last Modified Date/Time: Using this option, set the date/time of the local file as that of the Blob’s last modified date time.
Decompress: When this setting is enabled, downloaded files are automatically decompressed.
Last accessed and Last modified
Access tiers are designed to target data at different stages of its lifecycle and provide cost-effective storage options for other use cases. Cerebrata facilitates changing the access tier of multiple blobs according to their last modified and last accessed date/time.
Change the access tier for multiple Storage blobs and the snapshot and its version based on the last modified date/time. Various time interval options are available, or you can enter your custom date and time as per your requirements. Cerebrata allows you to change the blobs access tier based on the last accessed date/time. You have the same options as in the change access tier based on the previous access.
For example, if you select the option 30 days, all files modified before 30 days will be automatically moved to the given access tier.
Blob Container Inventory
There are times when we would want to find out details about all the blobs in a blob container to do further analysis on the properties of the blobs. The analysis could be to find the age of the blobs (when the blobs were created), when the blobs were last modified, their access tier, etc.
One of the many convenient features in Cerebrata is Blob Container Inventory, which lists all blobs in a container with details. All types of Blobs like Block blob, Append Blob, and Page Blob details can be fetched in a single file accordingly. Cerebrata can export all the blob snapshots and their versions, along with the blob properties in every storage container, as a CSV file in a specified designated local system folder as per your convenience.
Conclusion
This blog showed the five recommendations to amplify your Azure Storage Blobs management using Cerebrata.
Cerebrata enables you to manage multiple Azure Azure Cosmos DB accounts (SQL API Table API), Service Bus Namespaces, Cognitive Search Service accounts, Redis Cache accounts, etc. It is also cross-platform to manage your Azure resources from a platform of your choice – Windows, Mac, or Linux.
If you wish to experience Cerebrata, try our 15 days free trial.