Data Management: The Data Storage Dilemma

November 27, 2019
Backup, Cloud, Data Protection

At one of US Signal’s Beers with Engineers events, we asked attendees what some of their biggest data management concerns were. Data growth was number one. 

That’s not surprising. According to estimates by various industry analysts, the digital universe is expected to reach somewhere around 44 zettabytes — 44,000,000,000,000,000,000,000 bytes — by 2020. That’s a lot of data, especially if you’re an IT professional who’s responsible for managing it.

The Storage Side of Data Management

It’s not just gathering and processing all that data that concerns IT professionals. Much of the data has to be stored, sometimes for a short time and often for a long time, depending on legal and regulatory requirements, business needs, and numerous other factors.

Even if it’s stored for a long time, specific data still has to be easy to find and access. There are also the data backups to consider. If you only have one copy of certain data and it’s lost, that could be a big problem — and potentially a costly one.

Of course, data backups also add to the growth issues. The more data you have, the more backups you have. As a result, many companies either have to buy more storage or delete some of their data. The latter isn’t always an option given various industry and regulatory requirements.

Given all the issues associated with data storage, it’s also no surprise that the second leading concern among IT professionals at the Beers with Engineers event was storage performance. Data storage doesn’t do a lot of good if you’re dealing with issues related to throughput, latency, and input/output operations per second (IOPS).

The Object Storage Solution

One of the ways to deal with both data growth and data storage needs in a data management strategy is to use cloud-based storage, specifically cloud-based object storage.

With object storage, an object is defined as data and bundled with its metadata. The object is given an ID and is retrieved by an application by presenting the object ID to object storage.

An object isn’t limited to any type or amount of metadata. It’s possible to assign metadata such as the type of application the object is associated, the level of data protection required, when the object should be moved to a different storage tier or geography, and when to delete the object.

As such, object storage provides the extended identification, classification, and ownership information necessary to establish storage policies and automate functions such as billing, tiering and custom access management.

The metadata also allows for easily migrating objects between multiple tiers of storage, providing the flexibility to optimize a cloud storage environment based on whatever combination of cost, performance, availability, and data protection best suits a company’s data governance requirements.

One of the biggest benefits of object storage is that it can be scaled with almost infinite capacity. That makes it ideal for cost-effectively storing large quantities of data, including unstructured data.

Object storage is also easy to access simply by using a REST API via the internet. Storage API calls are made to highly available public API gateways, which parse data into smaller fragments. Those fragments are then encoded and stored on multiple backend cluster resources for data redundancy. This gives object storage its self-healing capability. In addition, erasure coding provides the redundancy needed to protect data against failures.

Your Data Storage Policy

Object storage offers several benefits, but it’s not a one-size-fits-all solution. Before considering its use in a data management strategy, look at your company’s data policy. If you don’t have one — and the data management platform to enact it — get one.

You’ll need to know and stay on top of the compliance requirements that affect your business. There may also be industry-specific guidelines and company requirements for data governance.

Use this information as a starting point to group data into categories based on long– or short-term retention requirements or needs. Assess how storage tiers can be employed for cost effectiveness and performance.

Some data may need to be stored local to the processing for fast access and overall availability. Other data may be able to be moved elsewhere for archival or long-term storage. Consider creating a system that allows for data groups to automatically move between storage  tiers as part of the data management lifecycle.

Take a look at all your data storage options. Cloud-based storage, including object storage, may be the way to go. However, there could be reasons to use on-prem disk or offline tape.

Talk to US Signal

To learn more about cloud-based data storage options and data management and protection strategies, contact US Signal. We’ve got options and the expertise and experience to help you use them to meet your company’s specific data needs.

Call 866.2. SIGNAL or email [email protected].