Software-Defined Storage

Avoiding the pitfalls and enjoying the pluses


Explosive data growth and the rapid adoption of virtualization and the cloud are major driving forces behind the need for huge increases in storage capacity. In search of scalable and flexible storage, enterprises are increasingly turning to software-defined storage (SDS). Transparency Market Research predicts that the global SDS market will grow at a compound annual growth rate of just over 29% between now and 2025.

The affordability of SDS makes it a compelling option, but as adoption of software-defined storage grows, it is important for organizations to be knowledgeable about current options, both in terms of what is available and what to be wary of. Compatibility problems, inconsistency issues or lack of features are all possibilities in today’s SDS market. Any one of these, let alone a combination of them, can create problems.

The Case for Unified Storage

A unified system helps you make the most of your SDS because it combines all the 'flavors' of storage together. For instance, if your current file-based storage system offers support for object store as well, it can save the hassle of managing and balancing many different complementary storage systems. First, this unified approach is easier to manage, and second, it makes better and more efficient use of resources in relation to performance and capacity. It’s similar to virtualization, where you cut back on hardware resources that are idling.

But buyer beware. Some software-defined storage companies claim to have flexible SDS with object, block and file storage, be both hyper-converged and hyperscale, and claim to support flash storage. However, many lack the features to back up those claims.

Oftentimes, SDS approaches only address one scenario, such as:

- Scale-out file systems

- Hybrid cloud

- Object storage

- Archiving

- Hyperconvergence


Vendors of these limited offerings can ask one-third the price of more comprehensive alternatives, but they also have one-third of the features. In addition, they are not focused on general-purpose NAS.

The Case for Consistency

Most enterprises would benefit from a general-purpose NAS offering that scales well. But just as with SDS solutions, not all NAS offerings are created equal.

One aspect of scale-out NAS that many enterprises don’t know about is that consistency is critical. Some storage environments are only eventually consistent. This means files written to one node are not immediately accessible from other nodes. Even when the other nodes have been updated to record the change made to the original node, a delay of just fractions of a second can cause problems with accessing applications or users. This can be caused by not having a proper implementation of the protocols, or not tight enough integration with the virtual file system.

Rather than eventual consistency, scale-out NAS needs strict consistency, which means that files are accessible from all nodes at the same time. The view of the file system through each node is strictly consistent, so that any modification on one node is instantly available from any other node. Make sure that your scale-out NAS is consistent between protocols as well. That means if you write something in SMB, for example, it should be immediately visible over NFS as well.

What to Look For

In addition to the features noted above, this is what a comprehensive SDS offering looks like:

- Hyperconverged – Being software-defined, this architecture “converges” compute, storage, networking and virtualization resources, and other technologies on a commodity server.

- Flexible and scalable – You have the option with SDS of starting small and rapidly adding multiple virtual machines to the same cluster as needed, eliminating the cost and hassle of building new clusters in order to accommodate scale-out. If a storage cluster is built on a symmetric architecture, linear scaling up to hundreds of petabytes and billions of files is possible, simply by adding more storage nodes to the cluster. Adding storage nodes and increasing capacity can be carried out during runtime and does not interrupt any ongoing operations in the cluster.

- File systems – For unstructured data, this type of storage is a must. Make sure your SDS setup includes crucial file features such as tiering, quota, snapshot, encryption, antivirus, WORM and retention. It should also be able to integrate into Microsoft Active Directory, have support for multiple authentication providers and enforce authorization checks. If your company is a large one, ensure that the offering has support for multi-tenancy, where you can create multiple file systems in the same environment.

- Disaster recovery – You can safeguard every application with a unique disaster recovery policy and remain highly available if you choose an SDS platform with a storage cluster that is important to back up.

- Hybrid cloud – If your setup includes a local presence and a presence in the cloud, part of your data needs to live in and be accessed from the cloud as well. For example, part of your local storage system will be exposed to virtual machines running in a public cloud like Amazon. That means your SDS file system needs to cover both environments so you can easily pass files between them.

- Hardware-agnostic – You can add additional hardware of your choice as needed to scale performance and capacity over time because SDS enables you to use standard commodity storage hardware and servers without a lock-in to a specific vendor and/or technology means.

Knowledge is Power

Organizations must address the astronomical volume of data generated by today’s business-critical technologies. Traditional storage architectures are too costly and inflexible to be practical, leading to the widespread adoption of SDS. However, enterprises need to be cautious as they enter the market to avoid the narrow offerings. Most organizations are well served by an SDS approach that uses a general-purpose NAS. It should offer compatibility and strict consistency as well. To get the data storage your enterprise needs, use the above checklist when researching your options.


Read next:

Why Blockchain Hype Must End