Innovations in storage have always been important. When our ancestors learned how to store grain, for instance, it meant fewer people would starve in the winter months when food was scarce. A high-yielding harvest meant building larger grain siloes, but the extra effort would ensure survival.
Today’s burgeoning data storage needs thankfully don’t put human lives at risk, but failing to meet them will likely put businesses at risk. Organizations can’t build data storage 'containers' fast enough or for a reasonable price to meet demand – at least, not using traditional, linear storage architectures.
Even the most industrious souls could not accommodate the storage demands brought on by digitization with the addition of multiple servers. That’s because vertical storage architecture contains bottlenecks that slow performance to an unacceptable level.
So, innovation has stepped in to help in the form of software-defined storage (SDS). Because SDS decouples the programming that controls storage-related tasks from the physical storage hardware, it dramatically reduces costs associated with hardware. Fewer, less-expensive servers can be used to improve both capacity and performance. Administration is simplified and made more flexible and efficient. SDS enables users to allocate and share storage assets across all workloads.
It’s not hard to see why those in need of better data storage options have embraced SDS. Gartner reported that by 2020, anywhere from 70% to 80% of unstructured data will be stored and managed on lower-cost hardware supported by software-defined storage.
Are All File Systems Created Equal?
Smartphones, social media, IoT signals and other data not contained in database form accounts for approximately 80 percent of data generated today. While it is widely understood that this unstructured data is best managed with a file system, for some reason, many SDS offerings focus solely on block or object store. Few offerings focus on file systems or do them well. Without a file system overlaying this data, it becomes very difficult to manage the data.
- The basic foundation of storage, block is used to store virtual machines or databases, but you need files as well to deal with all the unstructured data.
- Object storage, a relative newcomer, is used for machine-to-machine/IoT transactions and other applications that require extreme scalability. However, it isn’t that much better than block when it comes to managing data.
- The hard-working average Joe of storage, file systems aren’t as hyped as object, but they are the best at handling unstructured data.
Some SDS providers, aware that file systems are important, say they provide file system with their offerings. However, these file systems are usually based on a freeware module called Samba that excludes some features most Windows users are used to.
Samba enables support for SMB and allows end users to access and use files on the company’s intranet or network. However, providing file services through Samba, which is open source, often means going without needed features.
It’s a mistake to undervalue the importance of features. Organizations need file systems to deal with unstructured data, but file-related features are also necessary. These include:
Retention – This feature enables you to automatically create a single folder or a hierarchy of folders on file servers, to be deleted according to assigned policies.
Snapshot – This is a read-only copy, taken at a single point in time, of the contents of a file system or independent file set. When a snapshot of an independent file set is taken, all files and nested dependent file sets will be included.
Quota – This feature helps you monitor the amount of storage being used. You can set a soft limit quota that will warn you when part of a file system is close to reaching its storage limit but still allow data to be saved. If you set up a hard limit quota, no new data can be saved after the quota is reached.
Tiering – By using a policy, you create a filter which designates a specific file type to a particular tier. A policy enables you to designate where a specific file is to be placed and if and when the file will be migrated between file system pools. You can define both file placement and migration policies. Tiered storage is more efficient and boosts performance.
Bank on Storage Innovation
Our grain-saving ancestors would be astonished at the storage needs of modern civilization. But there’s also plenty of room for surprise among organizations in search of a software-defined storage solution that has a robust file system. Most SDS vendors offer one that doesn’t have the critical features organizations need today. Research solutions carefully to make sure you are getting one that can handle object, block and file system storage and has the features needed to handle unstructured data.