Dispersed Storage Blog

Today, Cleversafe announced the dsNet™ Object Store. The dsNet Object Store is targeted towards large content stores for digital content such as video, images and audio files.

What is object storage?
Object storage splits the metadata from the content and stores the two on different servers. Typically the metadata is stored by an application that is optimized for the digital content type. The application handles creating, accessing, and deleting objects from the dsNet Object Store, which utilizes Dispersal to store data with increased storage efficiency, security, and data protection.

Why is object storage important?
Object storage can scale larger than traditional block or file based storage systems. An object store is independent of a computer’s operating and file systems, so bypasses their limitations in terms of scale. Clearly the associated application needs to know how to handle the object, but it doesn’t need to use the file system for searching, sorting, or accessing objects.

When the majority of storage needs were structured data (typically textual based), the information found in the file system’s metadata (file name, creation date, folder location, etc.) was sufficient to efficiently locate information. Today however, file systems are proving less relevant for digital content storage since the metadata for objects is typically housed in specialized applications versus file systems.

A simple example is managing photos from a digital camera.  Users will not use the file system to find photos since they are stored by computer-generated names from the camera, and the photo application allows them to see, sort and tag photos much more effectively. Further, users aren’t sorting digital content into folders within the file system, instead they are using applications to make collections and relationships between content. So the actual storage of the objects can be a single flat container instead of requiring a tree-structure of a file system.

What are the interfaces and use cases?
The dsNet Object Store has several interfaces and access protocols. The underlying dsNet storage pool can be shared and jointly accessible by multiple access protocols, giving additional flexibility.

For use cases where an application is housing metadata, and only object storage is required, the Simple Object interface can be accessed with either a Java SDK, or an HTTP/REST API. In this use case, simple PUT, GET, DELETE commands are used to access Objects, and the resulting Object ID can be stored directly within the application.

For use cases where a file system is desired to store the metadata (versus within the application), or where integration is not desired, a file system interface on top of the Object Store can be accessed via dsFTP, a software client which can read or write files from and to the dsNet Slicestor appliances, without needing an Accesser appliance. dsFTP provides an interface similar to other command line FTP programs, with both an argument-based and interactive sessions.

Cleversafe’s still offers Block Storage using standard interfaces such as iSCSI and NFS for use cases where standard storage interfaces are required.

What storage appliances does the dsNet Object Store run on?
dsNet Object Store and it’s associated storage vaults can be built and managed using Cleversafe’s existing products. Slicestor® appliances, which act as storage nodes in a dsNet system, are configured as either block or object devices. (Existing block Slicestor appliances can be redeployed as object devices.)

Once the Slicestor appliances are configured, they are available to be provisioned within vaults (like a LUN). Vaults can either be configured block vaults or object vaults, and leverage the associated Slicestor appliances. Cleversafe’s Omnience™ Storage Management can manage both block and object vaults under the same unified management system.

When directly integrated into applications, the Object Store can be accessed from the software client versus through the Accesser appliance. This provides greater scalability in performance since there can be a plethora of simultaneous readers and writers, not dependent on using a gateway appliance.

Conclusion
Cleversafe’s unique combination of object storage and dispersal allows for both unlimited scale, which is crucial as unstructured content continues to expand, and superior data protection. It’s a viable method for building large-scale systems in the multi-terabyte to petabyte range.