By Kumar Abhijeet, Business Development Manager, Cleversafe
I was at the Storage Visions conference earlier this month and sat through multiple sessions. Overall, I came away from the conference with three emerging trends:
- Storage requirements in the Media and Entertainment market are growing and different from enterprise storage
- The unsolved problem in long term retention and archiving of data is for the storage system to protect the data from latent failures and operational failures of drives
- Cloud storage is moving from just an idea for organizations, towards real implementations
The first trend is the new storage requirements that are emerging within the Media and Entertainment market segment. If you look at this segment, the market as a whole is moving from SD to HD to 3D HD to 3D-4K. Storage has to manage more and more files, the availability of the system has to be greater than 5 nine’s, archives need to be maintained at the highest resolution( which are typically 20x larger), production times are shrinking, workflows are going tapeless and early films are being digitized and restored at high resolutions. All these trends translate to a good thing for the storage companies.
Conversely, typical enterprise storage requirements are very different from the requirements of rich media content within the Media and Entertainment segment. Enterprise hot buttons are Deduplication, Heterogeneous ILM, Cloud storage etc. Rich media content requires performance for 2K and 4K capture (368MBps per stream, often multiple streams), instant access to archive data (eliminating tape for news, TV episodes, sports, pre-recorded live), graphic representation of millions of video and audio assets and repurposing duplicated assets in various resolution and formats. When we look at these requirements against Cleversafe dispersed storage capabilities, we believe Cleversafe’s dispersed storage fits very well within the storage workflow as a nearline storage to the front end applications such as AVID, Final Cut Pro, Sienna TV, Artesia etc. Media asset management application such as Front Porch Digital and Tedial are also going to be an important piece of the work flow. In order to meet the ingest and retrieve throughput requirements there is definitely a play for Flash and SSD in front of a reliable, secure and cost effective nearline geographically dispersed storage cloud. The geo-dispersed nature of dispersed storage provides protection from data loss and provides continuity of operation even when a site goes down without the need for replication.
The second trend I noticed was the discussion around the unsolved problem in long term retention of mission-critical data. SNIA 100 year task force recently released 100 year archive requirements survey report which highlighted that long-term digital information retention are real. 68% of respondents said they keep the data for more than 100 years and about 80% of respondents have information they must keep for more than 50 years and 70% of respondents say they are ‘highly dissatisfied’ with their expected ability to read their retained information in 50 years. The goals of digital preservation are that digital content should remain accessible, usable, and undamaged for as long as desired and beyond the lifetime of any storage technology at an affordable cost. All disk-based storage systems are going to be facing a challenge of how to protect large amounts of data against latent failures. Disk capacity is becoming larger and larger according to Moore’s law (we already have 2TB drives coming out), but Unrecoverable read errors (URE) is fixed at about 1014, meaning once every 1014 bits, there will be a bit that is unrecoverable. Doesn’t seem significant? In today’s large storage systems, it is. Unfortunately, the likelihood of having one drive fail, and encountering a bit rate error when rebuilding from the remaining RAID set is highly probable in real world scenarios. To put this into perspective, when reading 10 terabytes, the probability of an unreadable bit is likely (56%), and when reading 100 terabytes, it is nearly certain (99.97%). Cleversafe is the leader in protecting the stored data from data loss or corruption due to URE’s, disk failures and correlated disk failures by in implementing advanced erasure codes in storage software to protect from data loss. My colleague has written a very detailed blog on how Cleversafe protects the data from latent failures: http://dev.cleversafe.org/weblog/?p=259
The third trend is around the reality of cloud storage – and this is what got me really excited. Come to think of it, unlike Internet or Web2.0 based applications, cloud storage is one of the services for which end consumers, small, medium and large enterprises are actually spending money. There were a lot of numbers thrown around during the conference, but according to iSuppli, the cloud storage market will triple in three years from $1.6B in 2009 to about $5B in 2013. Amazon is the leader in providing storage services today, but when the telco operators such as AT&T, Verizon, Comcast, Savvis and others enter the cloud storage space, this will be an exciting space to be in. There are a multitude of vendors that are emerging in this space. EMC Atmos and Cleversafe are emerging as end to end cloud storage platform vendors. Parascale and Mezeo are emerging as software only vendors. I participated in a panel discussion on this topic along with IBM, Bycast and Asankya. The panel was hosted by Robin Harris. Robin has published his take on his storagemojo blog: http://storagemojo.com/2010/01/13/cloud-at-storage-visions-2010
Overall, I did not see a cloud storage provider who provides protection from data loss without replicating data objects. This is where Cleversafe fits very well. Cleversafe’s dispersed storage provides the data protection without the need for replication, hence reducing cost for the service provider and end consumers. Cleversafe’s object based dispersed storage technology provides for a platform that addresses the challenges of cloud storage: scalability, privacy, data protection, system uptime, data security / privacy and easy manageability.
Keep an eye out for the cloud storage market to evolve further in 2010. And I am excited to be a part of this evolution.
