Running a business that specialises in the development of Cloud infrastructure services we see a myriad of technical and business requirements from our customers. By far and away the most common area for businesses to consider first when looking at Cloud migrations (and indeed the biggest area of pain in terms of spiralling costs) is their storage infrastructure; whether that's primary storage, archive or backup.
If you've considered the use of the Amazon S3 service at all, or in fact any public cloud storage infrastructure, then you'll be aware Object Storage, but maybe less so erasure coding. This post is written to give you an introduction into both as well as giving our view on why Object Storage will become the defacto standard for storage.
So, lets get started, what do we mean by the term Object Storage?
Object storage is a storage architecture that manages data as objects, as opposed to other storage architectures like file systems which manages data as a file hierarchy and block storage which manages data as blocks within sectors and tracks. Each object typically includes the data itself, a variable amount of metadata, and a globally unique identifier.
Object storage seeks to enable capabilities not addressed by other storage architectures, like interfaces that can be directly programmable by the application, a namespace that can span multiple instances of physical hardware, and data management functions like data replication and data distribution at object-level granularity.
And what's erasure coding?
Storage bits fail, and as a result storage protection systems need the ability to recover data from other storage. There are currently two main methods of storage protection: Replication and RAID.
Erasure coding is the new kid on the block for storage protection. It started life in RAID 6, and is now poised to be the underpinning of future storage, including cloud storage.
An erasure code provides redundancy by breaking objects up into smaller fragments and storing the fragments in different places. The key is that you can recover the data from any combination of a smaller number of those fragments.
It can be shown mathematically that the greater the number of fragments, the greater the availability of the system for the same storage cost. However, the greater the number of fragments that blocks of data are broken into, the higher the compute power required.
Hopefully these explanations are making sense for you, but the big question is why should you care?
Well, it may sound complex but ultimately the use of Object Storage and Erasure Coding boils down to some key wins for IT and the business:
- Fewer copies of your data to manage (the same applies to SWIFT Openstack too)
- Greater data availability
- Lower cost through reduced footprint
- Increased ability to scale
- An API driven approach enabling tighter integration into business applications
In our opinion the traditional logo's you'd associate with building SAN infrastructure today are playing catch up. There are a new breed of storage vendors arriving on the scene that are leaner, more cost effective and more scalable in their approach. For those customers in the sub 300TB range Public Cloud storage can certainly be an attractive prospect to make use of these shifts in technology, and for those organisations with greater storage volumes new Object Storage vendors give the opportunity to deliver the same elastic storage capabilities, at the same (and in some cases better) cost points in the confines of your own network.
As John F Kennedy once said "change is the law of life. And those who look only to the past or present are sure to miss the future". Object and Cloud storage will be a different approach to that which you're used to but they will be the future (at least we think so!) it really is a game changer for those businesses willing to make the leap.