You are here

Object Storage and Erasure Coding - What’s It All About?

Written by admin Category:

Running a business that specialises in the development of Cloud infrastructure services we see a myriad of technical and business requirements from our customers.  By far and away the most common area for businesses to consider first when looking at Cloud migrations (and indeed the biggest area of pain in terms of spiralling costs) is their storage infrastructure; whether that's primary storage, archive or backup.

If you've considered the use of the Amazon S3 service at all, or in fact any public cloud storage infrastructure, then you'll be aware Object Storage, but maybe less so erasure coding.  This post is written to give you an introduction into both as well as giving our view on why Object Storage will become the defacto standard for storage.

So, lets get started, what do we mean by the term Object Storage?

Object storage is a storage architecture that manages data as objects, as opposed to other storage architectures like file systems which manages data as a file hierarchy and block storage which manages data as blocks within sectors and tracks.  Each object typically includes the data itself, a variable amount of metadata, and a globally unique identifier. 

Object storage seeks to enable capabilities not addressed by other storage architectures, like interfaces that can be directly programmable by the application, a namespace that can span multiple instances of physical hardware, and data management functions like data replication and data distribution at object-level granularity.

And what's erasure coding?

Storage bits fail, and as a result storage protection systems need the ability to recover data from other storage. There are currently two main methods of storage protection: Replication and RAID.

Erasure coding is the new kid on the block for storage protection. It started life in RAID 6, and is now poised to be the underpinning of future storage, including cloud storage.

An erasure code provides redundancy by breaking objects up into smaller fragments and storing the fragments in different places. The key is that you can recover the data from any combination of a smaller number of those fragments.

It can be shown mathematically that the greater the number of fragments, the greater the availability of the system for the same storage cost. However, the greater the number of fragments that blocks of data are broken into, the higher the compute power required.

Hopefully these explanations are making sense for you, but the big question is why should you care?

Well, it may sound complex but ultimately the use of Object Storage and Erasure Coding boils down to some key wins for IT and the business:

  • Fewer copies of your data to manage (the same applies to SWIFT Openstack too)
  • Greater data availability 
  • Lower cost through reduced footprint 
  • Increased ability to scale
  • An API driven approach enabling tighter integration into business applications

In our opinion the traditional logo's you'd associate with building SAN infrastructure today are playing catch up.  There are a new breed of storage vendors arriving on the scene that are leaner, more cost effective and more scalable in their approach.  For those customers in the sub 300TB range Public Cloud storage can certainly be an attractive prospect to make use of these shifts in technology, and for those organisations with greater storage volumes new Object Storage vendors give the opportunity to deliver the same elastic storage capabilities, at the same (and in some cases better) cost points in the confines of your own network.

As John F Kennedy once said "change is the law of life.  And those who look only to the past or present are sure to miss the future".  Object and Cloud storage will be a different approach to that which you're used to but they will be the future (at least we think so!) it really is a game changer for those businesses willing to make the leap.

 

About Nephos Technologies

Nephos Technologies is an independent Systems Integrator for the Next Generation Data Center; providing consultancy, management and technology services to organisations seeking to utilise cloud technologies and economics, whether that is on-site, off-site or a hybrid based approach

 

Keep In Touch

  •   [email protected]
  •   +44 8453 104 105
  •   Central Working, 2 Kingdom Street, London, W2

News And Events

London 01.03.2017 - Nephos Technologies, a leading independent provider of next generation datacenter infrastructure, has been selected by UK architectural practice, Glenn Howells Architects (GHA), to deliver a next generation infrastructure, based on hyper-converged, Cl

LONDON 06.02.2017 - Nephos Technologies, a leading independent provider of next generation datacenter infrastructure, has been selected by international engineering firm, Holmes, to deliver a centralised, Cloud-integrated storage system to enable global collaboration.

IRVINE, CA– August 20, 2015 (GLOBE NEWSWIRE) AllDigital (OTC:ADGL) is pleased to announce that its video workflow product, AllDigital Brevity, will be on display at IBC-2015 in Amsterdam from September 11th to 15th at the Micro

Nephos Technologies has been Shortlisted by CIO Review for the 20 Most Promising Storage

London, LDN (April 25th, 2013) - Nephos Technologies, a leader in Cloud Services Brokerage, is proud to announce that it has been included as one of only four “Cool Vendors” by leading analyst Gartner in their “Cool Vendors in Cloud Services Brokerage, 2013” report published on 18 April 2013 by Tiffani Bova, Daryl C. Plummer, et al.

Nephos Technologies are hosting an exclusive breakfast roundtable with our partners Morpheus Data, to discuss Cloud management, orchestration and automation and how together, we can overcome some of the challenges that arrise from trying to deliver a Cloud first strategy.

File data continues to grow and spread across the enterprise creating a massive file sprawl problem. Until now, legacy technologies like distribution, replication, WAN acceleration and storage tiering have been used to deal with the problem – with limited success and at great expense. 

Nephos Technologies, along with our partner Cloudian, held a brief webinar to introduce the Cloudian Hyperstore platform, and to introduce the use of object storage as a platform to support the exponential unstructured data growth faced by many of todays organisations.

File data continues to grow and spread across the enterprise creating a massive file sprawl problem. Until now, legacy technologies like distribution, replication, WAN acceleration and storage tiering have been used to deal with the problem – with limited success and at great expense.

Live Webinar | Tuesday, 29 June | 10:00 a.m. GMT

Nephos Defined: Get Hands on Experience Delivering Enterprise File Services in Microsoft Azure

Unstructured data is being generated at exponential rates, generating real value for those that can analyse it, with the compute power needed to process it rising too.  This combination makes it vital to find new, more scalable ways of storing and protecting that data.

Join Nephos Technologies and some of our leading partners at a half day event where we'll be discussing the evolution of the datacenter, and how new disruptive technologies are giving enterprise organisations a more efficient and sustainable datacenter infrastructure.