What do you do with 300 million photos a day when their “shelf-life” in terms of viewer interest is no longer than a few weeks? You freeze them in what is increasingly being referred to as “cold storage” in the cloud space.
That is what Facebook is exploring according to this InfoWorld article. The article goes into Facebook’s explanation for why cold storage of photos, and from what I understand it comes down to a couple key reasons: they cannot simply delete them and they cannot afford to keep them (on their existing storage infrastructure). Facebook, like any cloud company, or any company for that matter, amasses a huge amount of data on a daily basis. Tiered storage and auto-tiering has taught us to keep the most important data in higher speed storage like SSD or 15,000 RPM HDDs, and move the less frequently accessed data to higher-capacity, lower-cost media like 7200RPM HDDs.
But, even higher capacity, lower cost media like 7200RPM HDDs are overkill for files that may never get looked at again, or at least it may be months, years before they are accessed. When was the last time you looked at the first few Facebook photos or albums you uploaded? For me, it’s been a couple years. Hence, this idea of cold storage. Cold storage is nothing more than high-capacity, lower-cost, and even lower-power storage designed, for the most part, to sit idle until the mother ship comes calling for one of those old photos, or files.
Facebook’s strategy is to utilize their software layer to, first, manage what gets placed in cold storage, the energy it takes to run, and the time it takes to retrieve files. I’m sure there is more to the system, but for simplicity’s sake, let’s say these are the top 3 design considerations. And this makes sense, if you consider what Frank Frankovsky, the ex-Dell man who oversees hardware design at Facebook, said about designing an open storage architecture in this Wired article on Gizmodo:
“We’re taking the same approach we took with servers: Eliminate anything that’s not directly adding value. The really valuable part of storage is the disk drive itself and the software that controls how the data gets distributed to and recovered from those drives. We want to eliminate any ancillary components around the drive – and make it more serviceable.”
The challenge in front of hard drive suppliers like Seagate is how to develop products for an increasing cold storage usage model? No doubt, Amazon’s Glacier and Facebook’s move to cold storage are indications that a growing trend to store massive amounts of seldom-used, but always-needed data will demand a different type of storage device. A storage device with the highest possible capacity and the lowest possible power envelope that seamlessly integrates with existing cloud or traditional IT storage architectures.
When’s the last time you looked at your first Facebook photos?