Saving data can be cheap, Part 2.

This is part 2 of my data storage talk. Make sure you see part 1 here.

Comments

[Justin Lomax]

"wicked, wicked cheap" =)

[Rob Shaver]

Wow! I think you've really presented a confused mix of apples and oranges here. First let me say that I've been doing exactly what you suggest using the "toaster" style connection box, so I agree this is a good way to save your data. But your comparison of the various "drive containers" seems to leave out the advantages those other methods offer over what you propose.

I'm just going to talk about the Drobo because I think you gave it very short shrift. I'm going to use the same model you did (I think) but starting with the sans drives version. So the right column of your table should contain $1495. Next I'd populate it with eight 2 TB drives, price from Amazon: $100 each X 8 = $800. Total cost $1495 + $800 = $2495 for 16 TB of raw storage. Using the Drobo capacity calculator for single disk recover, this gives us 12.48 TB or or $200 per TB.

So this is about a third less than your figure of $337 per TB but prices have dropped on HDs so that explains the difference. I have no particular problem with your figures.

My complaint about your tutorial is that you neglected to mention the huge advantage you get for the extra money.
A. Data is on-line: Your storage is on-line and you may be able to edit directly from the Drobo.
B. Data loss protection: If a single drive in the Drobo fails you loose no data.

With your system if you have a single drive failure you loose everything on that drive (assuming you deleted the data from your working drive to make room for another project).

So your article is about how to get the cheapest off-line storage, which is exactly what you said it was in part one. I just think the other factors are important enough to get more mention.

As far as hard drives being used for archiving, the jury is still out in my opinion. Many years ago I worked in the 9-track tape drive industry. That format was good for about 18 years, if stored in the proper environment, before the recorded data faded beyond recovery. The new hard drives are using vertical recording to pack this huge amount of data into smaller and smaller physical areas.

I've searched for data about this but haven't seen any longevity studies of how long the drives can see on the shelf before they become unreadable. But if I had to guess, I'd say that the higher the density, the shorter the shelf-life. So I'm not all that reassured by your anecdote about reading data from old (low density) hard drives successfully.

Anyway, you offer sound advice with these caveats.

Peace,

Rob:-]

[Chris Fenwick]

Hey Rob... you comments are all good.... yea,, my goal was inexpensive offline storage. I fully 'get' the benefits of the Drobo technology and yea, these number are probably all lame by now anyway. However... do you recommend routinely pulling one set of drobo drives out of a drobo and replacing them with a second set? I doubt the good people at Data Robotics would recommend that practice, (not like they are gonna recommend my practice either....) So, if you can't swap the drive sets on your drobo then really... you are buying ONE drive enclosure and ONE drive set. Where as, in MY system you by one enclosure and as many mechanisms as you want.

if you get to the part about Disk Tracker software it will explain that although this data is "offline"... its never more than a few minutes away from being found and collected if need be. I shared the example of being able to retrieve one layer from an AI file that I hadn't touched in 2 years and I was able to find it and GET it in less than 5 minutes...

I agree fully agree with all your comments...

my method is geared at the person that just can't afford to go to the store and by YET ANOTHER hard drive...

oh...and TWO copies, in TWO locations... definitely a good idea.

[Rob Shaver]

I guess my preference is to have a RAID 5 or a Drobo AND then use your raw drive method to back up to two separate off-line drives. Right now I have four one-terabyte drives in my Mac Pro plus two one-terabyte stand-alone FW-800 drives. Then I've got four 750 GB raw drives I pulled out of my Mac Pro when I put the one-TB drives in. I use those to back up project files and raw data that's not on tape, now that my camera captures to flash.

When I used to acquire footage on DV tape I considered that my back-up. Now I've got to reuse the flash drives so that means backing up the raw footage on a hard drive somewhere. Typeless acquisition is a blessing and a curse.

I see no reason to pull the drives out of the Drobo but I guess it wouldn't hurt to replace one each year. However, you're vulnerable during the rebuild time if a disk fails you could loose the whole data-set.

The Drobo does have a dual-disk redundancy mode which will protect against any two disks failing at the same time. This reduces the capacity of the raw 16 TB to 10.89TB. (There is evidence that disks from the same manufacturing batch have a higher likelihood of failing together.) If I was the DIT on a hight-cost production, I think I'd use that mode.

The Drobo is the only RAID box I've seen that lets you add drives to a live system and even drives with different capacities. It seems quite flexible.

I think the Disk Tracker sounds like a terrific idea. I can barely find the files on all of my on-line drives much less the ones that I've switched off or are on the shelf.

I'm looking forward to hearing more about your experiences with Adobe products. I've been thinking of taking the plunge but then I think maybe I should switch back to Windows hardware too. I do love the Apple hardware but it is twice the cost if I build out my own PC. Windows 7 64 bit seems almost on a par with OSX now.

Peace,

Rob:-]

[Ambarish]

Hi Chris,

First off, you have a treasure trove of useful information so I have to thank you in advance. I have a question about the reliability of the Weibetech enclosure you're using. Is it something you edit on, or simply use as backup?
I'm an editor who is quickly being confronted by the necessity of an efficient and cheap storage-edit solution. Would you reccomend the Weibetech as an edit solution using esata?

Thanks,
Rish
from France

[Chris Fenwick]

Rish... We often edit on the Weibetech enclosure, either via eSATA or FW800. I prefer NOT editing on the backed up media just for the sake that I don't want to over use the drive too much once we "put the data away". However, often we will pull a project 'out of storage' and do a little work on it. The Weibetech enclosures perform just as well as any other drive enclosure...

[Ambarish]

Thanks Chris, that helps me out a lot. I'm looking for a good way to manage backups and current edit projects. Your recommendation, really your whole system with the storage boxes, disktracker and the works, makes a lot of sense for me.

Best,
Rish

[A Photog Who Blogs]

This site has been super helpful.

I like what Disktracker does, but dude, that interface is way OS9! I googled and am trying out CDFinder: http://www.cdfinder.de/en/downloads.html which seems to be legit. Demo gives you up to 25 volumes to catalogue free.

[Dave]

Chris, I've listened to all the DCPodcasts...95? and feel like we're old friends. Thanks for all the great ideas. Now that SSD drives are coming down in price and we have the added feature of TB. Have you changed your ideas at all about the back up process.

Is it possible to build a RAID 5 inside the MP with the available HD bays? Or do you need to use a third party product like PROMISE or DROBO.

The techies at LA CIE said that the one advantage to using a 7200 RPM disk is that its cheaper to recover vs an SSD (which is very costly...Thanks again.