Learning the digital darkroom – This just in, RAID is NOT a backup system
Posted on 02. Feb, 2009 by Paul Burwell in Equipment, Everything
Don't keep your head buried, back up your images! (American Avocet searching for food)
Learning the digital darkroom – This just in, RAID is NOT a backup system - Surf the various boards on the Internet long enough and you’re bound to stumble across a conversation or two about disk storage. The great thing about digital photography is that you get to experiment without a lot of the cost. The down side is that those digital photos start taking up a lot of disk space. All disk drives will fail. They are mechanical devices and it is only a matter of time. When you realize this and you start getting serious about your photography to the point that you don’t want to lose your images, you start to investigate backup systems.
When people start investigating ways of protecting their files (remember that in this digital world that images are files too), they will eventually come across the concept of RAID (redundant array of inexpensive disks) disk drive sytems. And it’s that redundant word that causes all the confusion and leads people to believe that RAID is some sort of backup system.
My definition of a backup system is a set of methods and procedures that allows you to recover file(s) from a catastrophe. And some of those catastrophes could include drive failure, flood, fire, earthquake, theft or even the accidental or inadvertent deletion of a file or bunch of files.
And it’s in meeting this definition that a RAID system fails as a backup method. RAID systems were initially developed for businesses to keep them running in the event of a drive or even disk controller card failure. Take for instance a retail business with an inventory system online that needs to provide real-time inventory information. Without a RAID system, a drive failure would take the inventory system down until the problematic device was replaced. With a RAID system in place, a drive could fail and the inventory system would keep running. Then in the off-hours, the system could be brought down in a controlled manner and the offending device replaced. And RAID drive systems continue to work great in this regard.
So, we’ve discovered that a RAID system will probably let you recover from a filed drive, but will a RAID drive system help recover the data if the drives are six feet under water or destroyed in an earthquake? Will it help recover the files if all the equipment is stolen? Will it help you recover that folder of images you inadvertently permanently deleted? It won’t, and that’s the problem when people use RAID drive systems while under the impression that they provide true backup services.
I try to regularly backup my data files. For the sake of this discussion, let’s say I have my operating system on one drive and my image files on antoher. I backup my image files daily to a completely seperate drive. Weekly, that backup is backed up to yet a third drive which is then stored securely off site. (In reality I have four of these off-site drives so that I’ve got backups going back just over a month). I don’t regularly back up my system drive (where the operating system and applications are stored) because in the event of a catastrophe, I have all the pertinent information stored securely off site so that I could reinstall the operating system and programs I reguarly use. I do use a form of RAID for my system drive, disk mirroring that allows my system to keep running in the event of a drive failure. So, let’s see if the system I’ve described passes my definition of a true backup.
My backup system allows me to recover from a catastrophe. If the entire system in my home office is some how lost or destroyed, at most I’ve lost a week of work and/or images. If I wanted to be more paranoid, I could increase how often I update my off-site storage or move to several off-site storage devices. If I delete a file or folder and notice right away, it is available from my on site backup drive. If it is longer than that before I notice the deleted folder, I can retrieve it from my off site storage device. And so, I’ve covered off all the contingencies and I can recover from a catastrophe and lose, at most, a week’s worth of work.
At the present time I physically store my backup drives off site. I use a bank’s safety deposit box as my storage place. I figure the odds of both my office and the bank being destroyed in some sort of catastrophe are remote enough that if it does happen, getting my files back will be the least of my concerns. I pay less than $20 a month for this relatively secure storage.
I dream of the day when I can store my important files and images off site through the Internet. But at the speeds most of us connect to the Internet today, that day is a long day off. Let’s imagine a user with 500GB (gigabytes) of images they want to back up somewhere. They have a fairly good Internet connection whose upstream bandwidth (the speed it sends information to the Internet) is about 512Kbps (kilobits per second). Even if they found a online storage service that could receive their data at 512Kbps, it would take over 100 days for the data to transfer assuming no interruptions. And in those hundred days, how many more gigabytes of images would the photographer have to backup? I think you can see that for many active photographers this makes today’s online storage systems a non factor for all but a very few images or files.
I implore you to develop a plan for backing up your images. I’m not saying there’ll come a point in the future when you’ll thank me. I’m just sayin.
Do you have some ideas about backing up your images and making them secure? Please share them here.
![[del.icio.us]](http://www.paulburwell.com/blog/wp-content/plugins/bookmarkify/delicious.png)
![[Digg]](http://www.paulburwell.com/blog/wp-content/plugins/bookmarkify/digg.png)
![[Facebook]](http://www.paulburwell.com/blog/wp-content/plugins/bookmarkify/facebook.png)
![[Friendsite]](http://www.paulburwell.com/blog/wp-content/plugins/bookmarkify/friendsite.png)
![[MySpace]](http://www.paulburwell.com/blog/wp-content/plugins/bookmarkify/myspace.png)
![[Sphinn]](http://www.paulburwell.com/blog/wp-content/plugins/bookmarkify/sphinn.png)
![[StumbleUpon]](http://www.paulburwell.com/blog/wp-content/plugins/bookmarkify/stumbleupon.png)
![[Technorati]](http://www.paulburwell.com/blog/wp-content/plugins/bookmarkify/technorati.png)
![[Twitter]](http://www.paulburwell.com/blog/wp-content/plugins/bookmarkify/twitter.png)
![[Email]](http://www.paulburwell.com/blog/wp-content/plugins/bookmarkify/email.png)
13 Responses to “Learning the digital darkroom – This just in, RAID is NOT a backup system”
Leave a Reply
Alltop
Adam
02. Feb, 2009
I use a Drobo that sits next to my main computer in my editing room and have external hard drives off site that I bring home once a week to sync. REALLY important data is on more than one external drive in the off site location and I also carry smaller external drives back and forth that hold really important data. If I take a picture that I feel is really something special I will import it and back it up to the smaller drives before leaving the location that I shot it at. The one night a week that the externals are home is the only flaw in the plan at this point.
Paul Burwell
02. Feb, 2009
Thanks Adam. Sounds like you’ve got a good handle on protecting your images.
Michael Dao
02. Feb, 2009
With the prices of hard drives coming down, and with advances such as Time Machine from Apple, and Drobos from Data Robotics, a lot of people are being smart and backing up their stuff onto separate media. However, this is only the first step. The next one is getting an offsite backup. Some people are using Amazon’s S3 service, others are using Mozy. There is a new one out now called BackBlaze. They charge five dollars a month per computer for unlimited storage space. Their software will do a complete backup, and will also do incrementals. Now here’s where they shine. Say you do suffer a disaster and need your files. You can download them for no additional charge, or for 99 dollars they will send you a set of DVDs and for 189 they will send you a USB drive with it.
Paul Burwell
02. Feb, 2009
Mike,
I think what you say is a great idea for small collections, but for large collections (I have over 1.5TB of images and it grows weekly if not daily), it just isn’t practical for someone making a lot of images.
Like I mention in the article, uploading just 500GB of images, will take well over 100 days at what for most people, is an unachievable upload speed.
If you someone could ever even have a real T1 equivalent, it becomes a lot more feasible. But until then, I think the average person is better to rely on themselves and their own backups then on some of the online solutions that are great for documents and spreadsheets, but less useful for digital photo collections.
Jeff
02. Feb, 2009
I agree 100% that raid is not backup. It’s a good hedge and can save a bunch of time, but for safest backup, you really should consider online backup at places like http://MyOtherDrive.com or Carbonite. Storing your files offsite protects not only against natural disasters, water damage, but also against theft. And more important you can begin to share your files with others once they are stored online.
Sean Phillips
03. Feb, 2009
I have a very robust backup system involving multiples copies of my files in my house, AND at least one offsite copy at all times that I have manually backup (ie. go get the drive, back it up, and return it to the local offsite storage location).
I have been thinking for awhile about how to use an online offsite backup system, and had pretty much decided to work out a way to store a NAS server at my fathers house (ie. remote offsite copy, in a different province) that is online at all times.
The advantage of doing so is that I could seed the drives (ie. create the initial backup copy) at my house, and then take or send the NAS to his house. Incremental backups should be feasible to do over the internet.
The problem was that I hadn’t found an easy software solution to make this happen. I just found Crashplan at crashplan.com and I definitely plan to give them a try. If it works this would be much better than something like S3 or Backblaze, which could take weeks to seed and/or recover from. If I ever had a problem with the multiple copies at my house AND with my local offsite copy then I could have my remote offsite copy delivered overnight and pretty much be immediately back in business.
This solution is not without its own challenges, but it looks like a very good option for what I want to do…
Paul Burwell
03. Feb, 2009
Sean,
That sounds like a really interesting solution. As long as both you and your father have reasonable Internet connections (in terms of speed and reliability) it sounds quite feasible.
Sean Phillips
03. Feb, 2009
Yep. I haven’t test it yet, but I’ve heard of many other people doing something along these lines, so it should work. The pre-seeded drive is key though, and there’s no reason why it has to be a relative either. If you have an office or a good friend, they could house your off-site storage instead…
Stephen
05. Feb, 2009
I have a couple of Drobos that I use for local backups and storage. I also store External Drives in the safe at my parents place. Another option for critical data is that I use Jungledisk and the Amazon S3 service. Monthly storage is very cheap .15 per GB but it does cost putting stuff there and accessing, but it is still reasonable.
Steve Fogarty
07. Mar, 2009
I’m giving off-site network backup another try. Don’t tell my homeowner’s association, but (for reasons unkown) they’ve recently bumped up our bandwidth. Download is about 22mbps, upload is about 10mbps. I tried Mozy long ago and experienced what Paul talks about – never able to catch up. Mozy is right here in Utah, but I couldn’t talk them into letting me leave a seed drive with them.
I just downloaded BackBlaze, gonna try it for a couple weeks, see how far it gets with my terabytes of data …
BTW, RAID5 is not failsafe either. I had a nifty 2TB software RAID5 (Windows Server 2003) array, that I religiously backed up to a second 2TB RAID5 array. To fail, I would have had to lose 4 disks at the same time. What could go wrong, right?
I lost 4 of the 8 disks in one week. Lost a couple year’s worth of photos & data.
Recovery shops were talking $10-$15k just to look at it, no promises.
I’ve kept the 8 disks (4 good, 4 bad) wrapped in bubble wrap, waiting for the day I become fantastically wealthy.
Paul Burwell
07. Mar, 2009
Steve,
I’d kill for upload speeds of 10Mbps. With my Internet service, I get about 512Kbps. 10Mbps makes it entirely feasible I think.
Steve Fogarty
07. Mar, 2009
Just started backing up with BackBlaze. Interface seems much simpler, less options, than Mozy.
It estimates my upload at 7mbps, not the 10mbps I get from speedtest.net.
It’s first estimate for the initial upload was 727 days!
I then bumped the upload slider to maximum, and excluded some redundant directories.
It now estimates 22 days for initial upload.
it could work. Unless my HOA taketh away what it hath given …
Online Backup
22. Jul, 2009
Interesting article. Backups are more important today then they have ever been in my opinion, if you loose your data and you cant recover it then your business is dead. Thanks again.