Backup Workflow using BitTorrent Sync and Crashplan

Backups! We all need them, we hardly make ‘em!

Over the years, as a digital artist, I have on occasion lost bodies of work. I lost them when building a new PC, I lost them when my drive crashed, fell, burnt, and I even lost them while watching in horror as I mistakenly said yes to “are you sure you want to format drive D”?.

Data keeps piling up, and to keep everything archived we need at least double the amount of drive space.

Luckily I don’t do animation anymore so my projects aren’t that super large anymore, but I have been photographing and editing work for a couple of years now and the data pile keeps growing. Next to that I kept running out of space locally, and kept buying new drives for my Drobo to accommodate my expanding archive. The problem with new drives to store your projects is that you need to copy your data over and over, you get sloppy, you forget projects, you ignore folders, and in the end you lost some precious pictures in the transfer process.

I’m a creative artist, and even though I have OCD tendencies, I get bored with tedious tasks and mistakes are easily made. I needed a better backup plan to safeguard my body of work, without having to spend too much time on this task.

Now there are numerous ways to maintain a backup, but I’m going to share one that has been working for me the past few years, and makes me feel secure about my data.

here’s my hardware setup:

I have a Laptop to carry around and work on. It has all my running projects on it and all my new photo’s are created on its mere 500 GB drive. At home I have a Mac Mini that works as a home theatre pc under my television. I have a Drobo connected to it, which houses a rough 8 TB of space. This Drobo is seen as a single drive and holds all of my media, from movies, music, and all of my photo’s, both personal and my professional work.

The Drobo is almost hassle free when it comes to disks dying and replacing disks with newer and bigger ones. But still, it is a piece of hardware that can (and probably will) die on me eventually.

On the fly backups

So how do I get my data from my laptop to my mac mini/Drobo and from there into the cloud, for off-shore backups.

For this I use a number of tools. Let’s start with data transfer from laptop to mac min/Drobo. I need my pictures folder to always sync back home. Here’s where BitTorrent Sync comes in. This little free app runs on both my laptop as on my mac mini/Drobo. On my laptop I just add a folder, in my case my entire “Pictures” folder. It then asks me if I want the other end to have read permissions or read/write permissions and send links via mail etc. Just leave it and close this dialog.

You can repeat the “preferences thing” for each folder you want to sync.

In the lower region of the preferences pane, is a fly-out menu called “View Key”, click on it and you’re presented by two keys. I choose the Read & Write Key, as this will allow me to change things on both ends.

From here I go to my Mac Mini/Drobo and open BitTorrent Sync.

I’m presented with the same windows, I select the “Options” button (the cogwheel on the right) and click on “Enter a Key”

I fill in the key I copied and I am asked to select an existing folder or create a new one.

I create a new folder on my Drobo inside my main Photo folder and I’m done!

BitTorrent Sync starts looking for files and starts sending them from one end to the other. If I change one thing inside the folder on either end it will update on both sides. So be warned when throwing stuff out!!

How to backup into the cloud?

I need a cloud based backup that is scalable and can grow with me instead of forcing me to delete stuff, at the most inconvenient times. For this I found Chrashplan to be my ideal partner. For a mere $59,99 a year I can have a backup running without limits on size.

Within Crashplan I can select different folders I want to backup from one drive or computer. So what I did was select my main Photo folder (which is different from the one I sync my laptop with) and it done, every time my mac mini/drobo is connected to the internet, crashplan communicates to its servers and syncs all the data to the cloud. If for any reason my house should burn down, I always have my latest version resting safe in an unknown location, from where I can safely restore my files again, once I have a new house, and a new mac mini/drobo 😉

Crashplan even comes with an iPhone app that will allow me to see my backups and even show and download compatible files, like JPG’s of projects I’ve done. This way I can always show clients some of my latest work, even if I didn’t bring my updated portfolio.

Wrapup

Below is a video of the whole setup. It’s a bit less boring than a step by step tutorial because you can listen to me snif. For this video I’ve used a folder with all my PDF’s as an example. And of course all the software has already been installed and is in use.

Daniel

Bittorent for the good of mankind
- dracorubio
  
  sharing your data since the dawn of the net
Sam

Interesting. does Bit Torrent Sync suck your bandwidth when you go into coffee shops?
- dracorubio
  
  there is off course a significant bandwidth usage when BTS is running, but you can either pause syncing, or go into the preferences and change the max. bandwidth for up and downloading
digital2go

I recommend to use Bitcasa. It automatically backups up files to the cloud and I can use it to store TB of files. Much simpler than having to use anything else. Crash plan is good but it is not like a ‘Dropbox’ type of service and there is no retention of files when not using it. Try Bitcasa and you’ll fall in love.
- dracorubio
  
  Bitcasa sounds cool, but to have it hold over 2.5TB of professional data will cost me € 79,- a month, which is more then I pay annually for CP
  - digital2go
    
    Scratch Bitcasa. They stopped offering unlimited. I spoke with a rep at box.com and for $45 USD you can store unlimited data (yes they said unlimited) for your files. They have a free trial and they should work well for you. They also said you could use WebDAV and store your files without syncing so your hard drive doesn’t get full.
Yngve Thoresen

About a year ago I had a drive fail on me. My manual backup routine was, of course, long overdue. Luckily, I thought, I had Crashplan+. But I was wrong. Of about 100 GB of photos, Crashplan only had about 15 GB. The software said everything was backed up, but it wasn’t. I tried contacting support, but they never answered. I lost 3-4 months of photos, including the summer holiday with my family.

I cancelled my subscription and uninstalled Crashplan soon after. I know use JottaCloud, a Norwegian-based company, and regularly verify that my files are backed up.

After reading this post I want to setup a small server and use BitTorrent Sync.
- Tom
  
  Following the “Backup Rule of Threes” is the safest practice:
  
  3 copies of anything you care about – Two isn’t enough if it’s important.
  
  2 different formats – Example: Dropbox+DVDs or Hard Drive+Memory Stick or CD+Crash Plan, or more
  
  1 off-site backup – If the house burns down, how will you get your memories back?
- SufferinJaysus
  
  I use Crashplan with more than 50 business clients. Over 4 years i have performed at 100+ restores, some mission critical, and have never been let down.
  
  This not to negate your experience but it only seems fair to share my very different experience.
Mark Emery

I use rsync on a server at home to copy data up from my two laptops. From there I rsync the data to a third system. Being an IT Geek & Photographer I’ve worked for/with numerous IT companies and used many drives myself from a wide range of vendors. Shoot Google Maps Business Views too and you’ll see the image archive grow at an incredible rate. I’ve been on this rock almost half a century, started using drives 30 years ago, I’ve seen easily 40+ drive failures. I had one fail in my home server only a couple of weeks ago. Thankfully the server was running RAID.

People’s thinking should ALWAYS be WHEN THIS FAILS.. not IF. I agree with Tom, 3 copies is essential for stuff you want to keep, and where possible 1 should be stored offsite. With more and more cloud services being hacked just be careful what you copy where. Also don’t set your backup software to delete the remote copy else you may find yourself accidentally wiping all three copies in one a short period.

As for different media, with the volume of images I’m creating DVD and DAT are too small, the industry really does need a storage revolution. Slower but very high capacity long term storage, preferably that can survive an EMP.
snub666

I think you meant to write “photos” in the two instances you wrote “photo’s.” The former is a plural version of the noun “photo.” The latter is possessive.
Eduardo Tabacman

Maybe I am missing something; but syncing two drives is not the same as having a backup, or is it? If I am syncing drive A and B, and I accidentally delete a file in drive A, I’d have to scramble and disable the syncing, or it will also be gone from drive B!

Sure, syncing can help with a complete drive failure, but a backup it is not.