How I store and backup my photography

Jan 11, 2020

Francesco Carucci

We love it when our readers get in touch with us to share their stories. This article was contributed to DIYP by a member of our community. If you would like to contribute an article, please contact us here.

How I store and backup my photography

Jan 11, 2020

Francesco Carucci

We love it when our readers get in touch with us to share their stories. This article was contributed to DIYP by a member of our community. If you would like to contribute an article, please contact us here.

Join the Discussion

Share on:

 

Storing and backing up efficiently and securely more than ten years of photography is a complicate technical task.

At the center of my setup I use a Synology DS1515+ NAS, with 8 disks configured in Raid 6 for a total of 14.54TB of available storage.

I wrote about this set up a year ago, but I recently updated it to be more streamlined and secure.

The Adobe Ligthroom catalog I use to organize and edit all my images must be stored on a local drive and not on a network drive.

This is a limitation of Lightroom that I have to work around.

I still want full automatic back up of the catalog and hourly snapshots to be able to quickly recover mistakes or corruption. To store and synchronize the catalog, I use Synology Drive, a software similar to dropbox. The files in a Synology Drive are stored locally, but they are automatically synced to the remote NAS as soon as there is any change.

All recent versions of the catalog and images are also stored on the NAS and can be easily recovered when necessary.

Another advantage of this setup is that I can share my Lightroom catalog with any machine on my network, be it my Macbook or iMac: in other words I can import images on my Macbook, edit on my iMac and eventually go back to my Macbook to prepare a prepare a print.

All changes are shared automatically across every computer.

It also means, for example, that I can import images from the camera on my Macbook while on the road and eventually see my images replicated on the NAS as long as I have an internet connection or as soon as I come back home.

Handy!

And automatic.

My older images that I rarely access are copied to a shared network folder that is only stored on the NAS (and not locally).

I can still access this network driver from anywhere in the world across the network in case of emergency. It’s obviously much slower than locally available files, but I don’t need to use precious local disk space.

All remote communication goes through an encrypted HTTPS channels, which is very secure.

Both the dropbox-like folder and the shared network folder are snapshotted every fifteen minutes by the Synology NAS: in other words I can recover anything image older than fifteen minutes up to a year.

Snapshotting has saved me more times than I can count at the expense of some limited NAS storage (the system is very efficient).

Last month, for example, Lightroom found that my catalog was corrupt and refused to open it. Ten years of carefully tagging and annotating every image gone. I immediately reached to the versioning database, but I couldn’t find a good version of the catalog to roll back to.

Snapshots to the rescue: I recovered a snapshot of the catalog from the day before that I knew was safe, restored it and Lightroom was again happy.

I had two more levels of redundancy beyond snapshots, just in case.

In fact, both the dropbox-like folder and the shared network folder are backed up automatically every night on an external USB disk connected to the Synology NAS, with full history. Should anything happen to the disks on the Synology NAS or should I make any mistake, I can recover images up to a year old from the external USB backups.

All my images are also constantly backed up automatically in the cloud.

I use Amazon drive which is free for Prime users: when any image is automatically stored on the Synology NAS, the images are also automatically copied to the cloud. along with the Lightroom catalog.

I must stress the point that, once set up, all the redundancy happens automatically without any intervention from me.

I use Lightroom as usual with the only caveat of closing it once the work is done and copying old images to the shared network folder once a year.

Lightroom handily keeps track of the location of all my images and lets me search by tags when needed.

I recently reworked an Image from Iceland I took in 2010: the raw file was stored on the server, indexed by Lightroom and readily available to me.

Having the peace of mind that I can recover all my work should any disaster happen is invaluable.

About the Author

Francesco Carucci is a landscape and travel fine art photographer. He was born and raised in Italy, but he is currently based in the Bay Area, US and has been a member of Professional Photographers of America since 2017. You can see more of Francesco’s work on his website and follow him on Instagram and Facebook. This article was also published here and shared with permission.

Filed Under:

Tagged With:

Find this interesting? Share it with your friends!

DIPY Icon

We love it when our readers get in touch with us to share their stories. This article was contributed to DIYP by a member of our community. If you would like to contribute an article, please contact us here.

Join the Discussion

DIYP Comment Policy
Be nice, be on-topic, no personal information or flames.

Leave a Reply

Your email address will not be published. Required fields are marked *

23 responses to “How I store and backup my photography”

  1. Amanda Rain Avatar
    Amanda Rain

    wish I had one of those

  2. Vincent Moschetti Avatar
    Vincent Moschetti

    Luck is my backup solution ?

  3. Robert Hicks Avatar
    Robert Hicks

    Yeah of Backblaze with versioning and unlimited space are roughly $100 a year if we’re doing product plugs

    1. Roku Avatar
      Roku

      And unlike Amazon and Google, Backblaze is zero knowledge storage.

      1. Robert Hicks Avatar
        Robert Hicks

        And I can vouch for the them email you external drive of your backups for approx 30$ round trip, they hold a 200$ deposit during the process and the drive was encrypted

  4. Graham Fielding Avatar
    Graham Fielding

    I honestly don’t think you need to backup absolutely everything you shoot. Most photos I take never get used for anything but I do put them on multiple external drives but I’m constantly deleting ones I don’t need or want anymore. My most important photos are stored on the cloud.

    1. Francesco Emanuele Carucci Photography Avatar
      Francesco Emanuele Carucci Photography

      That’s an interesting point. I definitely do not backup every single photo and I constantly delete the ones that are technically subpar.
      On the other hand:
      – it has happened that I worked and published photos that I initially didn’t like and they even ended up winning awards. This is an example:
      https://carucci.photography/blogs/blog/making-of-white-dunes-white-sands-new-mexico
      – clients chose photos that I didn’t like, so I have to to store almost all clients photos for a reasonable amount of time (1 year).

    2. Graham Fielding Avatar
      Graham Fielding

      Eric Lefebvre that was more so based off my own needs I don’t run a full time business and I have clients here and there but always put tham as priority. More so referring to photos I take on my own time

  5. Markó Richárd Avatar
    Markó Richárd

    HDD + HDD + Cloud + BluRay

  6. jonathan radcliffe Avatar
    jonathan radcliffe

    He should be able to create a iscsi volume on his synology nas and create his lightroom catalog there. Iscsi volumes behave identically to local disks – no application can tell the difference.

    1. Keith Avatar
      Keith

      And that’s a problem. iSCSI doesn’t support locking, so the shared volume’s filesystem can be corrupted if two or more clients attempt to create or write files at the same time, unless they are all running a clustered file system specifically designed for this type of configuration (https://en.wikipedia.org/wiki/Clustered_file_system).

  7. Clichey Avatar
    Clichey

    You can use Google drive for Lightroom catalog, it keep 100 versions of your files if you want to recover one.
    I put also Previews and Smart Previews on google drive, so I can use my library on every computer with LR

    1. Francesco Carucci Avatar
      Francesco Carucci

      I’d rather have all my photography local, including the catalog, and only backup an encrypted version remotely. For my clients I have an option for added confidentiality that they pay for.

  8. jeyare Avatar
    jeyare

    Francesco,
    just few recommendations to your approach:
    1. You can use 3x 8TB disks in RAID5 or SHR with better performance and fastest rebuild time, when your RAID will degrade.
    2. You can use 2 another 8TB disks as Spare disks, for automated RAID rebuild, even when you are miles away from home. You don’t have spare disk now.
    3. Then you can use immediate rsync or Snapshot/Replica or Syno Hypebackup sync to another (smallest NAS). In final stage you can still use 3rd stage of backup scheme to another external target (external disk, cloud, …).
    What you will get:
    +2TB of new space for your data
    + better backup scheme for your data, then better mental health to future issues
    + better speed of RAID rebuild (data availability) it counts
    + better speed of write/read operation for your data
    + 3 empty disk bay for your another experience with Synology (e.g. Syno Drive for 32 versions of your files).

    … more you can find in our independent Synoforum.com

    1. Francesco Carucci Avatar
      Francesco Carucci

      Hi, thanks for your suggestions. I opted for more storage over the spare disk for now, I might revisit the choice later when I move to a bigger NAS. I know have a second NAS where I replicate the Lightroom folder automatically for added redundancy and 3rd stage backups.
      About RAID5 vs RAID6, I opted for more redundancy over beer performance, since I have a SSD cache that helps with performance.

  9. Jason Tupeck Avatar
    Jason Tupeck

    Looking at the comments, I guess I’m not the only one who uses a combination of these technologies.

    I, too, use a Synology box with about 26TB as a target for local photo backups, with all my files on a series of USB disks attached to my photo editing rig. I keep my catalog in Google Drive. Plus, I also back everything up to the cloud with Backblaze ever since CrashPlan went business customers only and doubled their annual pricing.

    I knew about Synology’s iSCSI volumes, but didn’t even think about using this for a local physical machine in order to network attach the data file. I might look into it, because that would simplify a bunch of the architecture for me, allowing me to get rid of the USB drives.

  10. jeyare Avatar
    jeyare

    iSCSI is faster than Samba, but there isn’t real advantage for photo handling, even for full 64GB SD card of RAW. Then better is network optimization by fast switch (1Gbps and 1Gbps isn’t same for every vendors) include LAG (link aggregation), when your desktop device has more than single Ethernet port and your NAS also (1815+ has 4 1Gbps ports). Real bottlenecks are used disks (or disk pools), network and your desktop environment. DS1815+ is pretty fast for heavy photography workloads … experiences

    1. Francesco Carucci Avatar
      Francesco Carucci

      I have my NAS with 4 LAG, but at the end of the day, it’s too slow to work off the network until 10Gb/s becomes reasonably priced. That’s why I went for a two tier system and I effectively use fast local storage as a cache that is automatically shared across all my machine. Also a purely network based solution wouldn’t work when I’m in the field.

      1. jeyare Avatar
        jeyare

        Game changer for me has been total rebuild of network to Ubiquiti Unifi (USG Pro, Switch Pro, ..). It can boost your NAS horses to max level. I have few NASes in my multisite operation, include HA.
        Speed of LAG depends on setup of the LAG (in Syno setup as dynamic or static, or just load balancer, cold backup). Then when you have Dynamic LAG with 4ports in Syno, include fast switch (not just speed of the port) with dynamic LAG support, jumbo frames, … then you need same device in opposite side = desktop station for Photo edition = with 4 ethernet ports and dynamic LAG support. If you have such environment, you have right dynamic LAG environment.
        But how many Photo editors have such environment at desktop side?
        Finally:
        theoretical speed of 4x1Gbps is 512MBps, real speed in average LAG environment is 380Mbps (or worse). For such throughput you can transfer 12,8 RAW files per second max or 9,5 files in average environment (my Nikon D5 with 40MB per file). For single minute 570 or 760 files. Is it slow?

  11. Clippingpath Editing Avatar
    Clippingpath Editing

    Nice article about photography store. This is like instruction article. I think much effective.

  12. Eric Lefebvre Avatar
    Eric Lefebvre

    I closed my small business a few years ago and I was never that busy (it was a side job) but I used to have my working copy on my pc, backup to 2 external drives, backup online. Then after I finished editing I’d update the external drives and the web backup.

    When I did weddings, I didn’t go to bed until my 2 external hdd backups were done and the cloud backup started.

  13. Geoffrey Stoel Avatar
    Geoffrey Stoel

    Great article as I plan on setting up my workflow / backup solution…. did you ever run into issues with the Synology drive locking files? And could you elaborate a bit on the way you setup the snapshots? Thanks,

  14. Jordan Chapell Avatar
    Jordan Chapell

    Very interested if you have a plan for your cloud backup now that Amazon is going to do away with Synology’s backup integration: https://www.aftvnews.com/amazon-to-discontinue-access-to-amazon-drive-for-synology-nas-cloud-sync-and-hyper-backup/