Lightroom Tip: Backing Up Huge Lightroom Files
When searching for a backup solution I was most fascinated with the option of online storage. My major concern was the huge catalog files that Lightroom creates. I even listed the huge catalog files a deadly sin :)
I got a great tip on the comments explaining how to use XMP files to override the huge files issue. Thanks Jigar. (Unfortunately, the comment got deleted as part of my fight with a spam attack. Please post the comment again so I can provide appropriate credit).
My Lightroom workflow is very similar to the workflow Kerry showed on his Lightroom workflow video. That mean my catalog is always changing/evolving/growing. It also means that my backup software will notice that the file has changed and try to back it up after every session. Kinda heavy with a 60 Megs file. This is where XMP comes in to play.
One of Lightroom features is to allow you to save small portions of your catalog as XMP files. Those XMP files will reside next to your raw files and save the development changes you made with Lightroom. If you shied away from Lightroom, this may bring you back.
Once you enable XMP (Edit -> catalog settings -> Metadata -> Automatically write changes to XMP) Lightroom will save a small XMP file near every RAW file with the latest develop settings of this file.
The Good
- Once you enable XMP, you can backup your small-almost-unchanging XMP files instead of your huge-ever-changing catalog file
- XMP is a great format for on the go, instead of exporting from catalog and importing back to catalog to preserve Lightroom develop settings.
- XMP is a great back in case your catalog ever crashes (and they sometimes do).
The Bad
- XMP does not support develop history - it will only show you the latest state of the image. That means that you can not go back if you don't have the catalog as well.
- No support for Lightroom attributes - XMP files will not show you how your file was flagged, of which collections it belongs to.
- When setting Lightroom to support XMP, it will only do it from the moment of selection, you will have to manually export the files that are already in the system to XMP.
Get the DIYP greatness via RSS, newsletter and Twitter
Connect with the community: Facebook Page, Discussions
Share Ideas, Setups, Images and Projects on DIYP's Flickr, visit Readers Photos











Comments
Lightroom and huge files
Hi there,
I was the one who posted the tip regarding LR and XMP files on your other blog post :) I'm glad you mentioned the Cons to using XMP files as well. It's not a perfect solution, but as I mentioned yesterday and you've mentioned today, it's an additional way of not relying on a single file or back-up.
Good lookin' out!
- Jigar
Lightroom Catalog and XMP
I'm not sure how your suggested solution (saving metadata to XMP sidecar files) solves the conceived problem of a large Lightroom Catalog file.
Whether you save your information out to XMP files (or embedded metadata in DNG files) or not, it is still written to and saved in the Catalog. The catalog contains many important pieces of information that are not stored in the metadata of files including, but not limited to: develop history steps, pick flags, collections, etc.
To "manually export to XMP" as you say it, is as simple as selecting photos and tapping Command-S (Control-S on a PC).
Enabling "Automatically write changes to XMP" does not "enable" XMP, it just enables a sort of auto-saving to the files. A downside of this auto-saving, is that it can lead to sluggishness when editing metadata for multiple images, particularly in a large catalog. I prefer to manually save metadata out to XMP files (or, in my case, embedded in DNG files).
Thanks,
Scott
60mb isnt a huge file!!
I backup dvd images that usually are 3.5-4.2 GB.... 60MB files are pretty small in todays terms...
Peace!
DNG...
There is one more "bad" think about this... You can't use this with DNG files... as XMP file is embedded inside of DNG file.
I converted all of my CR2 RAW files to DNG. There are some benefits when using DNG, but now I see that there is one big "problem" about it.
Anyway, thank you for this tip. I turned this feature on several days ago, so my DNG files are always "ready to transfer with all metadata and develop settings".
btw: As Lightroom catalog is SQLite DB, it should be possible to write back-up utility, that will not copy the whole catalog, but just do some kind of DB SYNC. This may be good alternative. You will need original LR catalog and it's local copy (last back-up version). Than You will have to find differences and sync them to online back-up (this can be MySQL database from any web hosting service). Just like SQL databases doing with transactions logs ;)
Another negative...
I believe it's important to distinguish between online storage and backups. They are totally different concepts and what I do for the former (see my review of the ReadyNAS Pro) is not at all what I do for the latter. Offline backups are not meant to be a shortcut and to save time; rather, they exist in case the worst happens with your system.
One more problem with what you're suggesting as a backup is that virtual copies won't exist if you just archive RAW + XMP because they do not have their own sidecar XMP files created. Only the master will have its settings written out to an XMP file. This isn't an issue if you never create virtual copies, but I find myself creating and using them all the time.
What I do for offline storage is export the library or libraries I wish to archive into an independent catalog and back that up along with RAW + XMP files to optical media. I don't typically care where in my master catalog individual libraries reside, so this backup technique is perfect for my workflow. Plus, I'm only going to pull this archival data if a client wants something from a shoot a year ago.
re: XMP
This is a great set of comments, let me reply.
@Jigar - Thanks for sharing this tip. I believe many photographers are dealing with a way to keep online backups of their images.
@Scott Rouse - I do not have a specific issue with the fact that LR keeps huge files. The only thing is that the huge catalog file will have to be backed up any time I do an online backup. Until bandwidth get faster, I'd hate to waste 60 megs on each backup. I was thinking to back up only the XMPs
@lithivm - 60 MEGs in not a really huge file, the issue is that this file is ever changing.
@Ladislav Soukup - Thanks for the DNG review. I know some photogs swear by DNG, however, I never tried them out myself. would you say it is worth the effort?
@Jonathan - I agree, virtual copies will not work with XMP, one solution I found in the web is to use snapshots. Till Adobe works out the virtual copies issue, snapshots can be a tolerable workaround, as it saves the snapshots.
Sheesh, only 60mb? My
Sheesh, only 60mb? My catalog file is 10x that size.
Anyway, my advice is to back up your catalog file less often than the rest of your files. Once a week or so is probably a good compromise.
The other option is to do what I don't do and occasional export any archives that you're unlikely to work with again in the near future as a separate catalog. Back that catalog up along with the associated RAW files and keep the one you're working with as small as possible.
(Personally I like being able to search all my archives though).
DNG
@udijw: DNG should be an RAW standard supported by Adobe forever... but who knows. Fact is, that DNG is not made by camera manufacturer and it is not binded to specific digital camera. Also DNG is open source format.
But hey, these are just promotion lines by Adobe...
For me... All my photos are taken in RAW. for last few months, I'm converting all my photos to DNG (I did convert all of my older photos few weeks ago).
+ DNG is smaller then CR2 (it's not a big difference, but it is)
+ All settings (like White balance, saturation, camera profile) and metadata stored in XMP file are embedded into DNG. Let's say, that DNG is container for RAW & XMP files. This let you easily send RAW files WITH settings in one file. You can send RAW and XMP file... But it is always better to send only one file instead of two (you can forgot the XMP, recipient may forgot to save RAW and XMP file in the same location, ...). If you lost the XMP, you will see original image without develop settings.
+ Many stock-photo sites allow you to upload RAW file, but you can't upload XMP file. Also many buyers can just download RAW, but they can miss the XMP file. DNG is solving this.
+ DNG has small jpeg preview embedded (optional, turned on by default).
+ DNG is open source and is supported by most applications (I saw few photo viewers with DNG support, but without CR2 and NEF support).
- Any change of metadata will cause change of the whole file and backup softwares will copy the full image (if they are not backing up only changes).
- Sometimes, you don't want to give out RAW file with metadata and settings... It is "hard" to remove XMP from DNG file (in compare to just not copy the XMP file).
- They are still application that can't read DNG
- Import from your digital camera will take more time! (you need to download RAW file from card/camera and then convert it - unless your camera can shoot directly to DNG)
It's hard to say if you should or shouldn't switch to DNG. I did. I like the XMP and preview JPEG embedded in one file with my RAW.
LR catalog online backup
You can easily backup your LR catalog online using Mozy Online Backup service. You have 2GB FREE storage and MozyHome Client will only upload delta of the file.
I checked it today. You have to do initial full backup of your catalog. After this, only changed parts of the LR catalog will be uploaded. I did several changes to my catalog and only about 500kb was uploaded in next backup.
There can be more similar services for online backup. I'm using this one and it is working. Link is: https://mozy.com/?code=BY2RU0. There is my referral code. You can strip it or not - your choice. If you will use it, you will get another 256MB for free (I will get it too). So, you can have 2,25GB for free. Or you can buy unlimited storage (about $5/month).
Just a thumb up for mozy
Just a thumb up for mozy online back up...I've been using it for months now and it works great!
Thumbs down for mozzy
@Daniel, if you google mozy you will find many people who thought mozzy was great till they tried to restore. The backup works great, restore not so great always if you read the reviews.
Post new comment