Copying Large Video File Across Devices/Drives? This Is How To Make Sure You Don’t Goof Up

large-file-copy-09

Whether your are a professional photographer or an aspiring photo enthusiast a day will come, when you will need to transfer all your valuable multimedia materials from one storage location to another!

Obviously, a single copy & paste or drag & drop should do the trick! However, please be aware that any backup or data replication scheme is composed of two integral parts: copying and verification!

The fact that you see a complete file list after a quick, copy and paste operation does not ensure the data that you have copied from the source location to the target destination are not corrupted!

You may wonder why have I written this post in the first place? Well, as a matter of fact I have found myself in a position where I needed to send via air mail an USB stick with video material to one of the media agencies we collaborated with. The assignment came literally last minute, and the deadline was within 4 days.

The agency requested ungraded, C-log video clips, with no audio, in Apple ProRes HQ in 4:2:2 color space (at least). The most of the footage was supposed to be shot in 24p or 60p for slow-motion processing.

Why is all of that relevant? It is important as it indicates that resulting files will be relatively big (we are talking here about hundreds of megabytes in size, reaching gigabyte or two in some cases). The bigger the file, the easier to screw something up.

Truth to be told, I like DIY (do-it-yourself) projects and in general. Understanding how things work is essential in professional media maker workflow, especially if you have to deal with hundreds of gigabytes of photo material every month.

How did I solve my problem? (Please be aware, you need to have basic UNIX knowledge in order to proceed)

1. Simply copy & paste or drag’n’drop your files from source folder to destination. In my case I will be coping ProRes files from DOCK2/PRO-RES/ folder to my USBStick1.

large-file-copy-01

2. Find the Terminal application in your Applications / Utilities folder. (Hint: click your Finder icon and press ?+?+U key combination).

large-file-copy-02

3. Right click on your source folder (in my case DOCK2/PRO-RES/), and choose Get Info.

large-file-copy-03

4. A detailed information window will appear. Among various pieces of information you should find a complete path to your media source folder. In my case /Volumes/DOCK2 . Select the text with the path and copy it to clipboard (?+C).

large-file-copy-04

5. Open the Terminal application and execute change directory (cd) command. Localize the path to the source folder containing your multimedia materials, in my case /Volumes/DOCK2/PRO-RES/ (Hint: the external drives are always mounted under /Volumes/ directory)

large-file-copy-05

6. After changing the directory and jumping into your source folder, you need to compute an MD5 fingerprint of the files you want to archive.

MD5 digests have been widely used in the software world to provide some assurance that a transferred file has arrived intact. For example, file servers often provide a pre-computed MD5 (known as Md5sum) checksum for the files, so that a user can compare the checksum of the downloaded file to it. Most unix-based operating systems include MD5 sum utilities in their distribution packages. (ref. Wikipedia).

How to do it? You need to execute the following command (just copy paste this code below into your Terminal window and hit enter).

find -s . -type f -exec md5 -q {} \; | md5

large-file-copy-06

7. Once you execute the command, you Mac should start running the md5 for every single media file in your source folder. Eventually (it takes some time), your Mac will generate the summaryMD5 fingerprint, which is an unique identifier of the content of your media directory. In my case the fingerprint of all the media files in /Volumes/DOCK2/PRO-RES is f6e0ef7029d951e386ef2c6b8b2c3657

large-file-copy-07

8. Apply the procedure from Step 3 onwards, in order to generate MD5 fingerprint for your archived content. In my case it means applying the presented workflow to files in target /Volumes/USBStick1/ directory. Here is the result of this operation.

large-file-copy-08

9. In the final step, you simply need to compare the MD5 fingerprint generated for the source folder (in my case ) against the one generated for the target USB stick drive. If they are the same, you are good to go! Your files have been securely copied and there is almost no chance of having data corruption!

About The Author

Kamil Tamiola is a nature photographer and visual storyteller based in Aosta, Italy. You can follow his excellent blog here, his facebook here and his twitter here. For a full dose of inspiration visit his portfolio. This article was first published here.

  • http://www.t-rec.ca T_REC

    this is ridiculous.

    • http://www.alpine-photography.com/ Kamil Tamiola

      Why is it ridiculous? That’s exactly what fancy and expensive backup software is doing. What I have proposed here is a laborious, manual way, which is more on a ‘how things work’ end.

  • Andrew

    Or for Windows users, Teracopy. Job done!

    • http://www.alpine-photography.com/ Kamil Tamiola

      Sure! There are literally tens of good backup programs. However, 95% of them relay on the same method of checksum verification, either by MD5 or any other method. As a matter of fact, once can create a truly powerful DIY script that can do all what I have suggested in a completely autonomous way!

      • Aankhen

        Yes, or just use TeraCopy and have it all taken care of for you, plus extras like being able to resume interrupted transfers.

  • http://www.clipping4u.com Safi Sarkar

    This article explain all thing are very carefully. It is more helpful.

  • Brian

    If we’re jumping into the terminal to do this, why not just use rsync to copy the files? It does the checksum verification and comes installed on every OS I’ve used except for Windows.

    • http://www.alpine-photography.com/ Kamil Tamiola

      It is a very good suggestion, indeed! However, my point here was to demonstrate the most low-level method! I am perfectly aware it is laborious, however, it does have educational value!

  • Renato Murakami

    Nicely done and explained… for those talking about software, sometimes it’s just nice to actually understand what exactly they are doing, right?

  • Kay O. Sweaver

    I understand the academic or geeky appeal of knowing how to do this, but I’d much rather a simple little app do it for me so I can focus on shooting and editing.

  • Daris Fox

    With Windows it’s really, really simple.

    Robocopy folder folder. /z /log+:robocopy-log.txt

    Then use fciv

    fciv.exe -both path/filename r -XML pathfilename.xml

    Job done.

    http://technet.microsoft.com/en-us/library/cc733145.aspx
    http://support.microsoft.com/kb/841290

    If you prefer to avoid Powershell then you can use a GUI.