Digital camera post data workflow

For shoots on Arri Alexa, Red Epic, Sony F65, Canon 5D, Phantom, or any such digital camera, is there a preferred workflow for the post?

When the edit is complete and its time to extract files for promos, for VFX, or even for the final DI, most producers send the original hard disks (al 20, 30, 50 or more drives) to the DI facility. Places  like Prime Focus, Reliance, Prasad or wherever. 

In these studios there are a bunch of very over worked in-house data managers and conformists who use the hunt-and-copy technique to copy the selected files from original disk to their own Baselight or Resolve or whatever. 

Delays, wrong files, confusion, tempers… and other such dramas end up happening.

What’s the best way to do this? Is there some kind of data or file management in post? Can someone who can handle this workflow more elegantly?

Over the past few years, and after doing it for a dozen films, I’ve worked out a cleaner method.

1. First, I make a database of all your hard drives. Which should be anything between 10 to 20 to 50 individual drives. And data spanning anything between 10 Terabytes to 100 Terabytes. All this happens at the producer’s office, or they can send drives to my place. I save only the file names and paths, no pictures. Or sounds.
2. Then, as and when files are needed, I call for the Avid or FCP sequence. No need to make EDLs or XMLs or anything. 
3. Then, my software scripts go through the Avid or FCP (or even Premiere Pro) sequence extract needed files, and search my database for the needed files. No need to access the hard drives yet. So all this can happen at my place and you can simply email me the Avid or FCP sequence.
4. Then I build scripts to automatically copy the necessary shots to another hard disk for sending to Promos, VFX, DI or wherever.

The result is that instead of carting all your tens of hard disks to the DI place, you only send out a small drive containing only the used shots and nothing else. And, to save space and time at both ends, I can even ‘trim’ the original shots to the required duration plus handles.

Sending me a reference mov helps me to confirm that I have the right shots. But that’s optional. If the original conversion from camera files to FCP or Avid files was done accurately by the data manager on location, then my systems and software usually pick the shots fine.

As an example, for a movie that released last week, the original F65 data was spread over about 15 drives, about 30 Terabytes. The sorting took just about 2 days. And the sorted reels were ready at about 4-5 hour intervals. So, 4 hours after I got the edit, the DI could begin.

And the total sorted data with 1 sec handles was just 4 TB. And this is the archive the production needs to keep a backup of after the film is released.

Conversion, transcoding, dailies

After the shoot one needs shooting files need to be converted (or transcoded) to something before editing can begin. That something can be Apple ProRes for FCP or DNxHD for Avid. 

This conversion is usually done by the on-location data manager. But, with the sheer volume of shooting, multiple cameras being used, time to backup being huge with older laptops, data managers often find the task daunting. So, they sometimes end up with errors like not passing the original timecode, reel names, or even at times messing up file names.

If I or my systems do the original conversion on a daily basis, then the database of your drives is built as we go. So, in the end, shot spotting and copying is much faster and more accurate. So, productions send me one copy of the shooting drive on a daily basis after packup each day. And I do conversions overnight and you have editable dailies the next morning. For outstation shoots, they cargo the drive to me every few days and I do the conversions.

So, digital shooting need not be a nightmare in post and DI. Some planning, some competence in data handling, and its a breeze.