As we get closer to release, we’ve had more and more questions about workflow that we haven’t addressed in depth yet, so I think it’s about time to start talking software, storage options, bit budgets, and long term storage solutions.
We’ve thought about the workflow of the D16 since the first day we imagined it. Given the lack of CinemaDNG tools on the market, we always knew we would have to build our own software in order to take the best advantage of what RAW has to offer and to help push acquisition technology to a more RAW-friendly environment.
Over the past 18 months I’ve had many, many ideas about what the software would look and act like. At first I wanted to simulate the processing of 16mm film. I pictured a virtual lab with tanks, developer, fixer, and thermometers. Basically, I wanted to share the fun, experimental experience of processing film with a new generation of digital filmmakers. But as we started to analyze the current methods of working with RAW, and saw how few options were available, we realized we were going to need to build something very different, and much more streamlined.
One of our goals at Digital Bolex is to help break down some of the technical barriers to picking up a camera and making a film. One of our biggest challenges thus far has been combating the clunkiness of RAW workflows out there today, ones that more often than not create intimidating hurdles that are off-putting to filmmakers whose only experience has been with compressed video. We want to identify and remove some of these hurdles so that given the option between a video camera and a digital cinema camera, any newcomer would feel comfortable giving RAW a try.
So we began listing characteristics we thought would be useful in the software and trying to picture the workflow we wanted to see and how that would integrate with the larger post world of existing NLEs and color correction software. When comparing RAW to compressed workflows, the problems with most of the RAW workflows seemed to be:
1. You can’t start editing your footage right away. While some people are still stuck converting to ProRes to get their HD or 4K footage into an NLE, editing software is getting pretty advanced as far as what codecs can be played natively, and processors are getting fast enough to play even huge files. But most NLEs still don’t incorporate the ability to import a CinemaDNG sequence, so you have to do one or more transcodes during post in order to use your RAW footage to its best ability.
2. The online process is so problematic that many people go to post houses to finish their films, even though they have capable computers and software at home. Using XML to match an edited sequence back to the original raw footage almost never works correctly in existing software. So unless you have the budget to send your footage to a post house you end up wasting a lot of time re-editing to get proper sequences into a color correcting software.
3. The extra drive space from all the different versions of the footage makes the hard drive space issue for raw much worse than it has to be. RAW takes up more space than compressed video, without question. But having to transcode multiple times increases the space issue exponentially. In addition, you end up with so many versions of the same footage in different formats and connected to different software, keeping track of it all can become very confusing.
So with these problems in mind, we came up with three mantras for the new software:
- Keep it raw as long as you can.
- Transcode only once.
- Transcode only what you need to transcode.
These three directives allow for fewer versions of the footage, less drive space, less transcoding time, quick access to the editing stage, and no confusing online process.
I had Skype meetings with several companies, but none of them seemed in sync with what I was trying to do. Then Lars Borg from Adobe suggested I contact Pomfort. From our first conversation I knew this was the company I wanted to work with. Our long term goals were closely aligned and the guys were very excited about the project.
We spoke many times at length over Skype, but really got to sit down and work through some drawings at NAB in April.
My first day at NAB we sat down and made drawings.
These were the drawings we based the basic structure on.
There were many iterations for the look. And we even pulled inspiration from some of the original Bolex camera designs!
We’re getting close to a final design, now, and we’ll be releasing new image as the looks and features are locked down. We’re positive that it will be a unique post environment that will be simple and intuitive.
How it Works:
Within the software there are five screens we’re calling “rooms”. Each room is dedicated to a different facet of the post process.
The Copy Room is where you download the footage from cards to as many designated drives as you like. Like Pomfort’s Silver Stack, the software then reads the footage and compares it to the original cards to verify that all of the copies have been made correctly, before giving the user a big green check mark, indicating that it is safe to format the cards.
The Organize Room is where you label, categorize, and rename clips, edit file trees, incorporate script supervisor notes, and general organizational things.
The Color Room can be used both before and / or after the edit room. It is intended to let you apply your one light look non-destructively to a groups of clips, and then after a rough cut of the footage is complete you can go back and do a more refined color pass.
The Edit Room allows you to edit and play back your Cinema DNG footage in real time. To allow computers that are older to work well with the large files we have chosen to show the images in black and white while the image is playing, and render the color on paused frames.
The Export Room allows you to export your edit either as one file in a condensed rough cut, or as separate clips according to your edit. You may export to many different formats including Quicktime ProRes 444.
The idea is that you can have several people working on the same project simultaneously, using the same CinemaDNG files, and the only file that need to be sent from one computer to another is small and contains nothing but metadata. The assistant editor can be naming and categorizing clips while the editor is assembling sequences, while the DP setting looks for scenes. Hopefully this will create a friendly collaborative workflow for raw projects. On smaller projects where one person might be the producer, DP, and editor the “room” divisions define the work that is done and the work still needed to be done.
The file size for our raw footage at 2K resolution, 2048 x 1152 is 3.5MB per frame. At 24 frames per second this works out to be 84MB per second, 5GB per minute, and 300GB per hour. For capture we recommend you get two 128GB CF cards. We recommend getting CF cards that are rated between 400x and 1000x, as this will greatly reduce the wait time after recording.
A USB 3.0 CF card reader should be able to download a full 128GB cards in under 13 minutes (down to 7 minutes with 1000x cards). If you use two 128GB CF cards, your cards will fill up at exactly the same time as the internal 256GB SSD, at just over 50 minutes of recorded data.
At that point you should remove both cards, place them in card readers and allow the Bolex software to download your footage to the predesignated hard drive destinations for your project. After you get the OK from our software that the cards have been downloaded successfully, you can put the cards back in the camera and format both cards and the internal drive in the camera at the same time.
HARD DRIVE SPACE
The amount of hard drive space you’ll need always vary from project to project. The key thing that you need to remember is that 5GB = 1 minute of CinemaDNG footage. This is pre-transcode, pre-color correction, pre final render.
Here’s how we break it down:
Let’s be safe and say that on any given project you’re shooting a 10:1 ratio. Obviously everyone shoots differently, but this is a safe way to pace your hard drive; it’s always better to have more space than you actually need.
So let’s multiple a minute of footage (3.5MB x 24 x 60, or 5 GB) by ten to get 50GB of just CinemaDNG files for every 1 minute of footage intended to go into the final piece. Then let’s double that to account for the ProRes or other transcode you’re going to perform to get your CinemaDNG footage into an NLE. Again, this amount may vary depending on what codec you’re transcoding to, and if you’re transcoding ALL your footage or just specific clips.
So now we’ve got 100GB allocated to each minute of your finished film. If you’re shooting a 10 minute film, that’s a 1TB drive just to store your CinemaDNG and transcode files. Of course you’ll need even more room to store your audio files, your score, your “final-version-87.mov” wishful-thinking rendered timelines, your NLE render files, and your color corrected, full res, ACTUAL final version of the film. To be safe, let’s double that 1TB, and say that for a 10 minute film, we recommend a 2TB drive. And we recommend getting a SECOND, duplicate 2TB to make a backup.
If you’re making a film longer than 10 minutes, use the guide above to calculate your storage. 5 minutes? 1TB. 30 minutes? 6TB. 90 minutes? 18TB.
G-Raids are great for smaller projects needing an active hard drive. For bigger projects, I like the Promise drive for active hard drives. Western Digitals are good as back up drives.
LONG TERM STORAGE
Long term storage is tricky in digital, there’s no way around that. I prefer the Quantum LTO-5 Tape Drive for long term storage. Quantam tapes are relatively cheap, and if you invest in buying the drive, you can make as many long-term backups as you need. A lot of people shooting small projects tend to forgo this last, crucial step in making a digital film, preferring to rely on their backup drives to store the film. But hard drives, especially spinning disk drives, are fragile and become obsolete quickly. As everyone moves towards digital workflow, it’s important to get into the habit of thinking of the long-term life of your projects.
Hard drives still cost a lot of money, but thanks to Moore’s Law, every year they’re getting bigger and cheaper. Let’s go back to your 10 minute film. Today you can buy a 4TB G-Raid on sale on Amazon for $259. So let’s pick up one of those (because they’re cheaper than the 2TB version for some reason), a 2TB Western Digital ($120) for backup, a Quantum Drive ($2000), and 2 Quantum tapes ($40×2) for long term storage.
The grand total of your purchases is $2,459, and you’ll have all the space you need to edit, backup, and store your project for posterity. I know it sounds expensive, but you can reuse most of these drives for every project you make in the future. For each new 10 minute film, you’ll only need to pick up another 2TB Western Digital and two more Quantum tapes for a much cheaper $200 total.
$200 to preserve your film? Sounds good to me. And in 12 months? It might cost half that.
The last question that’s on everyone’s mind is the kind of computer that will be necessary to run the software and process all this raw footage. The very idea of raw tends to conjure up a scary image in people’s minds of some kind of crazy expensive super computer. That’s not true, so don’t worry. Because we’re not debayering in real time, there’s minimal RAM usage, so you should be able to use laptops and desktops alike, even ones that are a couple years old. Of course, newer processors and higher RAM will drastically reduce the time needed to transcode.
The one caveat is that right now, our software at launch will only work with OSX. We hope to build a Windows based system soon afterwards, but we’ve heard from most of our backers that a Mac workflow was a priority.
To that end, we’re future-proofing the software as much as possible by optimizing it for the Macbook Pro with Retina display, which in coming years will become the standard for Apple computing. The Thunderbolt and USB 3.0 ports are super speedy, and will get the fastest transfers from your CF cards. The Retina display is very high res, and can show a 2K image with real-estate to spare. The software will of course run on any computer that can run a current version of OSX, but we want to make sure it can handle and be able to take advantage of likely changes in the hardware Apple produces.