DNG: The solution to proprietary RAW files

Almost every new camera seems to bring a new, proprietary file format that nothing else can work with at first. It's quite likely that, 25 years from now, software capable of reading today's proprietary formats will be difficult or impossible to find- in which case your gigabytes of old photos would be unreadable. Is there a solution to this problem?

TL;DR: When shooting RAW, convert the camera raw files to DNG format, and save the DNGs instead of the original raw files.

The Problem

There are plenty of cases where it's useful to work with the original raw data from the camera's sensor, instead of starting with a compressed, processed JPEG file.

Each model of camera, though, has a different raw data format. They're not even close to consistent within a given brand- every time Canon or Nikon announce a new model, they also add yet another variant of their proprietary (CR2 or NEF) file formats. Until software vendors like Adobe get their hands on the new model's technical details, only the camera vendor's own (usually lousy) software can read the new model's files. Adobe tends to add the latest cameras' specs to their software within a few months, but others- such as the open source dcraw, which I consider the long-term gold standard of RAW converters- often take much longer.

Because the proprietary raw formats are- well, proprietary, it's difficult for third-party software vendors to reliably support them. To a programmer, camera raw formats are a dangerous moving target. How do you add keyword metadata to a NEF, for example, when there are dozens of variants of the format (and a couple of new ones coming out each year), none of which are fully documented? Every time you touch one, you risk corrupting it in an upredictable way. As a result, just about every photo editor that works with proprietary RAW files dumps a heap of "XMP sidecar" files all over your working directories. These separate files store all the metadata relating to each RAW file in an edit-safe fashion, but they're a nuisance to manage.

For how long can you expect software vendors to support your particular camera's peculiar RAW format? Corel office files, Borland databases and other proprietary formats from the 1990s are now all but unreadable without a lot of fiddling, 15 to 20 years later. I would hazard a guess that existing photo management tools such as Lightroom will support old formats for as long as they're around, but they'll eventually die off- and new software is unlikely to support formats from long before its time.

One RAW format to rule them all

Some clever engineers at Adobe saw this coming a decade ago, and hatched a plot worthy of Sauron- one master RAW format to unite them all. Digital Negative (DNG), released in 2004, was designed from the ground up to be a long-term solution to the problem of proliferating proprietary RAW formats.

  • DNG is (mostly) open. The DNG specification is freely published, and anyone can implement it without royalty or intellectual property concerns.
  • DNG is stable. The format is not redefined with every new camera. It is based on a well-established standard (TIFF/EP) and Adobe has offered it to the ISO for formal codification in that standard.
  • DNG is self-contained. DNG files include a description of the camera's sensor configuration, while other RAW formats assume this information is hard-coded in the software. The DNG format also contains its own edit-safe metadata, rather than relying on separate sidecar files that tend to get lost in transit.
  • DNG is lossless. There's no deterioration of the image data when converting proprietary files to DNG. If you're paranoid, you can even embed the original RAW inside the DNG (although I really don't see the point in doing so).
  • DNG saves disk space. The format includes an efficient lossless compression that often makes DNG files considerably smaller than their proprietary counterparts, without losing any image data.
  • DNG will be around for a long time. Archivists have taken a liking to it, and most photographic software now supports it. Key high-end vendors such as Leica now use it in their new products.

A few caveats

There's a rare variant called Lossy DNG that has all the disadvantages of JPEG, but without JPEG's popularity and broad support. I recommend avoiding it. If you want compressed, lossy files, stick with the proven and widely supported lossy image format: JPEG.

Some camera makers include tags in their original RAW files, such as in-camera post-process settings, that are not recognized by all DNG converters. This is rarely a problem- the reason for shooting RAW, after all, is to bypass said in-camera processing. If this is critical to you, though, you might want to keep the original proprietary files.

Summary

Canon, Nikon and several others continue to use proprietary, undocumented RAW formats- and while I can tolerate the 'proprietary' bit, the 'undocumented' bit is more than a little irresponsible for companies whose equipment is relied on by so many working professionals. My answer is to convert all RAW files to lossless DNG as soon as they're successfully copied from the card, and then- having confirmed that the DNGs are intact and not corrupted- throw out the proprietary originals.

Topic: 

Art: 

Add new comment