With so many new cameras being released each year that allow capturing images in RAW format using different compression levels, bit-rates and other proprietary data, it is becoming increasingly difficult for post-processing software to keep up with all the changes and provide full support for RAW formats. Although camera manufacturers have been bundling their own image converters, such tools are often underdeveloped, buggy and lack the necessary features to be able to rely on them for post-processing images. Despite the fact that some post-processing software tools as Adobe Lightroom and Capture One provide frequent updates to support new cameras, it is getting practically impossible to provide full support for every new camera and RAW format features. It is time for camera manufacturers to come up with a single universal RAW format that can be easily supported by all post-processing software and we as consumers need to push camera manufacturers to adopt this format.
I just came back from a two-week trip to Israel and for the past few days, I have been struggling with the several thousand RAW images from the Fujifilm GFX 50S that I used during the trip. First of all, my laptop struggled so much with post-processing images in Lightroom, that I gave up and decided to do everything on my high-end desktop. After arriving home, I started the image culling process using FastRawViewer and once I deleted a bunch of images, I started the image import process. The images were imported pretty quickly, but since I knew that I would most likely want to look at the image detail of some images, I thought it would be a good idea to generate 1:1 previews in Lightroom. I started the preview generation process before going to bed and thought that by the time I wake up the next morning, the previews would be ready. I was wrong – it has been two days now and Lightroom is still struggling with the process. Lightroom is eating up 15 GB of RAM and 20% of the CPU during the image preview generation process and it still has approximately 30% to go, which is another day! I have attempted to stop and restart the process, rebooted my machine and nothing seems to be helping – Lightroom is simply inefficient when working with the Fuji GFX 50S files, even on a high-end desktop machine. Frustrated with the outcome, I decided to give Capture One 10 Pro a try, only to discover that Capture One does not even support files from the GFX 50S. And when I went back and tried to import some of my photos from 2016, I discovered that Capture One 10 Pro does not support a number of RAW file formats, including pixel-shift RAW files from the Pentax K-1. The same with DxO Optics Pro, which has zero support for some RAW file formats such as Fuji’s X-Trans.
Considering that Lightroom, Capture One and DxO are the most popular post-processing tools on the market today, it is frustrating to see how poorly they support RAW images from some of the most popular cameras on the market. Having been frustrated with Lightroom’s poor handling of RAW images in the past, I have been looking for an alternative way to convert images, but none of the methods seem to offer a solid solution. And let’s not forget that most operating systems cannot read RAW files either, making it difficult to even see what is in an image folder without using a third party tool…
Converting RAW files to TIFF format is not an option, because TIFF files are flat, which means that the editing headroom is already pretty shallow. In addition, TIFF files take up too much space, making them impractical when dealing with thousands of images. Adobe’s DNG format might sound like a solution, but it is not fully supported by all post-processing software, it takes time to convert images and it is known to strip out some of the proprietary EXIF data that is embedded by the camera. I stopped converting RAW files to DNG format a while ago for these and other reasons, as explained in this article. In short, there is no good option today to deal with all the RAW formats. Given the fact that camera manufactures do not seem to give a damn about this issue, we have a pretty big problem with post-processing software developers digging into RAW files, reverse-engineering some of the proprietary data and extracting what they need. As a result, we have partial support for RAW files and varying degrees of post-processing headroom in different software tools.
Since RAW files are only getting more complex with so many options (lossy-compressed, lossless-compressed, uncompressed, 12, 14 and 16 bit data, pixel-shift and other proprietary data), I do not see how it will be possible to continue this RAW format insanity. It is time for camera manufacturers to come up with a single file standard that will provide all the necessary support for RAW files from every camera. For a while, many of us thought that Adobe’s DNG format would become such a standard, but after so many years, it has clearly failed to accomplish this task – only a select few camera manufacturers provide an option in their cameras to save RAW images in DNG format today.
I know that the photography community does not care if DNG or some other format becomes the standard – as long as there is one universal format that works in every camera and post-processing tool, we will be happy! If camera manufacturers do not want to standardize on Adobe’s DNG for whatever reason, they should either come up with a universal format and agree on its use, or engage the open source community to create a universal format. No matter what route we go, it should be a fully open format that is clearly defined, similar to some of the ISO standards we deal with today. This way, it will be easy for all operating systems, post-processing tools and cameras to provide full support for the new universal RAW format.
Personally, I am sick and tired of the constant headaches we as end users have to deal with when working with RAW images. I should not have to change my post-processing tool of choice just because some other tool does a better job at handling RAW files (Fuji X-Trans anyone?). I should not have to wait for my post-processing software to be updated to support my newly purchased camera. I should not have to buy and install multiple post-processing software packages on my computer to handle different cameras or file formats. The world is moving towards simplicity and there is a reason why smartphones and easy to use apps are taking over. If camera manufacturers do not come up with simpler ways to use and store data from their cameras, their sales will only continue to suffer…
What do you think? Please share your thoughts and feedback in the comments section below!
Dear sir
Our company is based in Portugal, Lisbon and we are expert in XRAY photo digitalization.
Our main work is photograph clinical X-ray archive and for that we develop a new concept, that is photograph the X Ray in a special box with illuminated by led lamps, using a Canon with Lents Sigma 500mm f14.
For capture the X Ray we use the program;
www.phaseone.com/en/Pr…ights.aspx
Our question is, how can we certificate this system and inform our clients that the Raw system is a negative film and have the guarantee of the authenticity of this type of image and is not manipulated .
My DJI Mavic Pro drone shoots RAW files as DNG! I wish other makers did this too. Would help solve the competition entry problem noted above.
Nasim – well argued but I simply do not agree.
There are upwards of 10 main stream camera manufacturers; each have optimised their cameras for their market and the format of RAW files is the one they consider works best for them. As a result we have a market of both dedicated sensor type and broad use RAW conversion software – PHOCUS, NDX are both sensor optimised and work with only one type of RAW files; whereas the rest work with all. It would surprise me if Capture One was not optimised to work better with Phase One Cameras/backs than any other RAw converters
Try telling Mamiya Leaf, Phase One or Hasselblad that they should stop providing digital backs that produce RAW files that work with both dedicated software and broad RAW converters. Why should they, let alone any other manufacter?
I guess the bottom line is that one considers a photography eco system, he or she needs consider not only the camera and the lens(es) but the software that would be used for RAW conversion.
I cannot speak for everyone else, but I feel that my powers to change Nikon’s/Canon’s or anybody else support for RAW format and/or editing software is limited.
Again, I personally got used to think of the “Camera + lens + software” combination rather than of “Camera + lens.”
Val
That is true. I calibrated darktable’s base curve for each of my cameras against Nikon’s generated JPEG. The results please me very much (more than the included base curves). It goes that far that the output is almost not depending on what camera’s profile I use. This speaks for Nikon as they take care a lot to keep their characteristics along the cameras.
But still you are right. The output of Nikons JPEG-engine is a grain finer. Since I quite often rely on the iso invariancy of the Nikon cameras there is no way around editing RAW-files anyway. There is also no way around if you want to give your photos e.g. the paper look (as if they were printed). But then you are way off to the camera JPEG-output anyway and the raw converter is turning into your personal photo lab (somehow). Like in the earlier days: It was the film (sensor) and the paper (raw converter and paper) which gave the characteristics to the photo most. So “camera + lens + software” is just like it has been ever since, isn’t it?
Hi, Jan,
You are right, it is like in the old film days. To me the RAW converter is like a film developer. In the film days I was in search for a magic B&W film developer recipe and I eventually found one. Nowadays, Capture One Pro is the magic film developer for me (I use Olympus OMD-EM 1).
Darktable produces pretty close results for me too. If I go Linux, I will use Darktable (I am using Mac at the moment).
Val
Hi Val
There is darktable for Mac, scroll down here: www.darktable.org/install/
The procedure of calibration is somehow complex and nothing a regular user would be able to do. The OMD-E M1 is fully supported: www.darktable.org/resou…a-support/
Cheers!
-jan
Thank you, Jan.
All the best,
Val
If you take the film era analogy further: considering the camera as a camera (!), a raw converter as a developing tank, and the LR/PS etc process as the enlarger; you will remember that medium format film required different spools and tanks, different film holders and enlarger lenses; not to mention the fact that when using medium format film, your final output was usually a larger sized (expensive) paper if not drum-scanned for offset litho output. The process required more dedication, time and expense compared to 35mm film, which itself wasn’t cheap.
Seems to me that if you’re shooting digital MF you should be aiming for MF output quality, which probably means a dedicated MF raw converter and specialist post processing software. LR etc are consumer level products, and $6,000+ camera bodies are not designed for consumer level output. There surely must be professional software used by magazine printers etc. that would be able to handle large RAWs effortlessly.
I started to look at changing to Olympus for a lighter system for us older folk. However, Lightroom 5 does not support the raw files for the new OM E1ii…or the lens profiles. I decided not to buy another camera because of the lack of support by Lightroom 5 and/or the learning curve associated with a new software system as an alternative to Adobe.
In other words Olympus lost a sale because the post processing of raw files is a nightmare. Maybe pressure should be put on camera manufacturers to pick a format to automatically convert their raw files too?
Hi, Victor,
It is not Olympus’s fault that Lightroom 5 cannot open its RAW file. Capture One Pro 10 opens all Olympus OMD EM-1 Mk2 files with no problem.
Go the the DPReview website and download RAW samples
www.dpreview.com/sampl…es-gallery
Then, download Capture One Pro 10 trial version.
www.phaseone.com/en/Do…wAodZMEDVQ
Enjoy! Life is good!
Val
Always save a JPG as second file along the raw file while shooting. While browsing them select and dispose all the photos you either want to keep or not. So you would not need processing power to convert hundreds of raw files to jpgs and you can compare the output of your raw converter with the original jpg of the camera (e.g. colour, grain).
As for the universal raw format there is one major problem. The raw format is called raw because it contains sensor information of the camera with minimal processing. As there are different types of sensors and the development of some techniques is far from being finished it makes it almost impossible to define such an universal format. A least there could be a base format which could be extended in the future.
Raw files contain JPEGs. Exposure for JPEG is suboptimal for raw.
You are right. I had the opinion the resolution is lower than that of the cam-generated JPEG, it is not. Thank you for correcting.
Additional information on the subtopic above: The extracted JPEG-file is missing the EXIF-data. These have to be applied separately. If I compare the size of both JPEGs (camera and extracted) the extracted image of the D800E is 1.1MB and the generated image is 1.4MB. (The generated image is of type ‘basic’ and ‘size priority’.)
The D4 with ‘basic’ and ‘quality priority’ delivers: out of cam JPEG: 1.7MB, extracted of raw: 0.6MB.
Since the resolution remains the same and the quality of the extracted JPEG is still very good they can be used for the quality control. (So if I review with my cam’s LCD I see the embedded JPG and not a on the fly converted raw file. So reviewing the JPEGs gives the better quality the reviewing the RAWs because they are less compressed?)
btw, I used: exiftool -b -JpgFromRaw 100_1234.NEF >100_1234.JPG
What is the real proportion (or close) of all camera users that exclusively or mostly utilize RAW (or similar format)?
Then — what is that proportion in different camera brands?
I think the proportions and preferences would be reflected in a correlated manner with volume (of sales) in case of the different camera brands and 3rd party softwares supporting and updating their RAW translators etc…?
It is all further complicated by algorithms underlying different formats and how transparently individual camera companies share those. Shades of PC vs Apple philosophies…?
At least it’s comforting to know I’m not the only photographer out there who’s frustrated with the current RAW model’s rough edges.
The more common message on the net doesn’t embrace much uncertainty: “If you can’t make magic 100% of the time with [Insert Camera + RAW converter combination here], it’s because you’re too stupid to be helped.”
From where I stand, however, the RAW model feels like something most of the camera manufacturers have simply left underdeveloped and unfinished. An afterthought, really. Sure, Phase One seems to have closed the circle–bundling their high-zoot medium-format digital backs with their own excellent Capture One application as the According-to-Hoyle converter–but when you throw in the camera body and a lens or two and a computer capable of running the files, you’re approaching a six-figure investment.
It leads me to the nagging, uneasy feeling that CaNikon et al. really just expect us all to be shooting JPEG. At which point everything comes together–the manufacturer’s color and tone models, processing speed, sharing, universal file compatibility–so long as you’re OK with the same editing latitude an iPhone shooter enjoys.
Seems like an awfully big hit to have to take.
Really, that makes sense though. The bulk of Canon and Nikon’s customer base are JPEG shooters. Most of Canon’s camera sales, for instance, are Rebels. Like, by a lot. I’m not disagreeing with you, but from a business standpoint, they have very little to gain by making their own Adobe-like software and everything to gain by just providing the minimal RAW software when you purchase your camera.
Is the issue really yet another proprietary raw file format or rather the computer processing requirements needed to handle a 125MB raw file ? Also how do you explain a very small software company such as PictureCode being able to update its product (Photo Ninja) in a very timely fashion… one of the first to support the GFX files. I would venture to guess that the internal business processes at Adobe and Phase One are more likely the reasons why these folks are never front and center… the underlying math needed to handle the sensor data of a Bayer sensor is already known.
Yes it may be an aggravating & time consuming process, but it’s still a huge improvement over developing film. Imagine having to process several thousand film images. It’d be weeks/months(?) not days!