Nikon HE* Versus Lossless Compressed .NEF Files

Nikon’s Expeed 7 processor – used in the Nikon Zf, Z5 II, Z6 III, Z8, and Z9 – added a variety of capabilities to the Nikon Z lineup. Among its most notable characteristics is that the handling of RAW (.NEF) files on these cameras has a new option called High Efficiency* (HE*) compression. Nikon asserts that this formats is “visually identical” to their traditional Lossless Compressed RAW files, yet much smaller – and more capable of shooting long bursts without filling up the buffer.

In this article, I’ll examine the significance of this claim, exploring the technology behind HE* compression and the essential role of IntoPIX and its TicoRAW software.

This article’s concept originated from my hesitation about using this format for space-saving. I doubted its effectiveness and felt compelled to verify its image quality performance. Luckily, I was able to enlist Jason Polak’s expertise for the scientific analysis; he handled the file comparisons and authored the “Chroma and Luminance Variation” section of this article.

In order to make a technical article like this one more entertaining, I’ve sprinkled in some photos of adorable mallard chicks doing their thing. Why mallard chicks, you ask? Because I had a delightful Saturday morning snapping pics of them at the lake, and their sheer cuteness was just too good not to share! Hopefully, their charm will help you waddle through this technical stuff with a smile.

The RAW Dilemma: Quality vs. Size

Traditionally, Lossless Compressed RAW files have been the preferred choice for photographers due to their ability to capture maximum image quality, without the size of shooting uncompressed RAW. However, even Lossless RAW is not a particularly small file. Some cameras offer smaller RAW files with lossy compression, but this comes at the expense of image quality.

With the Z9 (and then all the subsequent cameras using the Expeed 7 processor), Nikon introduced an innovative solution with HE* compression, promising the best of both worlds: smaller files without compromising visual quality.

Nikon’s Statement: “Visually Identical”

In the Nikon manuals and on its official website, Nikon emphasizes that HE* compressed RAW files are “visually identical” to lossless RAW files. This statement isn’t meant to be marketing hyperbole. Rather, it is meant to indicate that – while the file is technically compressed – the data loss is so minimal and intelligently managed that it’s imperceptible to the human eye, even after intensive post-processing.

IntoPIX and TicoRAW: The Masters of Visually Lossless Compression

The key to this capability lies in Nikon’s collaboration with IntoPIX, a Belgian company founded in 2006 specializing in high-performance video and image compression technologies. Their flagship product, and the technology behind Nikon Z9’s HE* compression, is TicoRAW, a RAW codec designed to preserve maximum image fidelity, particularly in critical areas such as highlights, shadows, and fine details.

On their website, IntoPIX states that their software can:

• Reduce file sizes by up to 16 times compared to uncompressed RAW, while preserving visual quality similar to the original.
• Provide low latency, which is important for high-speed image acquisition such as that of the Z9.
• Be energy-efficient, which is beneficial for battery-powered devices.
• Maintain the sensor’s full dynamic range and color depth, allowing for flexibility in post-production.

TicoRAW analyzes RAW data and applies an intelligent compression algorithm that identifies and removes redundancies without compromising information crucial for human perception. At least, that’s the theory.

Compatibility: Opening HE* Compressed NEF Files Across Software

While HE* compression offers significant advantages, it’s crucial to consider compatibility with popular photo editing software. Not all programs can natively open these compressed files, especially on older operating systems or with software that hasn’t been updated.

A significant limitation that Nikon Z9 owners encounter is the inability to natively open HE* compressed NEF files on a Mac using default system applications or quick previews. This doesn’t mean the files are unusable, but that they require specific software to open.

Fortunately, the major and most popular post-production software have adopted support for HE* NEF files, including on Mac. These include:

• Adobe Lightroom Classic / Lightroom
• Adobe Photoshop
• Capture One
• DxO PureRAW
• Affinity Photo 2
• Nikon NX Studio (Nikon’s proprietary software)

Practical Advantages of Smaller HE* Files

Shooting smaller files offers three significant benefits for photographers:

• Less Storage Needed for Travel Photographers: The reduced file size is beneficial for travel photographers managing numerous images. It means fewer memory cards, less external storage space, and simpler data backups.
• Faster Image Transfer Speeds: When transferring large numbers of images, a smaller file size speeds up the process.
• Higher Burst Rates and a Larger Buffer: With HE* RAW files, it is possible to shoot closer to the camera’s maximum frame rate. The buffer also takes longer to fill, even without requiring a top-of-the-line memory card.

Nikon’s HE* files average 30-32 MB each, whereas .NEF lossless files average 50-54 MB, with some reaching 70 MB.

Field Testing: Absence of Visible Issues

I’ve used the Nikon Z9 extensively in a wide range of real-world scenarios, shooting thousands of images in both Lossless RAW and HE* RAW formats. It’s important to clarify that, prior to this article, I did not perform shots under identical laboratory conditions for a pixel-by-pixel comparison. However, I subjected the files to a wide variety of standard post-production processes: I tuned the exposure and the contrast, recovered the shadows, pulled the highlights, and experimented with sharpening and noise reduction both with Capture One and PureRAW.

What I found, after this prolonged and varied use, was a complete absence of visible issues or limitations in the image quality of the HE* RAW files. I didn’t notice compression artifacts, color banding, loss of detail in critical areas, or reductions in dynamic range that could distinguish HE* file from a lossless file. This remained true even after post-processing.

However, I wanted to verify beyond my field impressions and put some HE* files through a more rigorous test. I’ll share those results next.

Lab-Style Tests and Findings

To conduct a meticulous comparison, I took some classic banknote images under controlled conditions to emphasize even the smallest nuances. The protocol involved shooting indoors with the Nikon Z9 camera mounted on a tripod and operated via a remote cable. I employed ISO settings of 64, 500, 3200, and 6400, taking seven different exposures at each ISO value, ranging from extremely bright to extremely dark, for both Lossless compressed .NEF and HE* formats. This resulted in a total of 56 distinct files. (I used a Nikon Z 105mm f/2.8 S Macro lens, which is incredibly sharp. I focused with AF-S using Pinpoint, and to avoid any difference, I didn’t refocus between the shots.)

To check for compression artifacts, I did a thorough pixel-level inspection at 400% magnification, from which I generated these screenshots. I could not find any differences that were meaningful. Instead of boring you with all 56 photos, here are a few sample comparisons. In each pair of images, the first photo is HE*, and the second is NEF Lossless compressed.

First, a comparison at ISO 3200:

Any differences in compression artifacts between those images are too small to be meaningful.

Next is a pair taken at ISO 500, underexposed two stops, and recovered. Here, I also see no major differences – only a little bit of differing contrast in the eyebrow, and perhaps in the letter “T” on the right, but not what I would consider a meaningful compression artifact:

And finally, a comparison at ISO 64, where the story is the same:

I conclude that under practical field conditions, any visible differences or compression artifacts are negligible, if not simply zero.

Chroma and Luminance Variation

Another aspect of compression is that it can cause subtle changes in colors and luminance values.

For example, a crop from a compressed JPEG will show a relatively low amount of color and luminance variation, because the JPEG uses a strategy of replacing some colors with similar colors to reduce file sizes. In the graph below, a higher variance value is a good thing, representing less compression. (I tested at multiple shutter speeds simply to provide a greater number of sample photos for the comparison.)

Take a look at the differences between a Lossless RAW file (dashed lines) and a JPEG file (regular lines), and you’ll see that the JPEG has a much lower chroma variance – AKA, blockier colors. In particular, the dashed line representing chroma variation is much lower on the JPEG, a sign of high compression:

Although the luminance variation in pixels is overall quite similar, it is clear from the difference in chroma variation that the lossless RAW is superior to a JPEG.

On the other hand, here is the corresponding graph at ISO 64 comparing Lossless RAW (dashed lines) to Nikon’s HE* format (regular lines):

Here, there is no statistical difference between either the luma or the chroma variation at the pixel level. The dashed lines are effectively interchangeable with the non-dashed lines. In other words, Lossless Compressed and HE* are comparable in image quality.

The results are actually similar at all ISOs, but here is the ISO 3200 comparison for reference:

There is a little more variation here, which is to be expected at ISO 3200. The random nature of noise means that there will be a little less correlation between any two photos at a higher ISO. However, according to statistical testing, this variation is within the expected bounds, so there is no measurable difference between them.

A slightly different way of looking at the variation is to measure entropy rather than variance. Although I won’t go into technical details, practically speaking, entropy can be more sensitive to color replacement compression than variance. That’s because theoretically, a camera company could do color replacement in such a way that hides changes in variance, whereas that would be more difficult to do for entropy. Here is the normalized entropy measure for luma and chroma noise (the dashed lines representing Lossless Compressed files, and the regular lines representing HE* files):

Again, there’s no statistical difference. What can we surmise from these computations? The HE*, while being a lossy algorithm, does not seem to be making any significant changes to colors at the pixel level compared to the crude compression of the JPEG algorithm. Therefore, although there are indeed some very minor differences between lossy and HE* files, the differences in color and luma variation are likely to be so minimal as to be invisible to the eye in any sane use case.

Conclusion

Nikon’s HE* RAW compression, using IntoPIX’s TicoRAW technology, marks an underrated advancement in digital photography. Nikon’s claim that the files are “visually identical” to lossless RAWs is supported by practical use. Photographers have long faced the choice between maximum RAW quality and smaller file sizes, and I believe that this is the best balance we have seen yet. Even for something like landscape photography, I see no practical reason to shoot Lossless Compressed any more, and certainly not for something like action or wildlife photography. HE* is simply a better use of space on the card, without any practical image quality drawbacks.

I hope you found this article on Nikon’s HE* .NEF files helpful. If you have any questions or feedback, please don’t hesitate to leave them in the comments section below!