Our Canon and Sony Lens Reviews Will Have Imatest Data Comparable to Nikon

I just wrapped up five long days in our lens testing lab with some really good news! After extensive tests, I’ve determined that we can measure Canon RF and Sony E lenses in the lab, while producing comparable sharpness charts to our existing Nikon Z / Nikon F lens tests.

This is a project that I’ve been working on for a very long time. I’ve aways wanted to add Canon and Sony lenses to our testing database, but as soon as I do, it’s inevitable that people will want to compare them against our existing Nikon F and Nikon Z lens reviews. Heck, I want to compare them! I’m a Nikon photographer, but some part of me is morbidly curious what I’m missing or gaining over similar lenses from Canon and Sony. Not to mention, one of the best ways to spur a lens manufacturer to improve their product is to show that another company has already done it better.

For this article below, I’ll dive into the full technical details of the lab tests I ran in order to determine how to get comparable Imatest numbers across brands. If you don’t care about the mathematical background behind everything, you can jump straight to the conclusion – basically, that the results were better than I expected, and it will be possible to compare Nikon vs Canon vs Sony lenses on similar footing in our future views. While we can never get 100% identical numbers across brands, it turns out that we are able to get quite close.

Background

In the early years of our lens tests at Photography Life, Nasim tested everything on the 36-megapixel Nikon D810. He then switched to the Nikon D850 as soon as it was announced, due to the 45 megapixel sensor, and began updating all of our old lens reviews with new data. To this day, every single lens review on Photography Life has comparable numbers, aside from a few older Nikon F-mount lenses which have an old-looking chart design.

Once Nikon introduced the mirrorless Z system, we confirmed that the Imatest numbers from the Nikon D850 are effectively the same as Imatest numbers from the Nikon Z7. Since then, we have used the same Nikon Z7 body in the lab when testing all of our Nikon lenses, both F mount and Z mount, so that all lens tests we run are fully comparable to one another. (By the way, the sharpness test we perform on our 7-foot chart is Imatest SFRPlus with MTF50 measurements, which you can read about in detail here.)

This method works great if you only ever need to test Nikon lenses. However, even though I’m a Nikon shooter, this is Photography Life and not Nikon Photography Life! Some of our writers have already reviewed a few Canon and Sony lenses, but never with Imatest data. I would like to start reviewing more lenses from different brands to be the broadest possible photography resources that we can be.

I will say it bluntly: There is no perfect way to get numbers from one camera to match numbers from another. However, I know that when we display our sharpness charts, people will use them to compare numbers cross-brand. If we only ever generate charts with totally incomparable numbers, it is doing everyone a disservice. Thus the tests below.

The Tests

I ran our usual MTF test in Imatest for the following eight setups:

• Sigma 40mm f/1.4 Art (F-mount) on Nikon Z7 with Nikon’s FTZ adapter
• Sigma 40mm f/1.4 Art (F-mount) on Canon EOS R5 with the Novoflex Nikon F to Canon RF adapter
• Sigma 40mm f/1.4 Art (F-mount) on Sony a7R IIIA with the Monster Nikon F to Sony E adapter
• Canon EF 35mm f/1.4L II on Canon EOS R5 with Canon’s EF-EOS R adapter
• Canon EF 35mm f/1.4L II on Nikon Z7 with the Fringer Canon EF to Nikon Z adapter
• Canon EF 35mm f/1.4L II on Sony a7R IIIA with the Metabones Canon EF to Sony E adapter
• Sony 50mm f/1.2 GM on Sony a7R IIIA
• Sony 50mm f/1.2 GM on Nikon Z7 with the Megadap Sony E to Nikon Z adapter

Unfortunately, there is no way to mount a Sony E lens on a Canon RF camera. No such adapter exists (or will probably ever exist). Note that the seven adapters used for this test are “hollow” – in other words, they don’t introduce glass elements of their own to the optical path. If you’re wondering why I chose so many different adapter brands, there is no single company that makes all the necessary adapters, so I simply used what was available on the market.

Finally, I picked these three particular lenses because they have a reputation for exceptional sharpness – which they lived up to. I wanted to collect data from the highest peaks of sharpness to the lows of f/16, so that I could have as much information as possible when creating a multiplication factor between the three cameras (more on that in a moment).

How to Use These Tests to Get Comparable Imatest Data

Every step of this process was chosen with the goal of getting the most comparable possible Imatest data across camera brands. For that reason, I picked the Canon EOS R5 and the Sony a7R IIIA because their sensors have 45.0 and 42.4 megapixels respectively, while the Nikon Z7 has 45.7. This similarity alone does not guarantee that the Canon and Sony Imatest numbers will match our existing Nikon test results. Many differences remain: differences in sensor design, filter stack, and image processing, among others. The Canon EOS R5, for example, has a weak anti-aliasing filter, while the other two cameras have none. In short, the cameras are as similar as possible, but there is still no way to get comparable charts across the three brands without employing a multiplication factor.

How to find that multiplication factor? I chose to test the same lens on all three cameras, then graph the differing results and see if there was a clear trend line that allowed me to translate one camera’s results to another’s. I still needed to start with three cameras of similar resolution, both to get a similar starting point for the MTF tests and to allow our chromatic aberration measurements (which are measured in pixels) to be directly comparable across brands.

By the way, if you’re wondering, I cannot go around the whole issue by testing Canon’s and Sony’s lenses directly on the Nikon Z7 using adapters. For one thing, there is no Canon RF to Nikon Z adapter. And while there is an adapter for Sony E to Nikon Z, as you’ll see below, using the adapter reduces critical sharpness in the corners of the frame, so it’s not a viable option, either.

First Test: Sony 50mm f/1.2 GM on Both Nikon and Sony

Maybe you’ve heard before that there are sharpness issues when using an adapter to mount mirrorless lenses from one company’s camera to another’s. Leica lens users famously have experienced some corner sharpness and color shift issues when adapting their lenses onto Sony mirrorless cameras (especially wide-angle lenses whose rear element sits close to the sensor). This is largely because of a difference in Sony and Leica sensors – different microlens designs and sensor stack thicknesses. To simplify, Sony designs their lenses for their sensors, and Leica for theirs.

Unfortunately, I found that something similar occurs when adapting a Sony E lens to a Nikon Z camera using the Megadap E to Z adapter (and it’s not Megadap’s fault – any adapter would do the same). Before I explain the situation, let me show you the two measurements that I recorded:

A few things stand out to me:

1. The highest value measured occurs on the Nikon Z7 in the center. This is a logical result, since adapters do not really impact sharpness in the center of an image, and the Nikon Z7 is prone to slightly higher MTF50 numbers than the a7R IIIA simply due to the variables I discussed earlier (including the 45.7MP vs 42.4MP sensor).
2. Corner sharpness measurements, however, are higher on the a7R IIIA than on the Z7. This is also a logical result, considering the aforementioned issues with adapting mirrorless lenses cross-brand.
3. Maybe unrelated, but the Sony 50mm f/1.2 GM is a seriously sharp lens! The central sharpness numbers, as measured on the Nikon Z7, are higher than any lens we’ve tested yet, aside from the Nikon 28mm f/1.4E. I knew it was a good choice to pick it for this analysis :)

My decision following this test was to use only the central sharpness figures of the 50mm f/1.2 GM when attempting to calculate a multiplication factor between Sony and Nikon.

Next Test: DSLR Lenses on All Three Cameras

I was curious if adapting a DSLR lens would also have corner sharpness/color shift issues, or not. Thanks to the much longer flange distance, light from these lenses is hitting the camera sensor at a more direct angle, so it’s conceivable that the issue would not be present.

After testing it, that turns out to be true! Here are the results for the Canon EF 35mm f/1.4L II tested on all three cameras:

My three takeaways:

1. The sharpness patterns are extremely similar in all three tests (i.e., the relative sharpness of center, midframe, and corners), demonstrating that the adapter-related corner sharpness issue is not present to a significant degree this time.
2. Measured sharpness numbers are highest on the Nikon Z7, like last time. Sharpness numbers on the Sony a7R IIIA are next highest, while sharpness numbers on the Canon EOS R5 are the lowest. This is not surprising considering that the EOS R5 is the only one of these three cameras with an anti-aliasing filter.
3. I measured substantial differences at the sharpest apertures, while the differences in less sharp regions (like f/16) are much more minor.

Based upon these results, my decision was to include all three areas of the frame – center, midframes, and corners – into my calculations of the multiplication factors from the Canon 35mm f/1.4L II. An identical pattern emerged on the Sigma 40mm f/1.4 Art when testing it on these three cameras, so I did the same for those results. (Although, as a side note, my adapter to test the Sigma lens on the Sony a7R IIIA broke partway through the tests – another sign that it’s still better to use native lenses when you can!)

Data Analysis

After running all eight of these tests, I graphed the data. For each data point, I made the y-value the Nikon measurement and the x-value the Canon or Sony measurement. As an example, when graphing the “Canon to Nikon” values, one of my data points was (1994, 2260), which are the two values for central sharpness at f/1.4 on the Canon EF 35mm f/1.4L II, as measured on the R5 and Z7 respectively. I did this for every measurement and graphed them below:

I have to admit, I was really thrilled when I saw the results and how tightly they were grouped. However, the simple linear regression that you see above is not the right match. Even though it’s a reasonably good fit, you can see a big problem if you look at the lower values of the trend line! Namely, that the point (0,0) is not on the line. If we ever test a very blurry lens on the Canon EOS R system (say, a Lensbaby with a super low Imatest measurement in the 200-300 range), the chart would recommend negative values when converting the numbers to Nikon Z. Obviously, that is not a workable multiplication factor.

I had my hopes, but the line of best fit cannot be fixed by simply locking the y-intercept at zero:

Although this does solve the (0,0) issue, now the line of best fit is a lot worse. In particular, it is substantially underestimating the higher values. I showed in my testing that a value on the Canon EOS R5 in the range from about 2800-3000 was similar to a Nikon Z7 value of about 3500-3700. If I used the multiplication factor above, however, it would output a result of about 3200 – making it appear that no Canon lens could ever approach the sharpest Nikon or Sony lenses, which is simply not true. We have already shown in these tests that the EF 35mm f/1.4L II is much sharper than that.

The reality is that the two linear trend lines above are flawed, for the reason that my test results don’t show linear growth in the first place! A test result of 2700 on the EOS R5 is roughly equivalent to 3400 on the Nikon Z7, but a test result of 1700 on the EOS R5 is more like 1800 on the Nikon Z7. Basically, as a lens becomes sharper, the difference between two sensors becomes increasingly more important. That’s what we’re seeing here.

So, instead of a linear regression, I next looked to a polynomial regression in the form y = ax² + bx. In order to keep the y-intercept at zero, I did not add a constant, c. That result was as follows:

No line of best fit in this situation would ever be perfect, but this one is clearly a close fit. I am not making the claim that it is definitely a second-order polynomial relationship rather than, for example, a third-order or exponential relationship – or linear at the lower values and something else at the higher values. My intent is not to investigate the full origins of the curve, but rather to have a simple curve that is predictive in the range of Imatest values from roughly 1000 to 4000 that we see in typical lenses we review. This one meets those qualifications.

After getting such a strong result for the Canon EOS R5 to Nikon Z7 conversion, I was nervous about the Sony a7R IIIA to Nikon Z7 conversion, but ultimately I had no need to worry. That trend line, calculated in the same manner, was an even closer fit:

Conclusion

It’s hard for me to hide my excitement at how comparable these results are! I began performing these eight tests not knowing if the charts above would look nice and neat like this, or if they would resemble something more like chickenpox. But based upon these results, I intend to commence testing more Canon RF and Sony E lenses as soon as possible – knowing that I’m not favoring one brand over the others, and without the camera itself exerting undue influence on the results.

To reiterate, there will never be a flawless way to compare Imatest results across camera brands. Even with how tight these data points are, you can see some variability both above and below the trend lines. However, I believe this to be easily close enough to allow Nikon, Canon, and Sony lenses to be compared head-to-head in our reviews.

I did all this testing for one reason, which I noted in the introduction to this article: Photographers who rely upon our lens reviews will inevitably try to compare across brands no matter what. Now, they can. All of our future Canon RF and Sony E lens tests will have these equations applied so that they are comparable to our existing Nikon Z test results. Meanwhile, I will be able to test each lens on its intended camera instead of relying upon finicky adapters. In short, a score will be a score will be a score, regardless of the brand of lens in question.