Why Downsampling an Image Reduces Noise

One of our readers, Mike Baker, sent the below email to me today. I thought it was a great and interesting analysis of why downsampling an an image reduces noise, so I decided to share it with you (with his permission, of course). Trying to digest this stuff makes my head spin, but it is a great read. You might need to read it several times to understand what he means, especially with all the mathematical formulas (I had to):

You recently commented about downsizing a high-resolution image to a lower-resolution in order to reduce the apparent noise. While I knew that this is an effective way to reduce noise visible in the images, I had not thought in much detail about the technical reasons why this works.

After a long evening’s thought on the subject, and running a few questions past my friend and fellow engineer, I believe I have a (reasonable, though perhaps not perfect!) handle on the subject…

If the image signal and the image noise had similar properties, averaging neighboring pixels in order to reduce the resolution would not improve the signal-to-noise ratio. However, signal and noise have different properties.

There is (in general) no relationship between the noise in neighboring pixels. Technical junkies call this “no correlation”.

Correlation is the long-term average of the product of two signals N1 x N2. If two signals have no correlation, then the mean of their product is zero.

The signal in neighboring pixels has a high degree of correlation. If you add uncorrelated signals, then their “power” is added, meaning the combined signal is the square root of the combined power.

N_comb = sqrt(N1^2+N2^2) and for N1 = N2 = N we get N_comb = sqrt(2)*N, where N1, N2 are root-mean-square (RMS) values of the noise.

However, if signals are highly correlated, then their sum is effectively the sum of their magnitudes:

S_comb = S1+S2 and for S1=S2=S we get S_comb = 2*S

So, if we add the content of two neighboring pixels, we get:

SNR_comb = S_comb/N_comb = sqrt(2)*(S/N)

So, the signal-to-noise increases by square root of two, which is about 40%.

Now, you may say that the signal in neighboring pixels is not always 100% correlated. The correlation between the signals depends on the image content. If the image content is very smooth, the correlation is high. If the image content varies very fast, the correlation is low. Of course, noise will be more noticeable in smooth areas and the effect of resampling the image will be stronger.

Adaptive noise filters take into account the absolute signal-to-noise and the image content. They reduce the resolution more in areas that are smooth and have poor signal-to-noise and keep the original resolution in areas that have strongly varying image content and high signal-to-noise. You can think of it as a joint optimization of SNR and resolution.

Now, we also need to look into the different sources of noise:

  1. The first source of noise is dark current which is caused by electrons that accumulate in the individual pixel well, even if there are no photons entering (lens cover on). Dark current becomes dominant for very long exposures. For normal exposures the errors from trapped electrons are negligible.
  2. The second source of noise is the read-out noise. This is essentially generated by two sources: A) Noise added by the amplifier and B) Noise generated by the analog-to-digital converter. It is a fixed amount of noise that is added to each image during read-out. When you choose the ISO setting on your camera, you essentially set the read-out gain and therefore the read-out noise. The higher the ISO, the higher the read-out gain and the less read-out noise. Of course if you pick an ISO which is too high you will get signal saturation. So for low-light situations always pick an ISO that is no higher than needed to capture the image you want.
  3. The third source of noise is called “quantization noise” and is a bit harder to understand. It has to do with the fact that (in low-light conditions) we don’t sample a smooth, continuous flow of photons but rather discrete bunches of photons. The problem is, that a source of light does not produce a stream of photons that are spaced equally in time. So, if you image a low light source that sends out (on average) 100 photons per second, you may receive 90 photons for the first second, 105 for the second etc.. The average error will be on the order of the square-root of the number of photons (or electrons in the pixel sensor well). A typical sensor well contains between 20,000 and 60,000 electrons when fully charged. The maximum amount depends on the pixel size. A sensor well with 20,000 electrons has an error of approx +/-141 electrons when fully charged or +/-0.7%. A well with 60,000 electrons has an error of approx +/-245 electrons when fully charged or +/-0.4%. While we may be able to reduce dark current and read-out noise by cooling the sensor, there is essentially nothing we can do about it. If we keep on shrinking the pixels, we will have smaller and smaller electron wells and less and less electrons trapped.

    The above errors of 0.7% or 0.4% appear rather small and we would not be able to notice them. However, in low-light situations, sensor wells will be only partially filled. If we only manage to trap 1000 electrons, the error becomes 3%. If we only trap 100 electrons, the error becomes 10%.

    Notice that the term “quantization noise” has nothing to do with the signal quantization by the analog-to-digital converter. It has to do with the fact that your signal actually arrives in quantums of energy.

What do you guys think? Anyone wants to challenge Mike’s analysis? :)


Support Photography Life!

We constantly work hard on adding unique content, gear reviews and up-to-date photography news, in addition to continuously expanding the site with new sections and useful content. However, we need your continuous support to deliver the best content and allow our website to expand its reach. If you would like to help us out, please consider purchasing gear from our links to our trusted partners like B&H Photo Video and Adorama. It won't cost you anything, but it will help us pay our contributors, hosting and other expenses to run this website. In addition, if you feel like we do a good job, you can pledge one-time or monthly. We do not run any advertising at Photography Life to keep it clean for your viewing pleasure, so your support is extremely important for us to keep it that way.

Please see the Support Us page for our partner links and a donation form. Also, don't forget to follow us on Facebook and Twitter!
Avatar of Nasim Mansurov About Nasim Mansurov

is a professional photographer based out of Denver, Colorado. He is the author and founder of Photography Life, along with a number of other online resources. Read more about Nasim here.

Comments

  1. 1
    ) Craig

    Nasim, this is crazy stuff, way over my head :) A very interesting read on types of noise towards the end!

  2. 2
    ) Wilson

    Interesting reading by skipping the formulas :)

  3. 3
    ) Ravi

    Nasim,

    Its way over my head but I am able to grasp the principle behind the explanation. But I am still left with the original question of the topic.

    His explanation seems to explain the “image capturing process” and the noise thereof. Once we have the image from the sensor onto the PC, how does down sampling the image reduce noise?

    Its late and I could have completely missed something here.

  4. 4
    ) Fred

    Interesting. May be I could use an example. Suppose I have a 3×3 picture (grand total of 9 pixels). For simplicity, let’s assume the pixel are of equal value, 10 (perfectly correlated). Each pixel will have some random noises, say either +1 or -1. Before any downsampling, the signal amplitude is about 10x noise amplitude. if we sum all the 9 pixels up and average them into a single pixel, the downsampled pixel will get a value of 10. But for the noise, since they are NOT correlated, in some pixels the noise could be +1, in some pixels the noise could be -1. So the noise level actually will be reduced as the noise levels could cancel each other out. Thus the downsampled pixel will have a signal amplitude of 10, but noise level much less than the original 1.
    I hope I am making sense. :)

  5. 5
    ) Tomasz

    Hi Nasim,

    it is great explanation. First part could be put in 2 cases to keep it simple:

    image with smooth content ->
    has high signal correlation ->
    so the noise is increased by 40% ->
    we have more noise ->
    resampling is needed and it descreases overall noise

    image with varies content ->
    has no signal correlation ->
    the noise keeps the same level ->
    we have the same noise ->
    resampling not needed

    Adaptive noise filters based on signal correlation (high correlation, no correlation) does resampling only on areas where it is needed (on smooth areas).

    Points 1 and 2 are clear however Point 2 is interesting when it says that you should not overexpose in low-light as it generates more noise. I read many times that when you overexpose a little you can get less noise and it is a common practice.

    Tomasz

    • 13
      ) Michael Baker

      Hi, Tomasz–

      Regarding your comment: “Point 2 is interesting when it says that you should not overexpose in low-light as it generates more noise. I read many times that when you overexpose a little you can get less noise and it is a common practice.”
      —————

      The simple answer to this is that (slight) overexposure masks amplifier noise. Examined in more detail, it is related to the dynamic range of the sensor and on how you interpret what you see in a *slightly* overexposed DSLR image. Essentially, when a CCD image sensor is (slightly) overexposed, apparent noise is “flattened” by the increasingly limited dynamic response. The dynamic range of a DSLR sensor is the ratio between the maximum charge that the sensor image wells can gather and the minimum accumulated charge that just barely exceeds the base noise level. Just as the luminance response curve starts to flatten at the threshold of the high end of the dynamic range limit, noise is partially masked as the wells approach saturation. The noise added by the use of very high ISO gains is still there, but it is gradually masked by the approach of the saturation point.

      For what it is worth, I make no claim to being an expert on this topic of DSLR CCD image sensors– however, I am a retired TV broadcast engineer and I do have some background in the technical aspects of noise considerations in TV cameras (going all the way back to the days when Plumbicon image tubes were king!) but the principles are pretty much the same.

      Regards–

      Mike Baker
      ———————–

    • Tomasz
      This point confused me a bit too. I think people can draw the wrong conclusions here.

      My interpretation is that the rule of thumb is (obviously) that you should neither over- nor under-expose your image. You should expose it like you want it to look in the end, always using the lowest ISO value that is practical.

      If you underexpose, you’ll have to boost the image brightness afterwards. This has a similar effect as raising the ISO (= more amplification noise = more image noise). If you achieved this underexposure by dropping the ISO the required post-processing brightening of the image will more than undo everything gained by using the lower ISO (in terms of image noise).

      Alternatively, if you overexpose, you might get blown highlights – not good. If you overexpose by raising the ISO you will, in addition to the possibly clipped highlights, also introduce more amplification noise. This noise gain will be partly cancelled when you again darken the image in post-processing, but the risk and effort is not worth the result except, possibly, in very specific situations (like astro-photography).

      But let’s say you’re into astro-photography and want to reduce noise as much as technically possible.
      If you only overexpose slightly (say, 0.5 stops), and then correct the exposure afterwards, you will reduce the effect of the read-out noise. This is because read-out noise is independent of the camera sensor’s exposure, and this noise signal’s intensity is reduced along with the rest of the image when you apply the opposite (-0.5 stop, in this case) post-processing compensation. But this trick will only be worth anything if you didn’t achieve the initial overexpose by raising the ISO, since raising the ISO raises the amplifier noise and masks any improvements to the read-out noise in the final image.

      Mike – do you agree with what I wrote?

      • 15
        ) Michael Baker

        Hi, Francois & Tomasz-

        There are a LOT of subtle (and subjective!) aspects in this subject,
        but in general, yes, I agree with your post…

        In summary- (as I saw Nasim say once) go out and take pictures!

        Mike Baker
        ——————

  6. 6
    ) Bandit Phachanda

    This is good articles to read learn about camera DSLR and interest
    Thank you….

  7. 7
    ) Roman

    No way, Nasim, this is way too technical for an old-fashioned photographer like me. I’ll just stay here quietly with all my compositions and aesthetic qualities.. :)

    Though I’m glad there are people like Mike who make our lives easier when it comes to reducing noise in practice. :)

  8. 8
    ) Bob

    Everyone learns this on day one of any intro to photography class… :)

  9. 9
    ) Peter

    Uuuugh! Uuuugh! Is it April 1?

    Buy NIK Define 2 and skip this article.

    Nasim, you were batting 1000, but this article dropped your average down to .987.

    This article is more than “getting into the weeds”, it’s getting into the molecules.”

  10. Hey thanks for posting this – it is a good read for people who like engineering.

    For the normal people I’ll try to make an easier explanation (Mike, please forgive any inaccuracies this simplification introduces):

    If you downsample an image to lower resolution you are essentially averaging neighbouring pixels.
    The value of each pixel consists of two parts: signal (the good part) and noise (the bad part).
    * The signal is kindof smooth, and changes slowly across pixels. So the average of two neighbouring signal values is more or less the signal value itself (e.g. the average of 50 and 52 = 51).
    * The noise basically random for every pixel. It can be positive or negative, and on average it is zero. Therefore the noise of neighbouring pixels tend to cancel out if you average them. (e.g. the average of -10 and +8 = -1)

    Here is the example to make it easier to understand:
    You start with two pixels that have values of 40 and 59. They should actually be 50 and 52 if there was no noise, but you and the camera have no way of knowing that. But because the noise is random and zero-mean, you know that it will tend to cancel out. So if you downsample the image by 50% you average these to pixels to get (40 + 59)/2 = 99/2 = 49.5. Bingo – this is very close to the ideal average value you would have gotten if there was absolutely no noise, (50+52)/2 = 101/2 = 50.5.
    So in this example you almost eliminated the noise at the cost of halving your resolution. Instead of two pixels with noise values of 10 and 8, you now have one pixel with a noise value of 1.

    PS: This is the same reason you get really nice noise-free images if you average a lot of noisy photographs (as long as nothing in the scene moved, and your camera stayed perfectly still). The signal does not change over time, but the random noise tends to cancel out, and what you are left with is a smooth noise-free image. People use this technique for astronomy photographs. In practice this means that you have to let your camera track the stars (they move overhead), so of course it’s easier said than done.

    • 22
      ) Wilson

      Nice simpler explanation!

  11. wow, this is way too much for me.. I need a drink now!!

    • 12
      ) Peter

      Have one for me, too. In fact, have 3-4 for me.

  12. 16
    ) Amit

    Nasim, I have a question from you.
    It is a common beleif among the photographers that the too many pixels are bad because of the increased noise. If the noise can be reduced by downsampling the images then is there _any_ harm in having more megapizels? If you want details then don’t downsample the image but if you want less noise then simply downsample the image. This way you will get the same noise if the sensor had less number of megapixels.

    Lets have a scenario where there are tow similar sized sensors with similar sensor technology.
    sensor 1: 12 MP, Noise = N1 at a particular setting
    sensor 2: 24 MP : Image taken at similar setting and the image down-sampled to 12 MP. Lets say Noise now is N2 (for the 12 MP image)

    My question is: do you expect N1 to be similar to N2 or do you expect N1 to be lower than N2?

    • 17
      ) Amit

      ‘megapizels’, ‘beleif’, ‘the too many’
      too many mistakes in a couple of lines :)

    • 26
      ) Andrew Roos

      That depends on which source of noise dominates.

      When photon shot noise (noise resulting from the quantum nature of light, which Mike calls “quantization noise”) is the dominant noise source, then downsampling a higher resolution sensor should result in the same image scale signal to noise ratio. This is because increasing the number of photosites does not increase the amount of noise (which is a fundamental property of the light coming to the sensor) but simply redistributes it amongst more buckets; and down-sampling combines these buckets to get the same signal and noise as a lower resoluion sensor would have.

      When read-out noise dominates, then a down-sampled image from a higher resolution sensor will have more noise than the image from a lower resolution sensor, since the noise is generated by each photosite (and amplifier and ADC conversion) so more photosites means more noise.

      Comparison of the NEX-5N (16 MP) and NEX-7 (24 MP) SNR measurements on DXOmark shows no degradation in image-scale SNR despite the reduction in photosite size which suggests that the present state of the art allows photosites as small as 4 um to be dominated by photon shot noise.

  13. 18
    ) Peter

    I ask: “Ansel, where are you when we need you? How did you manage your genius without knowing about this?”

    Ansel replies: “Take heart my son. Do not get lost in the forest for the beauty of the trees require no technical equations. Your vision is worth more than the N1 or N2. Beauty has no formula.”

    Me: Wheww! Thanks, Ansel. I was beginning to worry. I feel OK, now.

    • 19
      ) Ravi

      Peter,

      From what I have read about him, he used the best equipment available at that time and a Pentax light meter. Not to mention he was an expert in Photoshop of that time, a darkroom. He was a gifted photographer no doubt. But he also used the best tools available to him. But in the end, yes its the vision that matters.

      • 20
        ) Peter

        Actually, Adams had 3 exposure meters: one SEI and two Westons.

        • 21
          ) Ravi

          I see. I got that info from here and he is a well known photographer as well…

          http://www.kenrockwell.com/tech/meters.htm

          • 23
            ) Peter

            Here’s where I got my info, near the bottom of the opening page:

            http://www.pbs.org/wgbh/amex/ansel/sfeature/sf_packing.html

            I read Rockwell every day, but he does make mistakes now and then. His camera recommendations need to be triple checked based on personal experience. He has clear biases.

            • 24
              ) Ravi

              Peter,

              You are right about him. And I don’t want to talk about him here. Let’s leave it at that. :)

  14. 25
    ) Andrew Roos

    Mike’s analysis is correct. However his use of terminology may be a little confusing. What he calls “quantization noise”, caused by the quantum nature of light, is usually called “photon shot noise”. The term “quantization noise” is usually used to refer to the noise introduced by the quantization process in the A-D converter. Since an A-D converter converts infinitely variable analog signals to a discrete number of steps, it has to approximate the value of the analog signal. This approximation introduces an error, and these errors are the source of quantization noise in the common use of the term.

  15. 27
    ) Rod

    Hi Nasim,

    I got to this post after reading your article on the new D800 to try to find out more about how ‘downsampling’ reduces noise. Afraid like most people before me, the technical explanation is way too complicated for me to understand.

    What I am more interested in is how to ‘downsample’ and if possible using Lightroom 3.5 or PSE10. Perhaps you have written on this subject already but I have not found it yet on your site. Can you advise?

    Regards.

  16. 28
    ) Lloyd

    Hi Nasim,

    I just want to ask if all cameras does “downsampling”? What I mean is, how does all cameras produce “medium” and “small” images from their native size of “large”. For example, a dslr has a 24mp sensor. So when set to “large”, the sensor will utilize all pixels. Now what i want to know is how does the “medium” and “small” images produce? Does the camera use only 12mp when set to “medium” and there is a mechanism that block the rest of mp in the sensor? or does the camera uses all of the 24mp and downsample it to the users setting of “12mp” medium and “6mp” small.

    Hoping for your response.

    Regards.

Leave a Comment

*