As a landscape and travel photographer, I take a boatload of pictures. Sometimes I take a series of shots when I do not want to miss a moment. Other times, I take multiple shots to properly expose a scene or to capture a panorama. The result of it all is a photo catalog with tens of thousands of pictures each year. While storage is cheap and I run everything off a fairly beefy NAS (Network-Attached Storage), my photo catalog over the years has bloated to many terabytes. In this article, I will go through what I had to do to significantly trim my photo storage.
Table of Contents
The Problem with “Photo Bloat”
One of my biggest problems I have had in the past, was importing too many images into my photo catalog. I would come back from a trip, then unload the whole memory card into Lightroom. I would delete truly bad images, edit a few of the solid candidates, then keep the rest. Until I started really paying attention to how I shoot and how I cull through my images, the process was somewhat messy for a number of years. As a result, I would average between 20-40 thousand images each year, which took significant effort to store and back up. This year, I decided to really clean up my photo catalog and add an additional process to my year end workflow.
The problem with most photography workflows that entail long-term storage of images, is that it results in a “photo bloat”, with hundreds, if not thousands of images, taking too much storage. Are you ever going to touch a blurry, badly exposed or a duplicate image? Not likely.
The thing is, if you have not gone back to pick another candidate for editing from your set of existing photos, chances of you ever coming back to do that in the future are close to nil. Those images will likely never be touched again. So why keep them on your computer or storage in the first place?
That’s what I decided to do with my old images.
What to Delete
Let’s go through what content should be a candidate for permanent deletion:
- Duplicate images
- Blurry images
- Badly exposed / processed photos you will never use
- Non-picked images that you will never post-process
- Badly edited TIFF and HDR / panorama-stitched images
- Any video content that you will never use
What Not to Delete
What content do I recommend against deleting from your catalog? Let’s go through those, one by one:
- Original RAW images of your top picks
- RAW images from HDR / panorama sequences
- Any other images or video you are unsure of
Deleting original RAW files, even if you are not likely to go back to re-edit them is always a bad idea. Post-processing technologies and tools we use today will not be the same tomorrow. Think of all the amazing tools that have been introduced to us in the past 10 years. Going forward, we will be seeing more AI-based post-processing tools that will make it even easier to quickly post-process images.
The way you post-process your images will also likely advance and get better. Once you flatten a RAW file to a JPEG, your post-processing options are thrown out of the window. Therefore, my recommendation is to never delete your RAW images. Do not worry about converting your RAW files to DNG or other formats to try to future-proof them either. I stopped converting to DNG a long time ago and I am not going to reverse that decision. Anyone who says that a RAW file might not be supported in the future is just full of it. In the past 10 years of being heavily involved in photography, I am yet to see any software that discontinued support for an old camera.
1. Backup
Before you decide on purging any content from your primary storage, it is important that you back everything up. Even if you know that you have a bunch of duplicates you need to take care of, it does not matter – back up everything from all of your storage. In case you mess up and you want to go back, you should have a full backup of all your content.
In fact, storage is indeed very cheap nowadays. If you can afford a large external drive, just keep your full backup there for a few years before you finally purge it all permanently.
2. Use a Proper File Management Tool
In order to make the process of shrinking your photo storage easier, you should use a proper file management tool that is able to assist you in quickly sorting through, tagging and finding specific images. If your file management tool comes with a built-in ability to find duplicate files, it will make the process of deleting duplicate images much easier.
For this article, I will be using Adobe Lightroom to eliminate duplicates, mark keepers and delete unused images. Although Lightroom does not come with a built-in tool to find duplicates among imported images, you can install third party plugins to do it.
3. Eliminate Duplicates
The first step is to find all duplicate images and get rid of them. Duplicate images can potentially take a lot of space and they are easy to tackle, especially if you have the right tool for the job.
There are plenty of third party and open source tools out there to find and delete duplicate images. Personally, I would recommend to use specific tools designed to identify duplicate images, not just any files. This becomes especially important when going through RAW images, which contain important EXIF information that helps identify duplicate images. For example, if two images are captured with the same camera, lens, exposure, date / time and they have identical file size, they are likely duplicates. Such files should be deleted.
I went through a number of different tools and the best one for the job turned out to be Duplicate Finder for Adobe Lightroom by Michael Bungenstock. It is freeware / donationware, and it does an excellent job at finding and properly marking duplicates in Lightroom.
To get it working on Lightroom CC, I had to download the ZIP file and manually copy it into the “Users -> ID -> Library -> Application Support -> Adobe -> Lightroom -> Modules” folder.
Once you install the tool, go to the root folder of your Lightroom, then open it up through Library -> Plug-in Extras -> Find Duplicates. Here is the way I configured the tool:
My 2019 catalog contained a total of 10367 photos. After it scanned through the catalog, it was able to identify 900 duplicates, as shown below:
The plugin will automatically create a “Duplicate” keyword and add it to the “Duplicates” Collection. It will leave out one image and mark all others from the same duplicate batch as “Rejected” (configurable through one of the tabs), so that you can quickly review the duplicates and delete them.
Unfortunately, the plugin is not perfect – if you have a series of images that you captured through shooting in a burst (with identical date / time), it will still identify those photos as duplicates. I recommend going through those photos and eliminating the ones you truly don’t need.
4. Properly Mark Your Keepers
Once you go through all the duplicates, the next step is to narrow your images to the ones you are truly intending to keep in your photo catalog. Personally, I always mark images I intend to edit with 1-5 stars as part of my image culling process. However, once I start post-processing, I sometimes pick images that I did not originally mark with stars. Those images, as well as the ones I already post-processed (via Lightroom or Photoshop), are always starred at least with one star.
This makes the process of identifying candidates for post-processing, post-processed images or keepers quite easy. If you have not been marking your keepers and edited images, now is the time to do so.
If you want to skip this process and simply find all the images you have previously edited, there is a filter within Lightroom that you can use in order to show all the edited images. You can access it by bringing down the Library Filter, then picking “Edit” from the drop-down. Once you do that, you will see two filters “Edited” and “Unedited”:
Clicking on the “Edited” filter will only show all the images that have been previously edited. You can mass select these images by using the CTRL+A / CMD+A shortcut, then either star them, or attach a temporary color label that you can use later to filter these images out.
5. Select All Keepers
The next step is to select all keepers. Either use the above filter, or use a particular filter like “Attribute” (or a combination of the two) in order to pick particular images that you have previously edited, or saved as candidates for editing. For my images, I marked every image I intend to keep with a star, so using the “Attribute” filter with a single star (≥ 1 star), only showed these images:
From there, I simply selected all images by using the CTRL + A / CMD + A shortcut:
The next step was to undo the filter and inverse-select the images I did not intend to keep.
6. Inverse-Select Candidates for Deletion
Once you select the images, click the “None” shortcut on the Library Filter, which will reveal the rest of the images. Now go to “Edit -> Invert Selection”. This will deselect all the selected images, and select the images that were not previously selected, as shown below:
7. Delete Images
The next step is to create a Collection where you will put all the candidates for deletion. Go to Library -> New Collection (CTRL + N / CMD + N Shortcut), then give it a name like “To Delete”. Make sure that “Include selected photos” is checked:
This will create a new Collection that you can navigate to any time, review images and flag the ones you want to get rid of as “Rejected”.
After you review every image, either select all the images from the Collection and press the Delete button to get rid of the images, or navigate to “Photo -> Delete Rejected Photos” (CTRL+DEL / CMD+DEL shortcut). When Lightroom asks if you want to remove the photos from the catalog, or want to delete them from disk, make sure that you select “Delete from Disk”:
8. Consider Shrinking TIFF and PSD Files
If you use Photoshop to edit your images, you will likely find a bunch of large TIFF / PSD files along with your RAW photos. Those files typically take a lot of space, especially if more than one layer is preserved.
To free up a bunch of space, consider flattening your TIFF / PSD files. If you want to preserve layers that are likely to change in the future, or contain your edited masks, keep those, but discard others. High resolution panoramic images can particularly take too much storage space – consider flattening those images to keep their size to the minimum.
Doing the above steps helped reduce my Photo storage considerably for 2019 alone. I am planning to go through the same process for all previous years, which should reduce my storage needs by at least 50%.
I hope you found this tutorial useful. If you have any questions or concerns, please let me know in the comments section below!