Over the holidays, I finally decided to start a thorough clean-up of my Lightroom catalog and images. Many years of shooting resulted in accumulation of almost 400 thousand images and videos in original formats, and I got tired of trying to keep my data backed up and synced between different storage solutions. I glanced at some of the photos in my Lightroom catalogs and realized that between personal and commercial work I have done over the years, most of the images I have been keeping are likely never going to be touched again. So what is the point of keeping them in the first place? It was time for a massive purge…
All photographers start their digital camera journeys with lots of camera clicks, and I was no exception. The first few years were filled with the following:
- “Experimental” images of pretty much anything around the house
- Images of the same subject and scene using different camera settings
- Poorly composed images without clear subject(s) or themes
- Similar / duplicate family photos
- Photos of subjects in and out of focus (after acquiring the first prime lens)
- Many similar images of birds and mammals (after acquiring the first telephoto lens)
- Many close-up images of flowers and insects (after acquiring the first macro lens)
It was also pretty funny to discover the time I got into HDR photography. Anyone remember early versions of Photomatix? I remember that at times I would feed Photomatix JPEG images with different brightness from the same RAW image to produce that popular “HDR look”. Yup, single-image HDR photos were a real thing, even though many of us wouldn’t admit it :)
People talk about the “HDR hole“, but the “panorama hole” turned out to be much worse for me, since I ended up doing panoramas of anything that looked pretty at the time. Below was my sequence, in this order:
- Horizontal panoramas (whoa, I can do this?)
- Unstitchable panoramas (nodal point and parallax discovery)
- Horizontal panoramas in vertical orientation (more resolution baby!)
- Vertical panoramas in horizontal orientation (tall prints anyone?)
- Ridiculously wide panoramas (not sure why, but will do it anyway)
- Multi-row panoramas (first panoramic head)
- HDR panoramas (Photomatix + PTGui craze)
- HDR multi-row panoramas (all of the above, time for computer upgrade)
I had them all – hundreds and hundreds of awful panoramas:
The worst offender was a panoramic scene where I somehow managed to capture over 500 multi-row HDR images with a telephoto lens. The software and my computer at the time could not even process that large of a pano, let alone that many images, so I had to extract smaller size JPEG photos from the original RAWs in order to stitch it all together. At the end, it was a terrible image anyway, but for some reason I kept it all – perhaps at the time I thought that it could be useful someday? I still don’t know.
Anyone remember a “pro” tip back in the early DSLR days, where we would shoot a burst of photos to reduce the effect of “mirror slap” in order to get sharper images? Found plenty of those. What about post-processed images of the same photos in color, as well as in black and white just in case? Check. Images of bricks and walls for lens sharpness? Another check. Images of the same subject zoomed in and out for no reason? Yup. Wide open and stopped all the way down? Plenty of those. Vertical and horizontal shots of the same subject? Absolutely. “Creamy” bokeh highlights? Oh yeah! Exposed to the left, middle and right? You betcha. With and without filters? Certainly. On-camera and off-camera flash? Yes to that one too.
I could go on and on, and it looks like the cycle repeated with more camera gear purchases, newer software and newly-acquired tricks.
All of the above, coupled with plenty of terrible imagery meant that the clean up process for my early work would result in over 95% reduction of my image storage and Lightroom catalogs. That’s a great start!
However, the problem turned out to be much harder than I thought – trying to go through hundreds of thousands of images in Lightroom and deleting them is no easy feat, especially when images are not starred or tagged properly. Lightroom was never good for fast image culling, so I figured it would be much faster to use FastRawViewer to delete images, then go back to Lightroom and re-sync my photo catalogs to remove all the missing files. But how do you go about doing this for 400K images? Sadly, one year at a time, and one folder at a time. While I was happy that I kept my images properly organized in Lightroom catalogs, the sheer volume of images potentially meant weeks of clean-up work.
At the end, I still decided to go through the pain and follow this process, no matter how long it takes:
- Back up all images to a large external drive before clean-up (about 16 TB of total data).
- Consolidate all yearly Lightroom catalogs into a single mega-catalog, but keep images in file system organized by year. This makes it easy to see the big picture.
- Save all Lightroom metadata into sidecar files for FastRawViewer (FRV) to be able to access starred and marked up images.
- Set up FRV to dump rejected images into a dedicated folder, since FRV does not actually delete any data.
- Go through each year, one folder at a time with FRV as quickly as possible. Start by rejecting all easily identified bad images (out of focus, bad exposure, duplicates, etc), unwanted HDR and panoramic images, and other photos that will never be used. Leave any images in doubt.
- Delete the “rejected” folder if it resides within the main “Photos” folder.
- Re-sync the Lightroom catalog to remove all missing images.
- Repeat #5-#7, but this time go through images slower to only keep the strongest of the bunch.
- Optimize the final Lightroom catalog, and rename / reorganize images, if necessary.
My biggest savings are expected from commercial work (headshots, weddings, corporate photography, etc) that Lola and I did for about 10 years from 2009 to 2019. Most of those images will be permanently removed from working image and Lightroom catalogs. Even though I don’t expect any of our past clients to request images in the future, I do want to keep the strongest images as part of the portfolio.
If you want to avoid image clutter and the resulting pains in the future, it is always a good idea to properly cull images after every trip and photo shoot, then purge the rest. I wish I did that from the start.
More to come…
Photo Mechanic for titles, metadata, culling. FRV for choosing the best exposure. Carbon Copy Cloner and Time Machine for backups.
Lots of smart people in here. I read the majority of the comments and did a quick scan of the rest, and not once did I come upon “The Cloud.”
I wonder what people will do when they need to get files, en masse, off of “The Cloud.” Plenty o’drives will be purchased, and band width/time consumed. OTOH, I have hard and SS drives out the wazoo. Sometimes, I think I’m going to need a dedicated drive for passwords, alone (oh, and of course, a back-up drive).
My go to organisational tool is PhotoMechanic. I bought it years ago while it was much more affordable than it is now. I like it because I understand it, it is a proper database which doesn’t do anything else, it is very quick, and the key words lists it allows you to establish and create for different purposes are impressive.
I do use it to select and rate photos to try to avoid clutter. It is worth remembering that storage on hard disks is relatively affordable. SSD systems are much more expensive so back ups of only the top rated photos from each catalog is the solution there.
As a sports photographer I shot hundreds of thousands of photos. I have about 11 TBs of photos (duplicate storage spread on multiple external HDs). In most cases I have the original NEF files, along with the processed JPGs. I recently suffered a stroke and lost about 50% or more of my vision. (I got a backlit keyboard, and replaced one of my twin 32-inch TV monitors with a 65-inch TV.) As a result I can continue to edit photos (DxO), however at this point I haven’t shot anything new in months. I have a back-log of photos to work on, but my focus had changed from photography to writing. (During the first three months of last year I wrote and/or completed five novels; about 283,000 words.) I can’t imagine even trying to cull my stock of photos with my current disability. (Besides my eyesight, I lost the feeling in the fingertips of my left hand, which means I’m no longer a touch typist.) Life continues, but with difficulties. I’d love to buy a discounted D850, but can’t justify the purchase since I will hardly get the chance to use it. Good luck to all of you out there, and Nasim I doubt I’ll ever submit any more free-lance articles.
As an amateur I use a single LR catalogue for each solar year. Photos are physically on a single folder for each shooting session (can be one hour or a 14-days trip). If the project covers more years then I create a new catalogue that will fish data from yearly ones (I’ve one running since more than a decade).
Usually I download everything and I label photos that have no meaning for later deletion, but this happens once a year.
Photo that have a minimal meaning are labelled with a white flag and I’ll never delete them. Then stars give them a relevance ranking. Blue labelled are HDR, Stacking and Pano output, green labelled the photos needed for them (always stacked).
The only extra things I do, only for the ones that are important for me or were used in some photobooks or prints, is to export them in a .jpg full res and with a separate backup.
This will allow other people (family, friends) to have them without knowing anything of Adobe products. In the past people were leaving behind a drawer full of prints that sons would have seen sometimes in the future, likely when someone pass-by.
My jpg folders are nothing else that these drawers.
I’m not a pro, and I know that this is not efficient. But it works for me.
One option to declutter your Lightroom catalogue (especially if you shoot pictures in burst mode) is, to stack similar photos of a series, and keep just the best of it on top of the stack. You can even let LrC find all photos shot within one minute and auto-stack them. Stacking increases the speed to scroll through your pictures, if you have a lot of series of the same subject.
Hello,
I can find a lot of articles regarding culling raw files, managing catalogs, etc. But what is your workflow before culling? I mean how do you dump your memory cards? Do you copy manually from your card reader to your computer? Or to external hard drives? Do you use a specific software for this kind of task?
Thx.
Simply connect the card to the computer and bring them over in the finder in my case.
Yes, in some “basic” use cases, it does the job perfectly. But let’s say you are a professional event or wedding photographer, you get home with thousands of photos on your memory cards, maybe it would be safer to make some backups on your computer and on external storage before culling and editing your photos right?
As a former pro (shot a lot of weddings and did other commercial photo shoots), I never backed up photos before culling them. After getting back from a photo shoot (often very tired), I would simply import all images into my computer. Until the images were fully replicated to my NAS (network-attached storage), I would never erase memory card contents, to make sure that I had them in at least two locations.
Geoffrey, I personally do not bother making backups of memory cards anymore. I either import images into my computer and then start culling them, or I cull them straight out of memory cards before the import process…
I’ve never understood why anyone would bother to use a catalog. It seems to me that a catalog’s main purpose is to make it hard to change softwares if you want to. I’ve never used one. I made a folder, euphoniously called Elaine’s Pictures, and then put about 30 subfolders in it divided into categories (birds, landscapes, beach photos, etc) and within those divided into years. At the top of the folder I have about a dozen work folders for holding raw files, finished jpegs, delete folders, holding folders, and editing folders . I use my Nikon software to process raw files, convert, then use other photo editing software to work on jpegs (Zoner, 2015 version). It works like a dream, and I don’t ever have to convert to DNG’s or other proprietary formats that would ruin my raw files. On 6 external ssd hard drives I have backups of my archives and Elaine’s Pictures. A long time ago I did my purge and deleted about 30,000 photos from the archive. From then on I took a couple of old spinney hard drives and use them for deleted raw files, after removing blurry and other not worth keeping pictures. This way I never really lose anything, but don’t trip over unwanted pictures. Once in a very rare blue moon I actually use a raw file from the deleted picture folders. I do this because long ago, when I deleted a file, it was gone forever and occasionally I regretted my choices. That is no longer a problem. Since then I have done mini purges after a year is finished and done with. Mostly I keep up with it as I go. As I said, it works for me.
Well, it depends a lot on how you work with your photos and what you use them for. I worked with a well organised folder structure instead of a catalog software for quite a while, too. But at some point, I found it inefficient to keep multiple copies of the same file in different folders (for example a landscape shot from a trip went into the folder of the trip, and into the landscape folder), and different edits/crops used for different purposes.
Now with Lightroom it’s easy to put a photo in different collections and have virtual copies with different edits without having multiple physical copies of the same photo.
Also, a good catalog software allows you to quickly filter by keywords, metadata etc., which makes finding photos for specific purposes much easier and quicker.
Of course, if you have no use for any of these features, you don’t need a catalog software.
I fully agree. Lightroom is excellent at keeping images organized, and it reduces chances of images getting duplicated across different folders. Any software that can read metadata from images into a single database can be very powerful in keeping photos organized…
Elaine, if you have never used Lightroom or a similar database-driven tool for image organization, you are certainly missing out quite a bit. Lightroom might not be the best tool for image culling, but it is amazing at keeping your photos properly organized.
If you have not seen my guide on organizing images in Lightroom, take a look when you have a chance, and watch chapters 3 and 4 of our post-processing course.
There are clear advantages to using a proper image management tool for organization…
Nasim, My images are perfectly organized and managed, all from using my own common sense! :) I never lose control of them. There is more to my system that I put in my post. My system is easy and logical. Over the years I’ve read posts about Lightroom, and my impression is that generally people don’t love it, even when they continue to use it. Besides, it is not any particular software that I don’t like; in fact I never even thought of Lightroom, it is catalogs that I don’t like. My software has a catalog, I just elect not to use it.
I did this 2 years ago. I had almost 500K of photos from over 25 years. My overall approach was that if an image was not part of a collection, then it was deleted (with some exceptions). Now I’m down to around 250K of photos, still a lot but it saved me from buying yet another hard drive.
That’s a lot of images, even after you were done with the clean-up. My target is about 10% of my total volume, which I think is a realistic goal. I will certainly keep all 4-5 star photos, but what are the chances of me ever needing 1-3 star photos, or anything that I have not touched in years? I think the answer is pretty obvious.
I make a lot of event-photography and at 20 fp/sec you end up with a 1000 shots or more; every event.
My workflow is 1st; Give all the photos a proper name and each a different sequential number so if you know the number you know the photo.
Then I make a first selection of selection of good photos that I keep and of these I make a selection I will work on for my client. Now I keep a about 25%.
so i do the shifting directly days after the shoot.
I started way before LR existed in the 1980’s so I have analogue and digital images I want to find. I use a database for this since then. The LR catalogue is too restricted for me – I want to move around my photos freely in the finder. I have about 17 harddisks archive (double) – My latest nr17 is a 16TB. I produce 4TB each year I want to keep. In terms of money that is about 150,- $ each year. For a professional not too much. In terms of space my complete archive could be in a shoebox- so no big deal. In terms of reliability: even my oldest ans smallest archive nr 1 a 320GB harddisk of 2006 is still working flawlessly (I do test them)
I still can find all my photos in a blink.
My useful and important programs are :
culling : FastRawViewer
database : Filemaker pro ( exists since the 1980’s !) ( also for invoices, adresses, assignments and interconnected)
for searching in my archive : Neofinder ( makes a cataloque of all the archive disks)
Pieter, thanks for sharing! I know another photographer who has a similar process as you for storing hard drives, and I know it works well for him too.
Personally, I like to keep all of my photos in one place, so I use an external NAS for storing all photos. It might be too expensive or simply not practical for other photographers, but I like being able to access older photos any time. However, RAW images do add up, and a NAS can be tougher to manage once it starts filling up, so my methodology certainly has its disadvantages.
The problem with me is that I did not keep a consistent process of image culling over the years. Often times, I would come back really tired from photo shoots or events, then just import all the images from memory cards, hoping I would deal with other images later. While I always try to eliminate obviously bad images from the get-go (wrong exposure, out of focus images, etc) by either deleting them in my camera or doing a quick image cull before import, I still ended up with too many images.
I think image culling is not a one-time process – one should go back and do it several times, especially after some time passes.