JPEGmini Server Review

After I had a chance to test and review the JPEGmini Pro software, I realized how powerful this software is not just for exporting images and being part of a Lightroom workflow, but also for many other uses, including optimizing images that are already sitting on our large storage devices. Another use I immediately thought of, was the web server where Photography Life traffic originates from. Given how much traffic PL serves worldwide on a day-to-day basis and the fact that images alone account for roughly 5 Terabytes of traffic per month, the thought of being able to compress JPEG images using the JPEGmini engine was something that I really wanted to implement sooner than later. So I embarked on a new project – to save both traffic and money in the long run for PL, using the JPEGmini server.

Photographers Beware: this is a very technical review of software that is not related to photography. I decided to publish the review at PL, since I feel that other photography-heavy websites could hugely benefit from implementing the JPEGmini server.

1) Server Environment Overview

Before I go into the review, I would like to point out a few potentially important bits of information about my web server setup. First of all, I run CentOS Linux on every server (and there are a few of them). The two back-end web servers that handle PHP calls from the load balancer is where I installed JPEGmini server, although only the first one really matters, as it is the one that handles all uploads to the site (WordPress cannot handle this directly, so it is only possible to watch for wp-admin calls and direct them to the appropriate server via nginx/apache). Unfortunately, there is no easy way to run more than one WordPress server without file upload hassles, as it is not designed to be used in a cluster environment (moving everything to AWS with EC2 running server instances, RDS running the DB and S3 handling the files would be a good solution, but after I tested it out, it was not a cheap solution by any means, especially once you start spawning a few EC2 servers that would handle the back-end load). Therefore, I have been syncing all uploads via rsync. Not an elegant solution, but it works fairly well. I have rsync monitoring the “wp-content” folder, so all changes are replicated one way (basically, once images are uploaded to server01, they automatically get picked up by server02). Takes a second or two to sync up, but once it happens, the images get served easily to load balancer requests.

All web server calls get handled by a load balancer, which only serves https web traffic. All images are handled by an external CDN. The main reason for implementing JPEGmini was to reduce CDN costs, which are only going up each month as we continue to publish more content.

Keep in mind that your web server must be running a flavor of Linux – JPEGmini server does not run on Windows servers. Here is the list of supported server platforms.

2) JPEGmini Server Installation

The installation of JPEGmini server is very easy, especially if you run RHEL, CentOS and other popular Linux distributions. For my CentOS server, JPEGmini provided an RPM file, so it was an easy install with a single command. Once the binary file was installed (/usr/bin/jpegmini by default), the next step was to copy the .jpegmini.cfg license file in the home directory of the user. From there, running “jpegmini” should output something like the following:

===============================
Start jpegmini 3.14.2.84235
===============================
-f option is required: -f=<jpeg file name>
Use -help for help

===============================
Finish jpegmini 3.14.2.84235
===============================

My initial testing started out with the JPEGmini server version 3.13, but after a few requested changes to the executable, JPEGmini provided an updated 3.14 RPM file. The major addition to the 3.14 version is the ability to skip already optimized files, which was a big deal for me, as I do use the desktop version of the software and I did not want JPEGmini server to re-optimize uploaded JPEG images.

3) WordPress Image File Handling

When an image is uploaded to WordPress, the admin scripts will either use GD or ImageMagick to process those images. By default, WordPress creates images of three sizes, in addition to the uploaded image (thumbnail, medium size and large size), but depending on how many add_image_size calls there might be added by the theme and plugins, there might be many more! Because of this, a single image upload could spawn a bunch of files on the server, letting the Uploads folder grow very quickly. And those smaller images are created by either GD or ImageMagick, so the files by default are going to be stripped of both ICC color profiles and EXIF data, which is not desirable on a photography website. They are also not going to be properly optimized for size, since neither GD nor ImageMagick have a smart algorithm like JPEGmini in order to be able to properly compress JPEG images. In fact, WordPress does a pretty horrible job with resizing images, often resulting in poorly colored (due to stripping of ICC profiles), soft and muddy images (due to heavy compression). To avoid this problem at PL, I have been only using ImageMagick to optimize images, with special options. We only strip EXIF data from thumbnails and compress them a bit more aggressively for a fast browsing experience. Once in a post, neither ICC profiles, nor EXIF data are stripped from larger images in order to make them look as good as possible. This way, we do not force our readers to click on an image to see the “right version” – images look consistent from previews to native uploaded sizes.

Therefore, in order to take a full advantage of the JPEGmini server, it is best to run the executable for each resize process – not just for the single uploaded version, as you want every file to get optimized by the engine, whether it is a thumbnail, a medium, or a large version of the original. This essentially means that JPEGmini should be intercepting every call to image_resize.

4) JPEGmini Server and WordPress Integration

Unfortunately, JPEGmini does not provide a plugin that automatically integrates into WordPress in order to do that, so I had to come up with a solution on my own. I started with the ImageMagick Engine plugin codebase (a pretty outdated plugin, but it still works), then added calls to JPEGmini executable in the ime_im_cli_resize function (I run a command line version of ImageMagick instead of a PHP module). If this modified version of the plugin is something that interests you, let me know in the comments section below and I will send you the plugin file. I am not sure if folks at JPEGmini are planning to release a WordPress plugin, but I would be happy to contribute some code for a good cause.

The code works and it has been tested with JPEGmini 3.14. As soon as each resized version is created, the code first optimizes those images, then it optimizes and overwrites the original JPEG image.

5) JPEGmini Server Test Results

There has been a lot of technical mumbo jumbo so far, so let’s get down to the meat. How much drive space was I able to salvage and how much did I save in CDN costs? In order to run the JPEGmini executable recursively on every folder, I had to request a script from JPEGmini engineers, which they provided very quickly. The provided file was a Python script called “jpegmini_recursive.py”, which only needed two commands – one to input the source folder and one to input the target folder (I modified the script a little after getting the new RPM version that can automatically skip already optimized JPEG images). After backing everything up, I created a folder called “uploads_jpegmini” and that’s what I used as the target folder. I ran the script and it took a while to go through each and every file. I came back after a few hours and the script finished executing.

Since JPEGmini only optimizes JPEG images and it does not touch PNG, GIF or other file uploads such as video, I had to make sure to copy the resulting folder back into my uploads folder. Again, make sure you fully backup everything before you take this step, since it is irreversible. Before I did that, I recursively changed permissions on the uploads_jpegmini folder by running “chown -R nobody:nobody /uploads_jpegmini”. Then the next command was “/bin/cp -Rpf uploads_jpegmini/* uploads/”, which overwrote existing image files with their JPEGmini optimized versions.

Let’s take a look at the before and after. Here is what my folders looked like before I copied all the contents over:

du --max-depth=1 | sort -k2
1252    ./2006
5272    ./2007
23332   ./2008
154872  ./2009
819580  ./2010
599084  ./2011
2124952 ./2012
2176548 ./2013
4504720 ./2014
6164472 ./2015
3812759 ./2016
559012  ./2017
Total Size: 20,945,855

Roughly 21 gigabytes of images. Now let’s take a look at what the folder looked like after all the images were optimized by JPEGmini:

du --max-depth=1 | sort -k2
1000    ./2006
2852    ./2007
15972   ./2008
127708  ./2009
647896  ./2010
461800  ./2011
1099676 ./2012
1252836 ./2013
3049696 ./2014
4378464 ./2015
2858628 ./2016
479416  ./2017
Total Size: 14,375,944

Whoa, that’s only 14.4 gigabytes now! Just in hard drive space alone I was able to reclaim over 6.5 gigs of space, which translates to roughly 31% in space savings. That’s basically one third of my CDN bill, which is a big number. And keep in mind that the last two+ years did not get as much space savings as earlier, since I already started optimizing my images on my desktop with JPEGmini Pro before uploading, so the numbers you see are uploads by other team members who are not using JPEGmini.

Here is a sample summary report from JPEGmini for June of 2012:

----------------------------------
INFO: Summary report for folder photographylife.com/wp-content/uploads/2012/06 [including subfolders]:
INFO: Total number of files: 372
INFO: Total size of input files: 42900 KB
INFO: Total size of output files: 28480 KB
INFO: Recompression ratio: 1.51X (34% saving)
INFO: ----------------------------------

Different folders yielded different numbers, but on average it was between 30-35%, which is a lot, considering that our team is pretty knowledgeable about keeping file sizes small during the export process (we usually keep our export settings at Level 10 in Photoshop, which is equivalent to Lightroom’s 77-84% “Quality”, per our JPEG Compression Levels in Photoshop and Lightroom article).

5) JPEGmini Server Quality and Metadata Settings

For sites that don’t necessarily care about preserving high quality JPEG images with their metadata, JPEGmini can actually optimize images much more aggressively. I did not want JPEG images to look any worse than originally uploaded, so I kept the default setting of “qual=0”, which preserves best quality. Other sites might choose to run with high or medium quality, which will reduce the footprint of JPEG files much more aggressively. Also, one can completely remove all the metadata as well with the “rmt=1” command and if that’s not enough, there is even an option to force progressive JPEG output on every image. I am sure social media sites like Facebook heavily utilize such tools, since images and videos are a huge part of their hosting bills. For a list of commands available with the JPEGmini server, please visit this page.

6) Conclusion

While the JPEGmini Server product is definitely not aimed at photographers, the software is a very versatile tool for those who own large websites with a lot of images and traffic. As can be seen from my implementation project, JPEGmini Server was able to save over 6.5 gigabytes of space, translating to roughly 31% in space and CDN cost savings, which is a lot for a business of any size. At $199 per month starting price, JPEGmini Server isn’t cheap for a small business, but for a growing company with a large hosting footprint where a single server instance might be costing more than that each month, the product might be worth a serious look. If you are a part of a hosting company, if you own a website loaded with a lot of images like PL, or your CDN costs are getting outrageous, you might want to reach out to folks at JPEGmini and talk to them about how they can help you. For a start, you could try this page out, where you can input your website and see how much you can expect to save in CDN costs.

If you have any questions about any of the above, please feel free to drop me a comment below.