Do you really need your photos in 4K?

08 Sep, 2023

Table of Contents

TL;DR
Why?
Optimising Videos
Optimising Images
Results

TL;DR

Install ffmpeg and imagemagick
Run this to output a list of the (soon to be old) video files

find . -type f -name "*.mp4" -exec echo "rm \""{}"\""  \; >> Files_to_be_deleted

Run this to compress your videos

find . -type f -name "*.mp4" -exec ffmpeg -i {} -vf scale=1280:-2 -c:v libx265 -crf 30 {}.newlycompressed.mp4 \;

Run all the rm commands from the Files_to_be_deleted file from step 2 (may need to do this manually if you have issues)
Run this to rename the compressed video files back to the original name (requires zsh, search for how to do this with mmv otherwise)

autoload zmv
zmv '(**/)(*).newlycompressed.mp4' '$1$2'

Why?

I was decluttering some of my old files, and noticed some funny old videos I took of friends, my girlfriend, etc. What stood out to me was these videos being in the realm of 300MB for a 2 minute video. Sounds like such a waste to me. Then I thought about a lot of the photos I've taken over years. 10MB plus for some of the larger and more complex ones!

What is the value here: Having a high quality photo of a place/time/person, or the memory that that photo brings back?

For the majority of people, and the majority of the media content they create, I think it's squarely the latter. So if I don't care about being able to see a rock in 10 pixels vs 5, I might as well run all my media through optimisation and downscaling.

Being a techie I can do this in a somewhat automated fashion, and for free. The part I can't do is fiddle around with getting the commands just right to do the job. Luckily, I'm not the only one who has tried to do this sort of thing, so there's usually an article or StackOverflow post lying around for it.

Optimising Videos

There is nothing as powerful as ffmpeg here. Luckily for us it's free and available on any platform.

Based on this SuperUser answer, in the top directory of what I want to optimise, the following should be able to do this recursively in any subdirectory:

find . -type f -name "*.mp4" -exec ffmpeg -i {} -vf scale=1280:-2 -c:v libx265 -crf 30 {}.newlycompressed.mp4 \;

With the caveat that since this creates a new file, we probably want to make a list of the files to remove first (run this before the one above!):

find . -type f -name "*.mp4" -exec echo "rm \""{}"\""  \; >> Files_to_be_deleted

Once the initial command is done, you can run all the rm commands from the Files_to_be_deleted file.

Then, you'll want to rename all the files back to their original name. I use zsh so I used zmv:

autoload zmv
zmv '(**/)(*).newlycompressed.mp4' '$1$2'

Figured this one out with help from here and here.

Optimising Images

For this one I wasn't sure what the right tool would be, so I just searched for recursively optimise images and found this SuperUser question asking how to do this using ImageMagick. The provided answer is for Windows, but I think I can adapt the find command used for the videos to run with ImageMagick. Of course I first need to know what the optimisation command is, so a quick search of ImageMagick Mogrify yielded this example page. With that and the SU question as my guide, I tested it on a copy of some of my photos. After the trial & error (and help from ChatGPT), here's the resulting command:

find . \( -name \*.jpg -o -name \*.JPG -o -name \*.jpeg -o -name \*.JPEG -o -name \*.png \) -size +1M -exec sh -c 'magick mogrify -resize 40% -quality 89 "$1" && echo "Processed $1"' sh {} \;

This command does the following:

Matches multiple extension matches to match common image file types I have - the various flavours of JPEG and PNG.
Only looks for files over a certain size - in my case, I think anything over 1MB can probably be cut down.
Resizes the image down to 40% of the original.
Sets the image compression quality to 89%. This number is chosen based on this documentation on how the -quality flag works for compression - namely that chroma channels are downsampled for values under 90. Compression around this range in my experience with image editing yields significant file size savings already, so I don't think it needs to go much further.
The compression command is run in a shell command so that I can output which file has been processed each time - this is done so that I have some indication of progress as by default magick mogrify doesn't output anything.

For a test image of around 4.5MB (a 3648 × 2736 image from a camping trip taken on an old 10MP Sony digital camera), magick mogrify took it down to 523 at a 1459 × 1094 resolution. Still good enough to have a nice amount of detail, and at 1/9th the size! For images around 1MB (usually pictures off of my phone's selfie camera), it came down to ~150KB and still a decent resolution (1764 × 3824 -> 706 × 1530).

One thing I avoided doing was converting PNGs to JPGs, as there are some edited images where I want to keep the transparency.

Results

My archive of photos & videos went down from 24.37GB to 3.34GB.

The ffmpeg command took several hours to complete. In the time it took, I figured out the image optimisation command (and ran it), wrote this article, and went to buy some groceries. There probably is some room for optimising or parallelising the commands further, but I didn't want to overwhelm my system while I did other things.

Image Magick wasn't happy with some of the files it got, but it worked well for the majority. Not sure if the issue was perhaps bit rot or something else.

#file storage #niksoftware