ImageMagick image averaging
The video for the Neurogami track Kempten used a collection of Web-cam images that show the town square (AKA Rathausplatz) of the city.
Having collected a year’s worth of such images the plan was to turn them into a proper video.
Ffmpeg is a fantastic free program that can take a series of image files and give you a video.
Very slick. The Kempten Web-cam images made for a nice video, with people and cars sliding though as the sun rose, set, and rose again, over and over. However, the direct approach with all this activity was a bit distracting. What I wanted was to show the change of the town square over time while not emphasizing all the transient details.
Average is good
The solution was to use image averaging to smooth out frame-specific details. This was done using another free tool, ImageMagick.
ImageMagick runs on OSX, Linux, and Windows. If there’s some kind of transformation, alteration, or manipulation you want to do with one or more images then chances are ImageMagick can do it.
Averaging images is quite simple. You basically provide the list images to use, plus the name of the file to create with the average:
convert kempten001.png kempten002.png kempten003.png -evaluate-sequence mean averaged.png
Note: This is being done at the command prompt. The general application is called ImageMagick but you execute specific operations by calling a program named something else. ImageMagick provides a set of programs for different kinds of operations. See Command-Line Tools.
An example
Let’s start with these images:
We can get an average image using this:
convert kempten_2004-11-30-0* -evaluate-sequence mean averaged.jpg
… which gives use this:
You can see that the time-stamp at the top has a mixture of different times, while the date is clear, since it is the same in all pictures.
More interesting is how the people in the images are now almost ghosts. Same with the vehicles that appear in but one image.
That example command works OK if you have a few images; the use of the asterisk (AKA the “wildcard” character) means that all files that begin with “kempten_2004-11-30-0” get grabbed for the averaging. This can be a problem if your command finds hundreds of images to average.
Besides, for the Kempten video I didn’t want one final image but a set of processed frames that could then be combined.
This meant selecting subsets of available pictures to create a series of averaged frames. That’s a little trickier to do at the command line, so I wrote a Ruby script to manage it.
Code!
I’ve been playing around with image averaging fora few projects and have not quite settled on the best way to wrap the code. In some cases there’s some additional processing applied to the resulting average image. In one case I wanted the images to “wrap” so that the end of the list of images to process was modified to include repeats of files that occur at the start. That doesn’t work for all cases and I have yet sort out how I want to pass around assorted options.
So, really, this is a “one way to do it” example to illustrate a few things. Use it if it works for you, or hack it up to make work better for your needs.
First, code, then some comments on it:
You need to pass in a string to be used as part of file “glob”, a pattern that will select only the images you want to work with. You also need to give the file extension for these images; this same extension will be used for the resulting averaged images. In practice you can tell ImageMagick to produce an image of a different file type. For example, you may want to average a set of GIF images but create PNG files, and you can do this just by changing the file extension used for the output file. But that’s yet one more thing that would have to be ether passed as a config value or maybe loaded from some config file, and for simplicity omitted.
Another option you do not have is just how ImageMagick does the averaging.
The -evaluate-sequence parameter in the call to convert can take a number of different values. mean is generally a good choice. You can also use medium. You can get the list of possible values by running convert -list evaluate.
An option you do have, and must provide, is the grouping number. How many images should be used for each resulting average image? The larger the number the more you’ll lose transient details.
The names of the resulting averaged images are automatically numbered, with the initial glob name used. By default each file name is prefixed with “avg_”. I found this makes it easier to select only these files for later manipulation.
All of this code could be placed into a single method that takes all the required parameters. There’s a run method because I yanked this out of a larger file where splitting out initialization and execution seemed to make sense. In that case the run method took some arguments as well. (And even that code was pilfered from an earlier, cruder, version; it’s evolving. It’s code.)
Calling run kicks off building a list of images using Dir.glob. The first time I tried this I ended up with something that felt wrong, but I could not immediately put my finger on it. I had a larger script that grabbed all the Kempten images for a month, did some cropping, then created a set of average files by taking five images at a time. Those images were then combined into an avi video. It seemed to work great, but off. It was supposed to show a series of days and nights over the course of a month, and yet it appeared to be one very long overcast day. Little wonder: Dir.glob does not return a nicely sorted list of files. To fix that the results are sorted before assigned to images.
A check is made that at least something was was found. Depending on how your files are named, and where they are kept, and what else is in the same folder, a simple size check may be sufficient. Otherwise you may want to do something to ensure that files are found, and only files you want to use.
For each set of files to average an output file name is constructed. Here they are just given a sequence number. You may want to change this to include the grouping number. If you’re still experimenting with values it can be handy if the results somehow encode the parameters so you can reproduce them later.
Iterating over the list of images, a subset is extracted. If the subset does not have enough images the method simply returns. Note that because you are dealing with groups of images you will have fewer averaged results than total source images.
I apologize if that sounds painfully obvious, but it’s the sort of detail one might overlook. For example, in creating a video of Kempten images for a single day, with an image for each 20 minutes on the clock, if the images are averaged in groups of three then the last 40 minutes of the day will not fully make it; the last shot will average 23:20, 23:40, and midnight. (Assuming the first shot is at 00:20.)
Generating the “Kempten” video required working in small chunks because of the number of image files and the amount of assorted image processing (resizing, smoothing, color boosting, averaging, video creation, etc.). To avoid gaps where possible the code had to “overreach” when creating that array of images to average, pulling images from what would otherwise be separate sections.
In general, if you have N source images to be averaged in groups of size G then you will get N-(G-1) output images.
With a list of images to average the script creates a command-line string to invoke. The command is displayed, and then executed. The displaying of the command is handy when you are not getting the results expected. You can then run that same cmmand by hand and tweak the values to see what needs changing, then update the code.
Final thoughts
Season to taste. I’m still hacking around with assorted image manipulation and conversion to video and game assets, and yet the find the One True Way to manage options and multiple operations, but playing around is part of the fun.