Sorting images using hue comparison between original and smooth version on OpenCV: is that enough?

Question

The question is about sorting pictures(camera) vs. other types of images. In another question, I tried to leverage a scripted solution using ImageMagick which was provided. I'm now exploring another answer to the same question, using OpenCV.

---

As described in the solution, the idea is to change color mode to HSV, smooth an image and compare its hue channel to the original; then make an average of the difference as supposedly comic-like images differ from real life images in that respect(in that they react differently to the operation). So I tried:

#include "imgproc/imgproc.hpp"
#include "highgui/highgui.hpp"
#include "cv.h"  
#include <iostream>   
#include <stdio.h>  

using namespace std;
using namespace cv;

int main(int argc, char *argv[]) {

//takes filename as argument i.e. ./smooth one.jpg  
    cv::Mat input, output, hsv;
    input = imread(argv[1],1);

//Resize to 300
    Size size(300,300);
    resize(input,input,size);

//Change original image color mode to HSV
    cvtColor(input,hsv,CV_RGB2HSV);

//Separate channels; keep hue on h_image
    std::vector<cv::Mat> hsv_channels, hsv_channelsOUT;
    cv::split(hsv, hsv_channels);
    cv::Mat h_image = hsv_channels[0];

//Apply smoothing/blur on h_image, to output        
    bilateralFilter(h_image,output,60,120,30);

//Split output(already just H) to extract H array for mean later
    cv::split(output, hsv_channelsOUT);
    cv::Mat O_imageH = hsv_channelsOUT[0];

//Average all differences between H values pre/post blur
//Compare h_image with h_image+bilinearF visually to see
    Scalar h = mean(hsv_channels[0]-hsv_channelsOUT[0]);

//Print that to console 
    printf("H: %f\n", h[0]);

//Showcase all the steps; comment to use to binary in scripts
//All windows piled up on top of another one at runtime; move to see
cv::imshow("input",input);
cv::imshow("hsv",hsv);
cv::imshow("h_image",h_image);
cv::imshow("output",output);
cv::imshow("h_image+bilinerarF",O_imageH);

//exit on keypress on image displayed
    waitKey();   
    return 0;   
  }

Which yields something like this on this image and this portrait (two "real" images, chosen to see if there is similarity in the values i.e. that they're of the same type):

We are comparing the 3rd and 4th image in the sequence, which is one channel against the same one channel image smoothed. Results for the average of the difference between elements in the H channel(0):

JFK        H: 7.313056
Landscape: H: 8.265544

I don't really know how to set the bilateralFilter as I don't know what sigma space is etc. I had initially set it to maximum values generated in the OpenCV documentation example(see intro). I doubled the values as this widens the difference in H values between some types of images(somewhat isolating their range, I would think). All black line art on white will yield 0 for their H value - just is a side effect of what's done, their hue channel is black either way. So those could easily be categorized as such. I do not have a sufficient set of images to make predictions and adjust the bilateralFilter accordingly. I seem to have camera images at H: 4-5; 7-9; and the smallest value I get(0.08) is from a blue to green pattern of identical non opaque triangles(a graphic); all these, are observations on too small a sample to infer anything meaningful.

---

Q. So I would like to know if I imlemented the idea properly(according to the solution I based this on)? Should I come to the conclusion that it's going to take a much more resilient program(using many features) to do what I want to do? Should I set the bilateralFilter differently?

Answer 1

This works somewhat. It's not clear how to tune this. But I had failed in properly assembling elements to make the index with ImageMagick. I also had some issues with the script used to glue this together but was provided with some help. So I was able to make it work(properly):

#!/usr/bin/env sh

##+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
## -=Try to sort images based on content (helper script)=-
## re: http://stackoverflow.com/q/26951796
## We will generate values for each jpg with our OpenCV thing, sort, then index
## accordingly with IM.
## Our challenge is that we cannot supply 307 jpgs at a time for final montage
## So we'll split that in rows of 15 images.
##+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

##OpenCV## find all files in current dir/subdirs, run our OpenCV binary
# which echoes the name of the file: H: value #################################

find . -type f -name '*.[Jj][[Pp][Gg]' -exec sh -c '
    for file do
    echo -n "$file: "
    ./smooth "$file"   #name of our OpenCV binary in local dir only
done
' sh {} + | sort --version-sort -k3 | tee Hout | cut -d ':' -f 1,1 > Hpaths_sorted

##ImageMagick - "montage" can't make an index with 100s of images at 100x100
# on my rig; so horizontal strips of 15 images; we will have $z many strips
# i.e. ceiling for not fully populated row ############################

a="1"
h="15"
j="$(cat Hpaths_sorted | wc -l)"
z="$(echo "a=$j; b=$h; if ( a%b ) a/b+1 else a/b" | bc)"

for i in $(seq 1 "$z"); do 

    strip=$(awk "NR>=$a&&NR<=$h" Hpaths_sorted)
    if [ "$h" -gt "$j" ]; then
        montage $(echo "$strip"| tr '\n' ' ') -tile x1 -geometry 100x100+0+0 $i.gif
    else
        montage $(echo "$strip"| tr '\n' ' ') -tile 15x -geometry 100x100+0+0 $i.gif
        a="$(echo $(($a+15)))"
        h="$(echo $(($h+15)))"
    fi

    target="$target $i.gif"

done

# Use our $target to supply the list of files(strips), make one index, 1 row x #strips to final.gif

montage $target -tile 1x$z -geometry '1x1+0+0<' final.gif

Which produces on our 358 image set:

Low values are on top and I can see some groupings(and we've properly generated our incomplete row). So it may not be so useful in its simplest form but it certainly can be used as is worth exploring...