java - Creating fisher vectors using OpenIMAJ -

i'm trying classify images using fisher vectors described in: sánchez, j., perronnin, f., mensink, t., & verbeek, j. (2013). image classification fisher vector: theory , practice. international journal of computer vision, 105(3), 222–245. http://doi.org/10.1007/s11263-013-0636-x

to try out , evaluate method want use openimaj library, since according javadoc use method create fisher vectors, can't work. i've tried creating sift feature vectors openimaj , opencv both same error: em algorithm never able compute valid likelihood given initial parameters. try different init parameters (or increasing n_init) or check degenerate data.

if has experience using method appreciate help. i've created little example should illustrate problem:

    // load image     localfeaturelist<keypoint> findfeatures = new dogsiftengine()             .findfeatures(imageutilities                     .readmbf(                             new url(                                     "http://upload.wikimedia.org/wikipedia/en/2/24/lenna.png"))                     .flatten());      // convert double array     double[][] data = new double[findfeatures.size()][findfeatures.get(0)             .getdimensions()];     (int = 0; < findfeatures.size(); i++) {         data[i] = findfeatures.get(i).getfeaturevector().asdoublevector();     }      gaussianmixturemodelem gaussianmixturemodelem = new gaussianmixturemodelem(             64, covariancetype.diagonal);      // error thrown here     mixtureofgaussians estimate = gaussianmixturemodelem.estimate(data); 

i guess problem you're not using enough data. train gmm should using many samples across training corpus, rather single image. note also, should applying pca reduce dimensionality of features before learning gmm (this isn't strictly required, helps performance shown in paper linked).

once done, can use openimaj fishervector class compute vector sift points each image.

as aside - you're doing classification, want using densesift variant on dog-sift if want kind of decent performance.

below sample code building fishervectors first 100 images of ukbench dataset:

    //load features disk     final list<memorylocalfeaturelist<floatkeypoint>> data = new arraylist<memorylocalfeaturelist<floatkeypoint>>();     final list<floatkeypoint> allkeys = new arraylist<floatkeypoint>();      (int = 0; < 100; i++) {         final memorylocalfeaturelist<floatkeypoint> tmp = floatkeypoint.convert(memorylocalfeaturelist.read(                 new file(string.format("/users/jsh2/data/ukbench/sift/ukbench%05d.jpg", i)), keypoint.class));         data.add(tmp);         allkeys.addall(tmp);     }      //randomise order     collections.shuffle(allkeys);      //sample 1000 of them learn pca basis 64 dims     final double[][] sample128 = new double[1000][];     (int = 0; < sample128.length; i++) {         sample128[i] = arrayutils.converttodouble(allkeys.get(i).vector);     }      system.out.println("performing pca " + sample128.length);     final thinsvdprincipalcomponentanalysis pca = new thinsvdprincipalcomponentanalysis(64);     pca.learnbasis(sample128);      //project 1000 training features basis (for computing gmm)     final double[][] sample64 = pca.project(new matrix(sample128)).getarray();      //project features basis, reducing dimensionality     system.out.println("projecting features");     (final memorylocalfeaturelist<floatkeypoint> kpl : data) {         (final floatkeypoint kp : kpl) {             kp.vector = arrayutils.converttofloat(pca.project(arrayutils.converttodouble(kp.vector)));         }     }      //learn gmm 128 components     system.out.println("learning gmm " + sample64.length);     final gaussianmixturemodelem gmmem = new gaussianmixturemodelem(128, covariancetype.diagonal);     final mixtureofgaussians gmm = gmmem.estimate(sample64);      //build fisher vector representations     final fishervector<float[]> fisher = new fishervector<float[]>(gmm, true, true);      int = 0;     final double[][] fvs = new double[100][];     (final memorylocalfeaturelist<floatkeypoint> kpl : data) {         fvs[i++] = fisher.aggregate(kpl).asdoublevector();     }