LIBPMK: A Pyramid Match Toolkit

About | Users | Downloads | Extensions | Documentation | FAQ | Links | Contact

Latest version: 2.5 (ChangeLog)

About LIBPMK

LIBPMK is a C++ implementation of Grauman and Darrell's Pyramid Match algorithm. This toolkit provides a flexible framework with which you can quickly match sets of image features and run experiments. LIBPMK has a wide range of features, including:

Built-in k-means and hierarchical clustering
Dealing with data sets too large to fit in memory
Building histograms and multi-resolution histograms
Quickly performing pyramid matches
Training SVMs and running experiments with them*

This toolkit is currently being actively maintained; please check back often for updates.

LIBPMK implements the algorithms described in the following papers (Show BibTeX entries):

@inproceedings{grauman-darrell-iccv2005,
  author    = {K. Grauman and T. Darrell},
  title     = {The Pyramid Match Kernel: Discriminative Classification with Sets of Image Features},
  booktitle = {IEEE International Conference on Computer Vision},
  year      = {2005},
  address   = {Beijing, China},
  month     = {October},
}

@inproceedings{grauman-darrell-nips2007,
  author    = {K. Grauman and T. Darrell},
  title     = {Approximate Correspondences in High Dimensions},
  booktitle = {Advances in Neural Information Processing Systems 19},
  editor    = {B. Scholkopf and J.C. Platt and T. Hofmann},
  publisher = {MIT Press},
  address   = {Cambridge, MA},
  year      = {2007},
}

K. Grauman and T. Darrell. The Pyramid Match Kernel: Discriminative Classification with Sets of Image Features. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Beijing, China, October 2005.
K. Grauman and T. Darrell. Approximate Correspondences in High Dimensions. In Proceedings of Advances in Neural Information Processing Systems 19 (NIPS), 2007.

* This module of the software package includes a version of LIBSVM which handles the running of SVM experiments. For details, please see:

Chih-Chung Chang and Chih-Jen Lin, LIBSVM: a library for support vector machines, 2001. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm.

When using this code, please cite LIBPMK as follows:

@techreport{libpmk,
  author      = {John J. Lee},
  institution = {MIT Computer Science and Artificial Intelligence Laboratory},
  month       = {April},
  number      = {MIT-CSAIL-TR-2008-17},
  title       = {LIBPMK: A Pyramid Match Toolkit},
  year        = {2008},
  url         = {http://hdl.handle.net/1721.1/41070}
}

Projects that use LIBPMK

Jianzhao Qin and Nelson H.C. Yung
Scene Categorization by Introducing Contextual Information to the Visual Words, ISVC 2009.
Kate Saenko and Trevor Darrell
Filtering Abstract Senses From Image Search Results, NIPS 2009.
Hasan Celik, Alan Hanjalic, and Emile A. Hendriks
Unsupervised and simultaneous training of multiple object detectors from unlabeled surveillance video, CIVU 2009.
Allen Y. Yang, Subhransu Maji, C. Mario Christoudias, Trevor Darrell, Jitendra Malik, and S. Shankar Sastry
Multiple-View Object Recognition in Band-Limited Distributed Camera Networks, ICDSC 2009.
Ashwin Thangali and Stan Sclaroff
An Alignment Based Similarity Measure for Hand Detection in Cluttered Sign Language Video, CVPR Workshop 2009.
Xiaofeng Ren and Matthai Philipose
Egocentric Recognition of Handled Objects: Benchmark and Analysis, CVPR Egocentric Vision Workshop 2009.
Tom Yeh, John J. Lee, and Trevor Darrell
Fast Concurrent Object Localization and Recognition, CVPR 2009.
Haiqiang Zuo, Weiming Hu, Ou Wu, Yunfei Chen, and Guan Luo
Detecting Image Spam Using Local Invariant Features and Pyramid Match Kernel, WWW2009.
Siyao Fu, Zengguang Hou, Zize Liang, Qi Zuo, and Min Tan
Multiple Kernel Learning from Sets of Partially Matching Image Features, UKACC Control 2008.
Tingxin Yan, Deepak Ganesan, and R. Manmatha
Distributed Image Search in Camera Sensor Networks, ACM SenSys 2008.
Shuiwang Ji, Liang Sun, Rong Jin, Sudhir Kumar, and Jieping Ye
Automated Annotation of Drosophila Gene Expression Patterns Using a Controlled Vocabulary, Bioinformatics 2008.
Tom Yeh, John J. Lee, and Trevor Darrell
Photo-based Question Answering, ACM Multimedia 2008.
C. Mario Christoudias, Raquel Urtasun, and Trevor Darrell
Unsupervised Feature Selection via Distributed Coding for Multi-view Object Recognition, CVPR 2008.
Ashish Kapoor, Pradeep Shenoy, Desney Tan
Combining Brain Computer Interfaces with Vision for Object Categorization, CVPR 2008.
Tom Yeh, John J. Lee, and Trevor Darrell
Scalable Classifiers for Internet Vision Tasks, CVPR Internet Vision Workshop 2008.
Kate Saenko, Trevor Darrell
Object Category Recognition Using Probabilistic Fusion of Speech and Image Classifiers, MLMI 2007.
Michael Oltmans
Envisioning Sketch Recognition: A Local Feature Based Approach to Recognizing Informal Sketches, Ph.D. thesis (2007).
Tom Yeh, John J. Lee, and Trevor Darrell
Adaptive Vocabulary Forests for Dynamic Indexing and Categry Learning, ICCV 2007.

This list was made by searching Google for "libpmk"-- if you would like to link to your project from here, please contact me!

Downloads

Source code

Download the latest version (2.5, released 2010-05-05): libpmk-2.5.tar.gz

Older versions are available at http://people.csail.mit.edu/jjl/libpmk/download.

Recent changes (full ChangeLog here):

We now include all of the LIBPMK extensions in this single package. This is because version numbers can get inconsistent, so it's better to keep them all in the same package. They still all compile independently.
Added libpmk_eth examples to the package.
Added the adaptive vocabulary tree extension (directory "avt")
Added a SURF binary to the feature extractor extension.
Added a demo UI to the ISM extension.
Fixed a bug in global-vg-pyramid-maker that caused inaccurate results due to not updating child nodes in the tree properly (thanks Popke Altenburg, Fabian Richter)
Fixed a bug in the ISM train-model example where it only used 1 image as training data instead of all of them (thanks Guo Xi)
Fixed 64-bit compilation problems (thanks Gal Frishman)
Fixed compilation on Cygwin (thanks Arvid Terzibaschian, Tom Yeh)

Example code and data

ETH-80 database

Extensions

LIBPMK is extensible and can be used for a variety of tasks. Below are some extension libraries I have written for LIBPMK. They include documentation and the full C++ source code. Each of these packages has a library component and a "tools" component, which shows examples of using the library. If you are interested in just using the code rather than writing your own applications using this library, the tools may be all you need and you will not have to write any code. Please see the included README files for details. Note: this section is obsolete and will go away soon. All of the extensions are now included with the main LIBPMK package as of version 2.5.

Feature extraction: libpmk_features.tar.gz (requires libpmk-2.2 or above)
Spatial pyramid match of Lazebnik et al.: libpmk_spatial.tar.gz (requires libpmk-2.2 or above)
Implicit shape model of Leibe et al.: libpmk_ism.tar.gz (requires libpmk-2.1 or above)

Documentation

Tutorial: How to install, use, and write new code using libpmk. Includes a FAQ that addresses speed and performance issues.
Version 2.x class reference: Details, code specifications, and class hierarchy diagrams.
Version 1.x class reference: For those using libpmk-1.x (obsolete)

Links

Pyramid Match project page: http://www.cs.utexas.edu/~grauman/research/projects/pmk/pmk_projectpage.htm
LIBSVM home page: http://www.csie.ntu.edu.tw/~cjlin/libsvm/

Contact

If you have any questions, suggestions, or bug reports about this implementation, please contact John Lee ()

To recieve updates about libpmk, please join the extremely low-volume (around once every few months) mailing list.