Abstract

Despite the recent trend of increasingly large datasets for object detection, there still exist many classes with few training examples. To overcome this lack of training data for certain classes, we propose a novel way of augmenting the training data for each class by borrowing and transforming examples from other classes. Our model learns which training instances from other classes to borrow and how to transform the borrowed examples so that they become more similar to instances from the target class. Our experimental results demonstrate that our new object detector, with borrowed and transformed examples, improves upon the current state-of-the-art detector on the challenging SUN09 object detection dataset.

Publications

• 2011 NIPS paper [pdf]
  • @inproceedings{lst_nips11,
    author = {Joseph J. Lim and Ruslan Salakhutdinov and Antonio Torralba},
    Booktitle = {Neural Information Processing Systems (NIPS)},
    title = {Transfer Learning by Borrowing Examples for Multiclass Object Detection},
    year = {2011}}


• Poster [pdf]

Dataset: Transfer Learning Challenge for Object Detection

There has been an increasing attention to learning with borrowing/sharing for the fewer examples class. Here, we introduce a new challenge on transfer learning for the detection. This dataset is based on the SUN 09, and it contains 4082 training and 9518 testing images. The categories include person, chair, car, truck, door, statue, and bag.

Downloads

• Download all images (.jpg) and annotations (.xml): transfer.tar (4.4GB)
  • Each xml contains each object's class ("name") and its bounding box ("bndbox") information.


• Splits
  • Train set images
  • Test set images
  • 100 Classes for train/test

This shows number of training examples per classes. The five classes with the most number of examples are plant (1356), person (600), cabinet (551), chair (501), and door (394). The five classes with the least number of examples are cake (17), trash can (13), keyboard (12), cars (10), and cup (8).

Learning which examples to borrow

Our goal is to develop a novel framework that enables borrowing examples from related classes for a generic object detector. Our model learns which examples are most useful to borrow for a model training together with original examples.

	Weight indicating how much examplar i to be borrowed for class c
	Loss function (from a standard binary classification)
	Group Lasso regularization; these two terms together will regularize weights so that examples from the same class to be similar. In other words, examples from the same class will be gently forced to be borrowed (or not borrowed) together.
More details can be found in the paper

(1) Borrowing examples from other classes within the same dataset, and
(2) Borrowing examples from the same class that come from a different dataset.

Borrowing examples from OTHER CLASSES within the SAME DATASET

(a) Shelves for Bookcase

(b) Chair for Swivelchair

Examples ranked by their learned weights (a) shelves examples to be borrowed for bookcase, and (b) chair examples to be borrowed for swivel chair. Green border indicates examples are borrowed, and red border indicates examples are not borrowed.

Borrowing examples from the SAME CLASS that come from a DIFFERENT DATASET

(a) From SUN09

(b) From PASCAL

(c) PASCAL cars ranked by their learned weights to be SUN09

(a) shows car examples from SUN09, and (b) shows car examples from PASCAL dataset. Finally, (c) shows PASCAL car examples sorted by learned borrowing weights for SUN09. Green border indicates examples are borrowed, and red border indicates examples are not borrowed.

Acknowledgments

This work is funded by ONR MURI N000141010933, CAREER Award No. 07471 20, NSERC, and NSF Graduate Research Fellowship.

Last update: Oct 15. 2011