Download Page

 

Dataset

Source images + retargets (.zip) (580mb, last updated: 2011-05-12)
Source images only (.zip) (60mb, last updated: 2010-08-24)
[Acknowledgments]

All images and results are supplied in lossless format (although not all were originally stored in lossless format). Each image resides in a different directory with the image name, under which the source and retargeting results are stored. The source image is named <image-name>.png, and the retargeting results are named according to the convention <image-name>_<r>_<op>.png, where <r> is a %.2f string representation of the resizing ratio, and <op> is the operator code.

 

Analysis Data

The accumulative user responses and metric results are available as MATLAB structs:
subjData-ref (.mat) - user responses with source reference
subjData-blind (.mat) - user responses without source reference
objData (.mat) - metric results

This data corresponds to the 37 images we chose for the analysis in the paper. The user data is based on fully completed questionnaires only. Each structure is comprised of two fields: datasetNames(i) = <image-name>_<r>, and data - a 37x8 (8 = #operators in this study) matrix such that data(i,j) = number of times operator j was favored over another operator on image i. The operator numbers match the list here.
For the metric structure, data(i,j)= distance of operator j's result from the original (source) image i under that metric. Notice that objData contains several such structures, one for each metric.

We also provide the complete user data we collected (more users, more images), stored in a MySQL database. Please follow these steps for obtaining and accessing it:

  1. Download the database (.sql) to a temporary location. (1.2mb, last updated: 2010-08-24)
  2. Import the data to MySQL server (we found an open source MySQL frontend called SQLyog convenient for that). This should create a 'survey' database on the server with all the data. You can now access the database using your preferred method, or continue to the next steps for accessing it from MATLAB.
  3. Download the MATLAB-MySQL connector (.zip)1 (Source code available here if you're working in a non-Windows environment). Place mysql.m and mysql.mexwXX (that matches your system) in your working directory, and libmysql.dll under C:\WINDOWS\system32.
  4. See the demo script and analysis script for examples on accessing and querying the data.

Documentation of the database schema is available here (or download survey-schema.zip)

This setup was used with Windows 32/64bit, MySQL 5.1 and MATLAB 7.9.0.

1 MATLAB's database toolbox also supports MySQL access, however we found the above connector simpler to use.


User Study

Our web-based survey system source code is available for download: survey-2.3.zip [README]. (110mb, v2.3 2010-08-30)
By releasing this code we hope to ease conducting similar user studies in the future, and to encourage authors to perform extensive evaluation of their methods.
Follow the instructions in the README file for installing and running.

Instructions for running with Mechanical Turk

Amazon's Mechanical Turk is a very useful and convenient tool for conducting large scale user studies, and has been used extensively in computer vision research in the past few years. Once you have the survey system up and running, follow the steps below for routing mechanical turk workers to your survey.

  1. Set up a 'Requester' account for submitting work (instructions)
  2. Create a HIT for your survey (instructions). For example, this is the hit we used:

Image Retargeting Survey

1. Please follow the link below and complete the survey.
Note that the questionnaire, and our analysis, are designed to detect randomly answered questions. Such hits will not be rewarded.

http://<SERVER>/index.php?mode=1&exptype=1&user=1

** in case of any server failure, please try again later. We have allotted enough time for the hit.


2. Once you've completed the survey, you will receive a code. Please enter this code in the box below to get paid.


3. Use the following space to enter any additional comments/questions/thoughts regarding the survey.

 
Thank you for participating!

Where <SERVER> is a placeholder for the address of your server machine. See the survey system README file for more information on the other parameters.

  1. The code that each worker gets at the end of the survey is a unique session key that is also used for storing his/her session on the server. So when a worker submits his HIT, you can verify against the stored session that (a) the worker indeed completed the survey, and optionally (b) that the worker has done a decent job (e.g. did not pick answers at random), and pay them accordingly.

That's pretty much it!

We requested that the workers had HIT approval rate >= 95%, and constrained each worker not to work on the HIT more than once. We rewarded $0.25 per HIT, which took the workers ~20 minutes to complete on average. The workers were pretty satisfied with this setup. You can find some feedback we got (answers to question 3 of the HIT) here.