*************** notes for Poser pose estimation code and data repository *************** http://ttic.uchicago.edu/~gregory/code/Poser Data and software are provided for research purposes only. Any other use of the data (in particular any commercial use) is not allowed. The data and the accompanying software are provided as is, with no warranty, implied or express - use at your own risk. Please do let us know if you find any problem with the data or the code, by sending e-mail to gregory@csail.mit.edu. http://ttic.uchicago.edu/~gregory/code/Poser in any publication that includes results on data, or using software, obtained from this repository. In addition, please reference the paper @InProceedings{Shakhnarovich-et-al-03, Author = "G. Shakhnarovich and P. Viola and T. Darrell", Title = "Fast Pose Estimation with Parameter Sensitive Hashing", BookTitle = "Proceedings of ICCV", Address = "Nice, France", month = oct, year = 2003, } which introduced the use of Poser for purposes of learning pose estimation methods. The software is distributed under GNU GPL. Note: the website contains the code and data in a few separate files, mostly to make it possible to download code without the huge data archives: PoserPython_GS.tar.gz : everything except for the data/data*.tar.gz files. Uncompressing this files will create all the directories (data, general, python, matlab) + this Readme file in the top level. The data directory will contain the usedPoses.txt file (see below), but not the images or pose labels themselves. data0003.tar.gz, data0003-l.tar.gz, data0007.tar.gz, data0007-l.tar.gz, data0012.tar.gz, data0012-l.tar.gz : these files contain images for three directories, arbitrarily chosen out of 56, with total of 90,000 pose examples (see description of data below for details). Putting all the 56 directories online is currently infeasible due to server space limitations. ************ general/ : general files This file (Readme.txt) provides general information about the archive. paramList.txt : describes the mapping of indices in the parameter (angle) vector, as it appers in Angles.txt (see below), to Poser "actors" (body parts/joints). See also python/humanparam.py, python/readparam.py, python/saveparam.py. jointList.txt : describes the mapping of indices of joint location vector to specific joints (described by their Poser names and the corresponding colloquial names). See also python/humanparam.py, python/savepose.py. figureClothed.pz3 : Poser figure used in some of the experiments, including clothing (with figure names corresponding to the assumptions in the code in scenemanip.py). ************ data/ : data archives This directory contains a sample for a data set based on motion capture data available (for free) at www.mocapdata.com. The data were created by running the script python/renderDB.py. An important note: the data are rendered by taking the MoCap poses, in arbitrary order, and rendering them from arbitrary point of view (equivalently, by rotating the body by arbitrary angle along the vertical axis, while keeping the camera fixed). Thus, the sequential order is lost. A set of more than 500,000 poses is found in file **** usedPoses.txt Of course, the same code can be used to render a coherent sequence, if the pose (angles) file contains frames from such a sequence rather than a randomly permuted set of poses. See python/runBVH.py for a script that creates such a pose file. The data are stored in the directories data0001,data0002 etc. The N-th directory contains 30,000 examples, from (N-1)*30000+1 through N*30000. For each example number I, there are three image files: ex%08d-r.jpg : intensity (color) image of a full body figure, obtained with the camera parameters described in plotJoint.m. ex%08d-fg.jpg : the foreground mask for the image in ex%08d-r.jpg. These data are available in separate archived files. Downloading, e.g., data0003.tar.gz to directory data/ and running (on Linux, UNIX, or Cygwin) tar -zxf data0003.tar.gz will produce a directory data/data0003 and in it files data/data0003/ex00060001-r.jpg data/data0003/ex00060001-fg.jpg data/data0003/ex00060002-r.jpg data/data0003/ex00060002-fg.jpg ... etc. up to 90000. Also, each data directory contains the following files that describe all the examples in that directory: Angles%04N.txt : the joint angles - these parameters describe the articulation of the body, and are sufficient to reproduce (up to some nuisance parameters such as finger pose) the given body configuration in Poser (see python/readparam.py). All numbers are in degrees. XYZ%04N.txt : the joint locations in 3D. These are in centimeters. The joint coordinates can not (to my knowledge) be used directly to render a pose in Poser, but are much easier to process to measure pose similarity or plot the projection of joints or body parts in a 2D image. Note: the X and Z coordinates can be normalized to a canonical location in the origin of the ground plane by translating the pose so that the Hip joint has X=0 and Z=0 (see normalizeLoc.m). Additional files for the data directories: --- The left image (comprising a stereo pair with the right image) is available for every intensity image in data. There are no silhouettes for these images. The left camera has all the same parameters as the right one, but is displaced by 5cm - i.e., its center is at [-.05; 2.3; 1]. Downloading, e.g., data0003-l.tar.gz to the same directory (say, data/) in which you have unzipped data0003, and then running tar -zxf data0003-l.tar.gz will add to data0003 the files data/data0003/ex00060001-l.jpg data/data0003/ex00060002-l.jpg etc., to 90000 *********** python/ : PoserPython code This directory contains code used in Poser to create and visualize the data. More information available in the documentation in the code; below is a list of files with a brief description for each. NOTE: this code has been used in Poser 5. It may not work in more recent versions of Poser, although in that case it should be fairly easy to fix whatever is not compatible with the newer Poser. If you introduce any such fixes, I will happy to know about that. **** fixnames.py A fix for a bug in Poser that mangles internal names for Twist parameters for the human figure. **** handmanip.py functions for some rather coarse hand manipulation (coordinated finger bending, grasping etc.) **** handparam.py Parameters and basic functionality for hand representation and manipulation **** humanparam.py Parameters and basic functionality for full body representation and manipulation (note: for detailed hand control including individual fingers, look at handparam.py) **** mirrorpose.py A script which, when run from Poser, flips the full body pose left-to-right (i.e., creates a "mirror reflection" if the mirror were to be placed at sagittal section) **** readparam.py Read a pose (i.e. angles) from a file, and apply it **** render1DOF.py Implements a function which renders a sequence of images in which a single DOF is changed over a range. **** renderDB.py This script generates a DB of pose-labeled images. Main script used to generate the data like those in ICCV 2003 paper (PSH) and in my PhD thesis. **** renderSequence.py A script to take a single pose sequence and render the images **** renderShape.py script to render and save high-quality images for given poses (for hand-only figure) **** renderhand.py This script reads a sequence of hand postures from the given file and renders them; when done, the read poses form frames in an animation **** runBVH.py This script goes over a list of .bvh files, and for each file: - cleans the figure - loads the BVH sequence; - for every frame in the sequence, applies the pose and saves the parameters **** runPoseSeq.py Provides the function that loads a file with a sequence of poses described by articulated angles, renders them, possibly modifying global location and orientation as well as other parameters, and saves images and parameters into files. **** saveCamParam.py Script that prints out the parameters of current camera into a file **** saveparam.py Script thar writes out the current pose as a list of joint angles to a file **** savepose.py Script thar writes out the current pose as a list of joint positions (in 3D) to a file **** scenemanip.py This file contains functions used in manipulating the Poser scene (changing clothing, rendering images, etc.) **** setcam.py Set camera parameters; location, orientation, lense params are currently hard-coded in this file (can change to read them from a file) **** setscale.py Implements a function that stretches/shrinks all body parts in the figure. The following three files have sample code for rendering various special configurations for hand-only figure (including motion by smooth transitions between poses; see also render1DOF.py). **** stimulusGrasp.py* **** stimulusGraspTransition.py* **** stimulusIndexFinger.py* *********** matlab/ : Matlab code getEDH.m : calculating the edge direction histogram, using integral images of the oriented edge maps. plotJoints.m : plot 2D projections of 3D joint locations on top of an image (using assumed camera calibration) Additional code for calculating EDH features used in the ICCV 2003 paper; see .m files for documentation.