Useful Information

Research Interest

My work mainly focuses on extracting information from photographs and videos. My goal is to obtain useful data for Computer Graphics, i.e. suitable for rendering new images. During my PhD with François Sillion at INRIA, I developed new solutions to image-based modeling, mainly shape reconstruction from image sequences and recovery of the 3D geometry of hair.

Since my post-doc at MIT with Frédo Durand, I am also interested in computational photography. I am looking at signal-processing techniques to apply them to digital photographs and videos. Compared to traditional film photography, this unveils a large spectrum of possibilities to modify and enhance the picture content. Conversely, photography-related applications raise specific issues such as user interaction and photorealism, and cast a new light upon known signal-processing methods.

I am now a researcher at Adobe, working in the Advanced Technology Labs where I pursue my research on computational photography and image-based modeling.

Internship

I am always happy to work with interns. If you are interested in doing an internship with me, here is what you need to know:

• These are research internships and most interns are graduate students. If you are not in grad school, make sure to include enough detail.
• I am collaborating with people who share the same interests as me. This page should help you figure out this point.
• Tell me what you have done in the past in terms of research, and what you would like to do during your internship.
• Most interns come for three months during the summer but other options are possible.
• Contact me at least four months in advance.
• The number of slots is limited. Unfortunately, we cannot accept everyone.
• I do not reply to generic requests. Make sure that your message shows that you know who I am.

So far, I have been lucky to work with the following interns:

• Eugene Hsu from MIT, now at Vistaprint, on Light Mixture Estimation for Spatially Varying White Balance (fall 2007)
• Sara Su from MIT, now at Google, on Interactive Visual Histories for Vector Graphics (fall 2007)
• Jiawen Chen from MIT, now at Microsoft Research, on The Video Mesh: A Data Structure for Image-based Video Editing (summer 2008)
• Elmar Eisemann from INRIA, now at Telecom ParisTech, on A Visibility Algorithm for Converting 3D Meshes into Editable 2D Vector Graphics (summer 2008)
• Ce Liu from MIT, now at Microsoft Research (summer 2008)
• Dhruv Mahajan from Columbia, now at Yahoo Research (summer 2008)
• Kalyan Sunkavalli from Harvard (summer 2008)
• Vladimir Bychkovsky from MIT, on Learning Photographic Global Tonal Adjustments with a Database of Input / Output Image Pairs (summer 2009, 2010, 2011)
• Mathieu Aubry from Technische Universität München and ENS/INRIA, on Fast and Robust Pyramid-based Image Processing (spring and summer 2011)

 

Publications

The main publications are highlighted with a thumbnail. Boxed publications without thumbnail are important milestones that have been extended in a journal paper. I also include a few more detailed talks, and a list of software products inspired from my research and to which I have contributed.

I have been lucky to co-author papers with the following people (by alphabetical order): Frederick Aliaga, Mathieu Aubry, Shai Avidan, Soonmin Bae, Jiamin Bai, Nicolas Bonneel, Tamy Boubekeur, Adrien Bousseau, Hector Briceño, Bert Buchholz, Vladimir Bychkovsky, Eric Chan, Will Chang, Jiawen Chen, Simon Chen, Matthieu Cunzi, Gilles Debunne, George Drettakis, Frédo Durand, Elmar Eisemann, Noura Faraj, Bill Freeman, Jean-Dominique Gascuel, Wolfgang Heidrich, Berthold Horn, Eugene Hsu, Wojciech Jarosz, Won-Ki Jeong, Kimo Johnson, Steve Johnson, Jan Kautz, Hao Li, Eric Kee, Pushmeet Kohli, Pierre Kornprobst, Oleg Kozhushnyan, Maxime Lhuillier, Linjie Luo, Wojciech Matusik, Tom Mertens, Mark Pauly, Hanspeter Pfister, Jovan Popović, Long Quan, Craig Scull, Szymon Rusinkiewicz, François Sillion, Sara Su, Michael Tao, Joëlle Thollot, Jack Tumblin, Michiel van de Panne, Jue Wang, Robert Wang, Thibaut Weise, Insu Yu, Gang Zeng, Matthias Zwicker.

Access by year: 2012, 2011, 2010, 2009, 2008, 2007, 2006, 2005, 2004, 2003, 2002.

Somewhere else: DBLP, Google Scholar.

 

2012

 

SimpleFlow: A Non-iterative, Sublinear Optical Flow Algorithm Michael Tao, Jiamin Bai, Pushmeet Kohli, and Sylvain Paris Computer Graphics Forum (Proceedings of the Eurographics'12 conference) pdf (45MB) | mp4 (38MB) | supplemental material

 

2011

 

Displacement Interpolation Using Lagrangian Mass Transport Nicolas Bonneel, Michiel van de Panne, Sylvain Paris, and Wolfgang Heidrich ACM Transactions on Graphics (Proceedings of the ACM SIGGRAPH Asia'11 conference) pdf (15MB) | project webpage The demo video and some code are available on the project webpage. This is a revised version of our technical report.

 

6D Hands: Markerless Hand Tracking for Computer Aided Design Robert Wang, Sylvain Paris, and Jovan Popović Proceedings of the ACM Symposium on User Interface Software and Technology (UIST'11) pdf (1.4MB) | video (25MB) | additional video (20MB)

 

Practical Color-Based Motion Capture Robert Wang, Sylvain Paris, and Jovan Popović Proceedings of the Symposium on Computer Animation (SCA'11) pdf (1.8MB) | video (40MB) | additional video (13MB) This is a revised version our tech report.

 

Spatio-temporal Analysis for Parameterizing Animated Lines Bert Buchholz, Noura Faraj, Sylvain Paris, Elmar Eisemann, and Tamy Boubekeur Proceedings of the International Symposium on Non-Photorealistic Animation and Rendering (NPAR'11) pdf (7.3MB) | video (28MB) | project page Our work will be also presented as a SIGGRAPH talk (formerly "sketch").

 

Local Laplacian Filters: Edge-aware Image Processing with a Laplacian Pyramid Sylvain Paris, Samuel W. Hasinoff, and Jan Kautz ACM Transactions on Graphics (Proceedings of the ACM SIGGRAPH'11 conference) full-res. pdf (47MB) | low-res. pdf (4.4MB) See our project page for additional results and source code. Hindsight: The fact that are our filters produce faithful low-resolution previews is related to our work on display-aware image editing. Our tech report further studies the properties of local Laplacian filters and describes a couple of extensions, including a faster algorithm.

 

Learning Photographic Global Tonal Adjustments with a Database of Input / Output Image Pairs Vladimir Bychkovsky, Sylvain Paris, Eric Chan, and Frédo Durand Proceedings of the conference on Computer Vision and Pattern Recognition (CVPR'11) pdf (3.7MB) | project page All the data are available.

 

Blur kernel estimation using the Radon Transform Taeg Sang Cho, Sylvain Paris, Berthold K. P. Horn, and William T. Freeman Proceedings of the conference on Computer Vision and Pattern Recognition (CVPR'11) pdf (11MB) Code and data are available on the webpage of Taeg Sang Cho's PhD dissertation.

 

Modeling and Removing Spatially-Varying Optical Blur Eric Kee, Sylvain Paris, Simon Chen, and Jue Wang Proceedings of the International Conference on Computional Photography (ICCP'11) pdf (2.2MB)

 

Display-aware Image Editing Won-Ki Jeong, Micah K. Johnson, Insu Yu, Jan Kautz, Hanspeter Pfister, and Sylvain Paris Proceedings of the International Conference on Computional Photography (ICCP'11) pdf (8.9MB) | video (99MB) Our SIGGRAPH'11 paper describes a family of edge-aware filters that are display-aware.

 

The Video Mesh: A Data Structure for Image-based Three-dimensional Video Editing Jiawen Chen, Sylvain Paris, Jue Wang, Wojciech Matusik, Michael Cohen, and Frédo Durand Proceedings of the International Conference on Computional Photography (ICCP'11) pdf (10.1MB) | project page This is a revised version of our tech report. More results, including videos, are available on the project page.

       

2010

 

Error-tolerant Image Compositing Michael W. Tao, Micah K. Johnson, and Sylvain Paris Processing of the European Conference on Computer Vision (ECCV'10) pdf (4.3MB) | slides ppt (2.5MB) | slides pdf (2.6MB) More details on the project webpage.

   

2009

 

Bilateral Filtering: Theory and Applications Sylvain Paris, Pierre Kornprobst, Jack Tumblin, and Frédo Durand Foundations and Trends in Computer Graphics and Vision pdf (11MB) | official website This survey is an extended version of our course. Supplemental material including code is available on the course webpage.

 

User-assisted Intrinsic Images Adrien Bousseau, Sylvain Paris, and Frédo Durand ACM Transactions on Graphics (Proceedings of the ACM SIGGRAPH Asia'09 conference) pdf (25MB) | mov (53MB) | supplemental material (74MB) | ppt (36MB) We provide the input and output images in supplemental material.

 

Single Photo Estimation of Hair Appearance Nicolas Bonneel, Sylvain Paris, Michiel van de Panne, Frédo Durand, George Drettakis Computer Graphics Forum (Proceedings of the Eurographics Symposium on Rendering '09) pdf (18MB) | avi (82MB) | supplemental material (57MB) We provide our input photos, the retrieved parameters, and the corresponding renderings in supplemental material.

 

A Visibility Algorithm for Converting 3D Meshes into Editable 2D Vector Graphics Elmar Eisemann, Sylvain Paris, and Frédo Durand ACM Transactions on Graphics (Proceedings of the ACM SIGGRAPH'09 conference) pdf (21MB)

 

QuickSelect: History-Based Selection Expansion Sara Su, Sylvain Paris, and Frédo Durand Proceedings of the Graphics Interface conference (GI'09) pdf (2.6MB) More on the project webpage

 

A Fast Approximation of the Bilateral Filter using a Signal Processing Approach Sylvain Paris and Frédo Durand International Journal of Computer Vision (IJCV'09) pdf (4.7MB) | ps (62MB) Our fast bilateral filter webpage provides details, code, data, and more. This article extends our conference paper and is a revised version of our technical report. Compared to the ECCV article, it includes new discussions, comparisons, and validations, and extensions to cross bilateral filtering and color images. Our SIGGRAPH'07 paper builds upon this article and shows that the higher-dimensional image representation that we introduce can be turned into a data structure useful a variety of other edge-aware tools. We also show that the corresponding algorithms, including the bilateral filter, are highly parallel and can be implemented on graphics hardware. All the images shown in the figures are available as supplemental material (23MB). The original publication is available at www.springerlink.com.

       

2008

 

Edge-preserving Smoothing and Mean-shift Segmentation of Video Streams Sylvain Paris Processing of the European Conference on Computer Vision (ECCV'08) pdf (1.7MB) | mov (98MB) This article follows up my mean-shift and bilateral grid papers. Hindsight: Here are a few more details about the real-time mean-shift algorithm.

 

Hair Photobooth: Geometric and Photometric Acquisition of Real Hairstyles Sylvain Paris, Will Chang, Wojciech Jarosz, Oleg Kozhushnyan, Wojciech Matusik, Matthias Zwicker, and Frédo Durand ACM Transactions on Graphics (Proceedings of the ACM SIGGRAPH'08 conference) pdf (4.5MB) | slides pdf (90 MB) | slides ppt (16MB) | zip with ppt and videos (105MB) | zip demo videos (248MB) The paper webpage provides additional material. This work extends our SIGGRAPH'04 article in many ways: a more accurate and more complete setup, an inference method to retrieve the hidden geometry, a model-driven image-based rendering technique,...

 

Light Mixture Estimation for Spatially Varying White Balance Eugene Hsu, Tom Mertens, Sylvain Paris, Shai Avidan, and Frédo Durand ACM Transactions on Graphics (Proceedings of the ACM SIGGRAPH'08 conference) pdf (1.9MB) The paper webpage provides additional material.

 

2007

 

Real-time Edge-Aware Image Processing with the Bilateral Grid Jiawen Chen, Sylvain Paris, and Frédo Durand ACM Transactions on Graphics (Proceedings of the ACM SIGGRAPH'07 conference) pdf (13MB) | movie (mpeg4, 61MB) This article focuses on an image representation inspired from our ECCV'06 paper. This data structure can be used for a variety of image processing tools such as bilateral filtering, scribble interpolation, edge-preserving painting, and contrast enhancement. The corresponding algorithms can be efficiently parallelized on graphics hardware. More details on the bilateral grid webpage. Hindsight: My paper on video streams further motivates the use of the exponential decay in Section 3.5.

 

A Topological Approach to Hierarchical Segmentation using Mean Shift Sylvain Paris and Frédo Durand Proceedings of the IEEE conference on Computer Vision and Pattern Recognition (CVPR'07) pdf (1.5MB) | ps.gz (14MB) | movie (mpeg4, 35MB) | poster (pdf, 2.8MB) C++ code is available. We tested our algorithm on videos with 6 seconds of the pylon sequence (from frame 80 to frame 229). In the supplemental material (tar.gz, zip), we provide results on high-resolution images and on the Berkeley benchmark. My paper on video streams extends this work to real-time video segmentation. Hindsight: Segmentations done on-the-fly are actually similar to off-line segmentations only at grid resolution. Differences may exist at pixel resolution. Thanks to Varsha Hedau from UIUC for pointing out differences.

 

Accurate and Scalable Surface Representation and Reconstruction from Images Gang Zeng, Sylvain Paris, Long Quan, and François Sillion IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI'07) pdf (3.1MB) This article extends our ECCV'04 paper and our ICCV'05 paper. A previous version appeared as a technical report. Our patchwork approach is demonstrated to have optimal spatial and temporal complexities in terms of object area and resolution. We also formally study the link between the proposed local optimization and existing global techniques (weak membrane, level set, and graph cut). New results illustrate various aspects: influence of the parameters, differences between propagation (presented at ECCV'04) and carving (ICCV'05), and failure cases.

 

2006

 

Two-scale Tone Management for Photographic Look Soonmin Bae, Sylvain Paris, and Frédo Durand ACM Transactions on Graphics (Proceedings of the ACM SIGGRAPH'06 conference) high-res. pdf (8.9MB) | low-res. pdf (1.5MB) | gallery More details on the project webpage. To achieve interactive performance, we use the fast bilateral filter described in our ECCV'06 article. Dani Lischinski and colleagues describe a complementary approach based on user interaction. Hindsight: Our ICCP'11 paper provides a user study that confirms the relationship between image histograms and photo editing.

 

A Surface Reconstruction Method Using Global Graph Cut Optimization Sylvain Paris, François Sillion, and Long Quan International Journal on Computer Vision (IJCV'06) pdf (1MB) | ps (10MB) | more... This article is an extended version of our ACCV'04 paper. Our ECCV'04, ICCV'05, and PAMI papers present a versatile technique to use the optimization method described in this article to recover objects with arbitrary shapes. The min-cut code is available. The original publication is available at www.springerlink.com.

 

A Fast Approximation of the Bilateral Filter using a Signal Processing Approach. Sylvain Paris and Frédo Durand. European Conference on Computer Vision (ECCV'06). pdf (3MB) | ps (9.5MB) | ppt (2.7MB) Our fast bilateral filter webpage provides details, code, data, and more. We have extended this article in an IJCV article that includes new discussions, comparisons, and validations, and extensions to cross bilateral filtering and color images.

   

2005

 

Progressive Surface Reconstruction from Images using a Local Prior. Gang Zeng, Sylvain Paris, Long Quan, and François Sillion. International Conference on Computer Vision (ICCV'05). pdf (1.4MB) | ps (22MB) | gallery Our PAMI paper extends this work with additional studies and results. This paper stems from the same inspiration as our ECCV'04 paper (reconstruction by patches) but it exposes a different technique and it provides additional formal studies. In short, we do not need points to drive the process, the visibility is better evaluated using a voxel representation, and we guarantee the smoothness of the resulting model. The optimization technique we use is detailed in our IJCV'06 paper.

     

2004

 

Extraction of Three-dimensional Information from Images -- Application to Computer Graphics Sylvain Paris Ph.D. thesis from Université Joseph Fourier (Grenoble, France) 2004 pdf (7MB) | ps (65MB) | ppt (26MB) | more... This dissertation compiles my work on 3D reconstruction (ECCV'04 and IJCV'06), face relighting (PG'03) and capture of hair geometry (SIGGRAPH'04). Compared to the papers, there are extended discussions of the previous work, more technical details and additional studies.

 

Capture of Hair Geometry from Multiple Images Sylvain Paris, Hector Briceño, and François Sillion ACM Transactions on Graphics (Proceedings of the ACM SIGGRAPH'04 conference) pdf (5MB) | ppt (12MB) | mpeg4 (32MB) | appendix (74kB) | more... The hairstyles shown in the article are available. Just contact one of the authors with a short description of your project (we are interested in the use of our data). Some image-processing code is also available. Florence Bertails and colleagues have published at GI'05 the rendering technique used in the last slide of our presentation. Our work is inspired by the hair acquisition technique of Stéphane Grabli and colleagues at GI'02. Yichen Wei and colleagues have published at SIGGRAPH'05 another method on acquisition of hair geometry. Compared to our paper, they use several viewpoints and require only a well lighted environment. The paper is available on Long Quan's page.

 

Surface Reconstruction by Propagating 3D Stereo Data in Multiple 2D Images. Gang Zeng, Sylvain Paris, Long Quan, and Maxime Lhuillier. Proceedings of the European Conference on Computer Vision (ECCV'04). pdf (0.8MB) | more... Our PAMI paper extends this article with additional studies and results. Our ICCV'05 paper builds upon this work: It introduces a voxel representation instead of points to improve the visibility estimation and it studies the smoothness of the resulting surface. The optimization technique we use is exposed in our IJCV'06 paper.

   

2003

 

Lightweight Face Relighting Sylvain Paris, François Sillion, and Long Quan Proceedings of the Pacific Graphics conference (PG'03) mpeg videos: explanation (9MB) | comparison (6MB) | various effects (17MB) pdf (0.6MB) | ps (9MB) | more... The described rendering technique is somewhat old fashioned. Readers will still be interested in the proposed analysis of the skin reflectance.

 

Dynamic Canvas for Non-Photorealistic Walkthroughs Matthieu Cunzi, Joëlle Thollot, Sylvain Paris, Gilles Debunne, Jean-Dominique Gascuel, and Frédo Durand Proceedings of the Graphics Interface conference (GI'03) pdf (5MB) | ps (4MB) | more... The project page provides a lot of supplemental material (video, texture, code).

 

2002

 

Talks

 

Beyond the Piece of Cardboard: Learning to Adjust Photographs ppt (38MB) | pdf (61MB) A presentation of my work on photo adjustment using a model (and bilateral filtering or local Laplacian filtering) or machine learning.

 

Methods for 3D Reconstruction from Multiple Images ppt (6MB) | pdf (2.7MB) A review of multi-view stereo up to 2006. It is inpsired from Section 2.2 of my PhD dissertation and this survey by Seitz and colleagues. Full-length references are in the note section of the slides.

 

Products

Image Adjustments in Revel Some of the editing tools are inspired from my work on local Laplacian filters.

Photomerge Style Match in Photoshop Elements 9 This tool is inspired from my work on photographic style.

HDR Pro in Photoshop CS5 This tool is inspired from my work on photographic style.

Bilateral Filter in After Effects CS4 This filter is inspired from my work on the bilateral grid.

 

Research Material

 

Courses

 

A Gentle Introduction to Bilateral Filtering and its Applications Sylvain Paris, Pierre Kornprobst, Jack Tumblin, and Frédo Durand Class at ACM SIGGRAPH'08 Tutorial at IEEE CVPR'08 Course at ACM SIGGRAPH'07 Course notes, slides, references, and source code are available on the course webpage.

 

C++ Code

I have developed these packages to implement some of techniques exposed in my papers. They may help you to get your own implementation. Sample code is included in the documentation. You may need the basic tools to use some packages.

Basic tools	Array, vectors, colors, and so on. Needed by the other packages. tar.gz (161kB) | zip (263kB) | documentation
Min cut	Min-cut implementation for huge grid graphs (supports millions of nodes and vertices). Needs the basic tools. tar.gz (119kB) | zip (161kB) | documentation
Filters	A set of 2D functions useful for image processing. Needs the basic tools. tar.gz (94kB) | zip (196kB) | documentation
FFT	Convenient functions to use FFTW for image processing. Needs the basic tools. tar.gz (35kB) | zip (62kB) | documentation
Fast bilateral filter Fast tone mapping	Code is provided on our project webpage on fast bilateral filtering.
Bilateral grid	Jiawen gives the code of SIGGRAPH'07 paper.
Mean shift	Implementation of the algorithm described in our CVPR'07 article. tar.gz (59kB) | zip (74kB)

Illustrations

Mean-Shift Segmentation and Morse Theory The beginning of this video shows how mean-shift segmentation is defined and illustrates some concepts of Morse theory.

Spherical Harmonics Have you ever wonder how SH of order greater than 3 look like? This page shows up to order 10.

PDE filters A few videos that illustrate the behaviors of PDE filters. The viewing guide shortly describes what you see. The movie files (quicktime, mpeg, image sequence): Heat equation: mov (2.1MB) | mpg (3.7MB) | zip (7.7MB) Heat equation / adaptive smoothing: mov (3.1MB) | mpg (3.7MB) | zip (14MB) Adaptive smoothing / Perona-Malik: mov (3.0MB) | mpg (3.7MB) | zip (14MB) Perona-Malik with an outlier: mov (1.9MB) | mpg (3.7MB) | zip (9.2MB) Perona-Malik / prefiltered Perona-Malik: mov (3.0MB) | mpg (3.7MB) | zip (16MB)

Useful Links

Here is a collection of websites that I often use. If you do not know them, take some time to have look, you will not regret.

Programming Tools for C++

libQGLViewer	A wonderful OpenGL viewer
QT	GUI made easy
FFTW	Fast Fourier Transform (1D or 2D, real or complex). The doc is not very clear but it is really fast.
GSL	A set of scientific tools well documented and quickly usable
Hoard	Speed up memory access (does not seem to compile on 64bits machines)
ANN	Computes nearest neighbors in arbitrary dimensions. Well documented and easy to use.

Bookmarks

Graphics and Vision people	This is an extract from my bookmarks. It can be directly imported into Firefox bookmarks (let me know if it works with other browsers). That kind of list is never complete, please forgive me if you do not appear or if some data are wrong, and send me an e-mail.

Miscellaneous

BruceLindbloom.com EasyRGB.com	Good references about color spaces (including translation formulae)
WebFX PNG Behavior	A simple workaround to get transparent PNG correctly displayed even with IE6 (IE7 fixed the problem).
Semantic Atlas	Graph approach for English-French, French-English and synonym dictionaries (this is a research project)
MathWorld	Good reference for mathematical formulae
Calibrated image sequences	Images and calibration matrices. My head is one of the data sets. I am a data set!

Old Projects

These are some past projects (mostly undergrade and Masters assignments). They are written in French. I do not maintain any of these. However, code is provided and might be useful.

The 12-coin problem	How to solve a logical problem using stepwise entropy reduction.
An illustrated calendar	I enjoyed a Computer Art course.
Mesh simplification	It follows a classical edge collapse strategy.
Image denoising	A comparison between several PDE filters.

 

Personal Area

Chinese Painting

	Nelson did a Chinese painting of my name when I was an intern in Hong Kong. He used his virtual brush. He comments the painting on his webpage.

Photography

Film

Thanks to Eugene, I have experienced film photography and darkroom development. Here are scanned versions of my two pictures (click to enlarge):

Portraits

Eugene has shot a few portaits of me with my digital camera. The top picture is one of them. See the other portraits.

Digital

Here is a selection of my pictures sorted by year.

---

Check XHTML | Check CSS