Boundary Matting for View Synthesis

Samuel W. Hasinoff, Sing Bing Kang, and Richard Szeliski

 

Publications

Samuel W. Hasinoff, Sing Bing Kang, and Richard Szeliski, Boundary Matting for View Synthesis. Computer Vision and Image Understanding, 103(1), pp. 22-32, 2006. [pdf]

Samuel W. Hasinoff, Sing Bing Kang, and Richard Szeliski, Boundary Matting for View Synthesis, Second IEEE Workshop on Image and Video Registration (with CVPR 2004), 8 pp. (DVD proceedings). [pdf]

Journal abstract

In the last few years, new view synthesis has emerged as an important application of 3D stereo reconstruction. While the quality of stereo has improved, it is still imperfect, and a unique depth is typically assigned to every pixel. This is problematic at object boundaries, where the pixel colors are mixtures of foreground and background colors. Interpolating views without explicitly accounting for this effect results in objects with a "cut-out" appearance.

To produce seamless view interpolation, we propose a method called boundary matting, which represents each occlusion boundary as a 3D curve. We show how this method exploits multiple views to perform fully automatic alpha matting and to simultaneously refine stereo depths at the boundaries. The key to our approach is the 3D representation of occlusion boundaries estimated to sub-pixel accuracy. Starting from an initial estimate derived from stereo, we optimize the curve parameters and the foreground colors near the boundaries. Our objective function maximizes consistency with the input images, favors boundaries aligned with strong edges, and damps large perturbations of the curves. Experimental results suggest that this method enables high-quality view synthesis with reduced matting artifacts.

Supplementary Materials

Acknowledgements

This work was supported by Microsoft Research.