Multi-View Scene Flow Estimation:
A View Centered Variational Approach
Tali Basha              Yael Moses              Nahum Kiryati

Abstract   We present a novel method for recovering the 3D structure and scene flow from calibrated multi-view sequences. We propose a 3D point cloud parameterization of the 3D structure and scene flow that allows us to directly estimate the desired unknowns. A unified global energy functional is proposed to incorporate the information from the available sequences and simultaneously recover both depth and scene flow. The functional enforces multi-view geometric consistency and imposes brightness constancy and piecewise smoothness assumptions directly on the 3D unknowns. It inherently handles the challenges of discontinuities, occlusions, and large displacements. The main contribution of this work is the fusion of a 3D representation and an advanced variational framework that directly uses the available multi-view information. This formulation allows us to advantageously bind the 3D unknowns in time and space. Different from optical flow and disparity, the proposed method results in a nonlinear mapping between the images’ coordinate, thus giving rise to additional challenges in the optimization process. Our experiments on real and synthetic data demonstrate that the proposed method successfully recovers the 3D structure and scene flow despite the complicated nonconvex optimization problem.
[IJCV'12]    [CVPR'10]

Contains the calibrated multi-view datasets. If you use these datasets in any publication, please refer to our paper.

Sythetic Data -  Rotating ball & Rotating Background
This dataset was generated in OpenGL. It consists of a rotating sphere placed in front of a rotating plane. The plane is placed at Z=700 (the units are arbitrary) and the center of the sphere at Z=500 with radius of 200. The scene is viewed by five rectified cameras.
The calibration is available here.

z u
v w
Download frames Download ground truth

Real Data

These datasets were acquired by three USB cameras (IDS uEye UI-1545LE-C).
The cameras were calibrated using the MATLAB Calibration Toolbox.
All test sequences were taken with an image size of 1280 X 1024 and then downsampled by half. The full calibration (intrinsic and extrinsic) for all the datasets is available here

cat maria
Download frames
Cars - small moving obect
Download frames
Cat - Large motion in depth
Download frames
Maria - face rotation
Code [Download]