|
Multi-View
Scene Flow Estimation:
A View Centered Variational Approach |
 |
|
Tali
Basha Yael Moses
Nahum Kiryati |
|
|
Abstract We
present a novel method for recovering the 3D structure and scene flow
from
calibrated multi-view sequences. We propose a 3D point cloud
parameterization
of the 3D structure and scene flow that allows us to directly estimate
the
desired unknowns. A unified global energy functional is proposed to
incorporate
the information from the available sequences and simultaneously recover
both
depth and scene flow. The functional enforces multi-view geometric
consistency
and imposes brightness constancy and piecewise smoothness assumptions
directly
on the 3D unknowns. It inherently handles the challenges of
discontinuities,
occlusions, and large displacements. The main contribution of this work
is the
fusion of a 3D representation and an advanced variational framework
that
directly uses the available multi-view information. This formulation
allows us
to advantageously bind the 3D unknowns in time and space. Different
from
optical flow and disparity, the proposed method results in a nonlinear
mapping
between the images’ coordinate, thus giving rise to additional
challenges in
the optimization process. Our experiments on real and synthetic data
demonstrate
that the proposed method successfully recovers the 3D structure and
scene flow
despite the complicated nonconvex optimization problem.
|
-------------------------------------------------------------------------------------------------------------------------- |
|
|
Paper
[IJCV'12] [CVPR'10]
--------------------------------------------------------------------------------------------------------------------------
|
|
|
Presentation
[ppsx]
-------------------------------------------------------------------------------------------------------------------------- |
|
|
Data
Contains the
calibrated
multi-view datasets. If you use these
datasets in any
publication, please refer to our paper.
|
|
|
Sythetic
Data -
Rotating ball & Rotating Background
This
dataset was generated in OpenGL. It consists of a rotating sphere
placed in front of a rotating plane. The plane is placed at Z=700 (the
units are arbitrary) and the center
of the sphere at Z=500 with radius of 200. The scene is viewed by five
rectified cameras.
The calibration
is available here.
|
|
|
Real Data
These datasets
were acquired by three USB cameras (IDS uEye UI-1545LE-C).
The cameras
were calibrated using the
MATLAB
Calibration Toolbox.
All test sequences were
taken with an image size of 1280 X 1024 and
then downsampled by half. The
full calibration
(intrinsic and extrinsic) for all the datasets is available here.
-------------------------------------------------------------------------------------------------------------------------- |
|
|
Code
[Download] |
|
|
|
|
