To test our approach, we used a 12 or 16Mpixel camera with a
wide-aperture fixed focal length lens. We captured images across 13
apertures for each of 61 focus settings, and designed our workspace
to be 17cm deep, giving about 3mm depth resolution between
successive focal settings.

[results video]

Our first dataset is the scene containing a wig and artificial
plants that we've been looking at throughout the talk.

In these results we compare our AFI model fitting method with a
shape-from-focus method, based on maximizing the variance within a 
3x3 spatial window. For the rest of the video, the top-right image 
will cycle thorugh the widest aperture images of the dataset which
are precisely those used by 3x3 variance.

At a coarse level, both methods accurately recover the overall shape
of the scene, however for background pixels, which are untextured
and outside of the workspace, the estimation is unreliable.

For a zoomed-in region of the scene, we can see that the direct 
application of confocal constancy, to columns of the AFI, is not 
particularly useful. For other scenes, such as the ground-truth 
dataset we present later, confocal constancy performed relatively 
well. The issue for confocal constancy is that most pixels have 
multiple modes, so a greedy pixel-level minimization of the focus 
measure is insufficient.

On the other hand, the AFI model fitting method leads to very 
detailed depth maps, at the level of individual strands of hair. 
Variance, by contrast, produces halos around narrow  structures, 
does not reconstruct the foreground hairs as accurately, and shows 
further reconstruction artifacts as well. I should emphasize that all 
reconstruction results here are shown on a pixel level without any 
spatial smoothing.

We also explored filtering out pixels for which the metric is
uninformative, such as the completely textureless case, by removing
pixels whose peak for the focus measure is too broad. We have found
that this filtering can significantly help reject outliers for AFI
model fitting. Note that for 3x3 variance, such filtering has little
visible effect, since the worst artifacts there are caused by
multiple incorrect modes.

Although confocal constancy does assign low error to the correct
in-focus setting, the issue is that multiple modes are usually
produced for each pixel, so a greedy pixel-level minimization of the
focus measure is not sufficient to resolve structure.

For another region of the scene, at the silhouette of the hair, the
AFI model fitting reconstruction resolves fine structure, despite
the fact that depths in the scene vary greatly within small
neighborhoods. By contrast, spatially-based methods like variance
produce uniformly lower-resolution results. Again, filtering the
pixels with broadest peaks improves the quality for the AFI case
only. All of this suggests that the AFI metric is more informative.

One area where spatially-based methods like variance have an
advantage is in smooth, less-textured regions like the flower
petals. Because the smooth surface models they assume are satisfied,
they can exploit focus information from nearby points more directly.
AFI model fitting produces more outliers in this region, even if
this effect is somewhat mitigated by filtering.

Finally, to evaluate quantitative accuracy, we consider a dataset
consisting of a tilted box, wrapped in newsprint, for which we
recovered ground-truth depth information for the front plane, using
a sub-millimeter-accurate 3D probe.

While the varying texture characteristics of the box leads to
outliers, for both variance and AFI model fitting, most of the
reconstructed 3D points agree well with the ground truth data.

For a smaller region of the box scene, we can see the level of
reconstruction accuracy more clearly, with the ground-truth plane
shown overlaid as a blue 3D mesh. Although the planar scene can
be thought of as a best case for the spatially-based variance
method, AFI model fitting performs at the same level or better in
terms of accuracy and number of outliers. Here again, filtering the
AFI model fitting results significantly improves quality of the
reconstruction.

[table of quantitative results]

In numerical terms, all methods performed on a similar level in
terms of ground truth accuracy: around 2.5mm error, or about one
focus setting. AFI model fitting had a slight advantage in terms of
accuracy and number of outliers, however all methods were in-line
with previously reported quantitative results.