2017-09-05

Sample mixing layer, rotation invariance and AA

Rotation layer: performance is a problem
• Large kernels ($$21\times21$$)
• Race condition in writing the kernel's gradient: many updates from irregular sample locations: atomics are slow.
Rotation layer: no clear improvement over standard convolutions (AA)
• Standard convolutions with same spatial footprint are on-par.
• They work with less information: no subpixel xy location provided.
• Task too easy to make a difference? (Edges are sharp)
• How does our subsample, rotated kernels help with localizing the edge more precisely?

Average MSE over 30 images, 1spp reconstruction.

pixel no angle ours fc
Edges 0.000289 0.000374 0.000237 --
Sines 0.000340 0.000330 0.000225 --
Triangles 0.000733 0.000847 0.001081 0.000924

Rotation layer: other concerns
• How does this generalize? Separable 2D embeddings in generalized coordinates (e.g. y-t)?
• No scaling at this point: we might need the full 2D anisotropic formulation to see a difference w.r.t. standard convolutions.
• Why discretize the kernel at all (TM)?
Other option? Learn pairwise sample interactions directly
• In the rotated case, the samples are mixed as $$\phi_s^O = \sum_{s'\in\mathcal{N}(s)}w(s, s') \phi^I_{s'}$$, where $$w$$ is fixed (learned) and depends on the relative position of $$s'$$ w.r.t. $$s$$ in a rotated/normalized frame.
• Can we learn a more complex $$w$$ function, without discretizing it as a kernel?
• What do we seek? Essentially, perform a local clustering of the samples and average within groups, rather than across.
• Relative quantities matter: difference w.r.t. to the center sample.
• Some sort of diffusion process? Alternate between 2 stages: sample interaction and features mixing within a sample.
• Shouldn't we preserve the untouched sample radiance somehow? residual skip connection?
• Multi-scale diffusion