# 2017-09-05

### Sample mixing layer, rotation invariance and AA

- Quadratic sample-sample interaction
- Large kernels (\(21\times21\))
- Race condition in writing the kernel's gradient: many updates from irregular sample locations: atomics are slow.

##### Rotation layer: no clear improvement over standard convolutions (AA)

- Standard convolutions with same spatial footprint are on-par.
- They work with less information: no subpixel xy location provided.
- Task too easy to make a difference? (Edges are sharp)
- How does our subsample, rotated kernels help with localizing the edge more precisely?

Average MSE over 30 images, 1spp reconstruction.

Edges |
0.000289 |
0.000374 |
**0.000237** |
-- |

Sines |
0.000340 |
0.000330 |
**0.000225** |
-- |

Triangles |
**0.000733** |
0.000847 |
0.001081 |
0.000924 |

##### Rotation layer: other concerns

- How does this generalize? Separable 2D embeddings in generalized coordinates (e.g. y-t)?
- No scaling at this point: we might need the full 2D anisotropic formulation to see a difference w.r.t. standard convolutions.
- Why discretize the kernel at all (TM)?

##### Other option? Learn pairwise sample interactions directly

- In the rotated case, the samples are mixed as \(\phi_s^O = \sum_{s'\in\mathcal{N}(s)}w(s, s') \phi^I_{s'}\), where \(w\) is fixed (learned) and depends on the relative position of \(s'\) w.r.t. \(s\) in a rotated/normalized frame.
- Can we learn a more complex \(w\) function,
**without discretizing** it as a kernel?
- What do we seek? Essentially, perform a local clustering of the samples and average within groups, rather than across.
- Relative quantities matter: difference w.r.t. to the center sample.
- Some sort of diffusion process? Alternate between 2 stages: sample interaction and features mixing within a sample.
- Shouldn't we preserve the untouched sample radiance somehow? residual skip connection?
- Multi-scale diffusion