Sharks,

i'll be presenting this as a poster on thursday @ iclr! please come by & chat if any of this sounds interesting to you :)

thanks for reading. link to the paper again: openreview.net/forum?id=IqH...

13/13

in sum, we used dominant components of the neural response to get an **axis-sensitive** measure of similarity.

this work fits into a broader look at (R)epresentational alignment (cf. work by @taliakonkle.bsky.social, @itsneuronal.bsky.social, @sucholutsky.bsky.social, & others)

12/n

we also used connectivity matrices to capture behaviorally-relevant information. a lot like rsa! but with a sparse coding structure

11/n

using sca and a few different pre-trained models, we found markedly higher alignment to the ventral stream.

rotationally invariant methods were less sensitive to this finding, providing an answer to question 2: DNNs are more similar to the ventral stream along a native axis of neural tuning

10/n

the resulting matrices represent the activity of sparse sub-populations of neurons/units and, unlike some methods, are quite sensitive to rotations in neural space

you can thus interpret sca as measuring similarity along a specific set of tuning axes

9/n

we applied the same decomposition to DNN activations and used them in a method we call **sparse component alignment** (sca). sca compares representations at the population level using image x image connectivity matrices.

(see the paper for complete derivation)

8/n

these response profiles gave an answer to question 1: there are interpretable and functionally-distinct representations across the brain's three visual pathways.

nice to see! but also heightens the mystery of question 2: why aren't these differences picked up by standard similarity metrics

7/n