Roy Eyono
@royeyono.bsky.social
PhD Student in NeuroAI @Mila and @McGill
Wouldn't have been possible without you! Thanks Dan!!
March 19, 2026 at 6:25 PM
Wouldn't have been possible without you! Thanks Dan!!
🫶 This work was done in collaboration with @dlevenstein.bsky.social @tyrellturing.bsky.social @arnaghosh.bsky.social @repromancer.bsky.social
Check out the full paper to dive deeper into our findings!
Paper: arxiv.org/abs/2603.17676
Check out the full paper to dive deeper into our findings!
Paper: arxiv.org/abs/2603.17676
Inhibitory normalization of error signals improves learning in neural circuits
Normalization is a critical operation in neural circuits. In the brain, there is evidence that normalization is implemented via inhibitory interneurons and allows neural populations to adjust to chang...
arxiv.org
March 19, 2026 at 3:27 PM
🫶 This work was done in collaboration with @dlevenstein.bsky.social @tyrellturing.bsky.social @arnaghosh.bsky.social @repromancer.bsky.social
Check out the full paper to dive deeper into our findings!
Paper: arxiv.org/abs/2603.17676
Check out the full paper to dive deeper into our findings!
Paper: arxiv.org/abs/2603.17676
These findings complement several theories on how SST-subtypes targeting apical dendrites (where error-related signals arrive) are critical in supporting learning.
Potentially, via the normalization of error signals!
e.g. Payeur et al. : nature.com/articles/s41...
Potentially, via the normalization of error signals!
e.g. Payeur et al. : nature.com/articles/s41...
March 19, 2026 at 3:27 PM
These findings complement several theories on how SST-subtypes targeting apical dendrites (where error-related signals arrive) are critical in supporting learning.
Potentially, via the normalization of error signals!
e.g. Payeur et al. : nature.com/articles/s41...
Potentially, via the normalization of error signals!
e.g. Payeur et al. : nature.com/articles/s41...
Hence this led us to believe if the brain uses inhibition for normalization, it must have a dedicated mechanism to normalize learning signals!
tldr;
In the end, we found that centering the gradients via lateral inhibition was enough to recapitulate performance on the task!
tldr;
In the end, we found that centering the gradients via lateral inhibition was enough to recapitulate performance on the task!
March 19, 2026 at 3:27 PM
Hence this led us to believe if the brain uses inhibition for normalization, it must have a dedicated mechanism to normalize learning signals!
tldr;
In the end, we found that centering the gradients via lateral inhibition was enough to recapitulate performance on the task!
tldr;
In the end, we found that centering the gradients via lateral inhibition was enough to recapitulate performance on the task!
Upon further inspection, we indeed noticed that despite our high alignment in neural activity, we had poor gradient alignment with layer norm.
March 19, 2026 at 3:27 PM
Upon further inspection, we indeed noticed that despite our high alignment in neural activity, we had poor gradient alignment with layer norm.
Why you may ask? It turns out that layer normalization actually implicitly normalizes back-propagated error signals "under the hood" !
March 19, 2026 at 3:27 PM
Why you may ask? It turns out that layer normalization actually implicitly normalizes back-propagated error signals "under the hood" !
Surprisingly!
Despite successfully normalizing feedforward neural activity, this didn't reflect in performance. As we were unable to recapitulate layer normalization's performance on the task.
Despite successfully normalizing feedforward neural activity, this didn't reflect in performance. As we were unable to recapitulate layer normalization's performance on the task.
March 19, 2026 at 3:27 PM
Surprisingly!
Despite successfully normalizing feedforward neural activity, this didn't reflect in performance. As we were unable to recapitulate layer normalization's performance on the task.
Despite successfully normalizing feedforward neural activity, this didn't reflect in performance. As we were unable to recapitulate layer normalization's performance on the task.
With our simple inhibitory loss function on our EI network, we were able to successfully *learn* to layer normalize neural activity!
March 19, 2026 at 3:27 PM
With our simple inhibitory loss function on our EI network, we were able to successfully *learn* to layer normalize neural activity!
Our network was trained on a robust perceptual invariance task: FashionMNIST with randomized brightness levels during both training and testing. (bounded by epsilon)
March 19, 2026 at 3:27 PM
Our network was trained on a robust perceptual invariance task: FashionMNIST with randomized brightness levels during both training and testing. (bounded by epsilon)
The brain is believed to recruit inhibitory neurons to perform normalization, but how does it actually help us learn?🤔
To find out, we built an ANN with separate excitatory and inhibitory neurons. And trained the E weights on the task and the I weights to do layer normalization
To find out, we built an ANN with separate excitatory and inhibitory neurons. And trained the E weights on the task and the I weights to do layer normalization
March 19, 2026 at 3:27 PM
The brain is believed to recruit inhibitory neurons to perform normalization, but how does it actually help us learn?🤔
To find out, we built an ANN with separate excitatory and inhibitory neurons. And trained the E weights on the task and the I weights to do layer normalization
To find out, we built an ANN with separate excitatory and inhibitory neurons. And trained the E weights on the task and the I weights to do layer normalization