Equivariance is dead! 😢
Or is it? 😈
Genie 3 is out! Our latest protein design model achieves SoTA results for binder design and motif scaffolding, greatly improving on BindCraft and Proteina-Complexa.
It does so using all-atom SE(3)-equivariance based on a branched polymer representation👇
Or is it? 😈
Genie 3 is out! Our latest protein design model achieves SoTA results for binder design and motif scaffolding, greatly improving on BindCraft and Proteina-Complexa.
It does so using all-atom SE(3)-equivariance based on a branched polymer representation👇
Introducing Genie 3, a generative protein model that substantially advances the state-of-the-art for binder design, increasing in silico success rates by up to 20x on hard multimeric targets. It also debuts a form of inference-time scaling unobserved in other design models. 🧵1/8
May 8, 2026 at 2:10 PM
Equivariance is dead! 😢
Or is it? 😈
Genie 3 is out! Our latest protein design model achieves SoTA results for binder design and motif scaffolding, greatly improving on BindCraft and Proteina-Complexa.
It does so using all-atom SE(3)-equivariance based on a branched polymer representation👇
Or is it? 😈
Genie 3 is out! Our latest protein design model achieves SoTA results for binder design and motif scaffolding, greatly improving on BindCraft and Proteina-Complexa.
It does so using all-atom SE(3)-equivariance based on a branched polymer representation👇
Excellent talk by @johannbrehmer.bsky.social
On “Does equivariance matter at scale?” At NeurReps workshop
arxiv.org/abs/2410.23179
www.neurreps.org
On “Does equivariance matter at scale?” At NeurReps workshop
arxiv.org/abs/2410.23179
www.neurreps.org
December 14, 2024 at 7:14 PM
Excellent talk by @johannbrehmer.bsky.social
On “Does equivariance matter at scale?” At NeurReps workshop
arxiv.org/abs/2410.23179
www.neurreps.org
On “Does equivariance matter at scale?” At NeurReps workshop
arxiv.org/abs/2410.23179
www.neurreps.org
"Does Equivariance Matter at Scale?"
Hear from @johannbrehmer.bsky.social, a physicist turned machine learner and a research scientist at CuspAI in Amsterdam.
Hear from @johannbrehmer.bsky.social, a physicist turned machine learner and a research scientist at CuspAI in Amsterdam.
December 14, 2024 at 7:03 PM
"Does Equivariance Matter at Scale?"
Hear from @johannbrehmer.bsky.social, a physicist turned machine learner and a research scientist at CuspAI in Amsterdam.
Hear from @johannbrehmer.bsky.social, a physicist turned machine learner and a research scientist at CuspAI in Amsterdam.
*Relaxed Equivariance via Multitask Learning*
by @tkrusch.bsky.social @mmbronstein.bsky.social
They propose a regularization approach for exploiting symmetries over data (penalizing variable predictions over augmented data).
arxiv.org/abs/2410.17878
by @tkrusch.bsky.social @mmbronstein.bsky.social
They propose a regularization approach for exploiting symmetries over data (penalizing variable predictions over augmented data).
arxiv.org/abs/2410.17878
December 13, 2024 at 2:28 PM
*Relaxed Equivariance via Multitask Learning*
by @tkrusch.bsky.social @mmbronstein.bsky.social
They propose a regularization approach for exploiting symmetries over data (penalizing variable predictions over augmented data).
arxiv.org/abs/2410.17878
by @tkrusch.bsky.social @mmbronstein.bsky.social
They propose a regularization approach for exploiting symmetries over data (penalizing variable predictions over augmented data).
arxiv.org/abs/2410.17878
Tomorrow, in the reading group, Johannes Brehmer will present his paper "Does equivariance matter at scale?" arxiv.org/abs/2410.23179
Join us on zoom at 12pm ET / 6pm CET: portal.valencelabs.com/logg
Join us on zoom at 12pm ET / 6pm CET: portal.valencelabs.com/logg
January 5, 2025 at 8:08 PM
Tomorrow, in the reading group, Johannes Brehmer will present his paper "Does equivariance matter at scale?" arxiv.org/abs/2410.23179
Join us on zoom at 12pm ET / 6pm CET: portal.valencelabs.com/logg
Join us on zoom at 12pm ET / 6pm CET: portal.valencelabs.com/logg
Ooh! New equivariance opportunities unlocked!
June 10, 2026 at 11:20 PM
Ooh! New equivariance opportunities unlocked!
Why do diffusion models generalise at all? It's not obvious that they would. It turns out underfitting plays an important role, as well as the architectural inductive biases of locality and translation equivariance. What other kinds of symmetry and structure could we hardcode? 🤔
Excited to finally share this work w/ @suryaganguli.bsky.social Tl;dr: we find the first closed-form analytical theory that replicates the outputs of the very simplest diffusion models, with median pixel wise r^2 values of 90%+. arxiv.org/abs/2412.20292
January 1, 2025 at 12:26 PM
Why do diffusion models generalise at all? It's not obvious that they would. It turns out underfitting plays an important role, as well as the architectural inductive biases of locality and translation equivariance. What other kinds of symmetry and structure could we hardcode? 🤔
My (I think funny) take on AlphaFold3 and equivariance:
There are many versions of AF3 which may be better at different tasks —
The one they publicly released is as diametrically opposite in terms of architecture (no equivariance, no frames, generative) to RosettaFold-AllAtom.
There are many versions of AF3 which may be better at different tasks —
The one they publicly released is as diametrically opposite in terms of architecture (no equivariance, no frames, generative) to RosettaFold-AllAtom.
December 12, 2024 at 3:48 AM
My (I think funny) take on AlphaFold3 and equivariance:
There are many versions of AF3 which may be better at different tasks —
The one they publicly released is as diametrically opposite in terms of architecture (no equivariance, no frames, generative) to RosettaFold-AllAtom.
There are many versions of AF3 which may be better at different tasks —
The one they publicly released is as diametrically opposite in terms of architecture (no equivariance, no frames, generative) to RosettaFold-AllAtom.
Repost from Twitter from 2020, before ViTs: I trained a CNN w/o pooling on MNIST and produced a colored visualization of the feature maps, translating a digit on an input canvas. This illustrates the equivariance property of convolutions.
More on why:
chriswolfvision.medium.com/what-is-tran...
More on why:
chriswolfvision.medium.com/what-is-tran...
January 20, 2025 at 10:30 AM
Repost from Twitter from 2020, before ViTs: I trained a CNN w/o pooling on MNIST and produced a colored visualization of the feature maps, translating a digit on an input canvas. This illustrates the equivariance property of convolutions.
More on why:
chriswolfvision.medium.com/what-is-tran...
More on why:
chriswolfvision.medium.com/what-is-tran...
Genie3: all-atom SE3-equivariance for fast and performant protein design.
@yeqinglin.bsky.social @moalquraishi.bsky.social
www.biorxiv.org/content/10.6...
@yeqinglin.bsky.social @moalquraishi.bsky.social
www.biorxiv.org/content/10.6...
May 11, 2026 at 9:42 PM
Genie3: all-atom SE3-equivariance for fast and performant protein design.
@yeqinglin.bsky.social @moalquraishi.bsky.social
www.biorxiv.org/content/10.6...
@yeqinglin.bsky.social @moalquraishi.bsky.social
www.biorxiv.org/content/10.6...
I've been around the block a few times. When deep learning first became hot, many older colleagues bemoaned it as just tinkering + chain rule, and not intellectually satisfying. Then came SSL, equivariance, VAEs, GANs, neural ODEs, transformers, diffusion, etc. The richness was staggering.
🧵👇
🧵👇
Great post that captures the tension between classic ML approaches and modern deep learning while acknowledging the nuances of both.
“Working with LLMs doesn’t feel the same. It’s like fitting pieces into a pre-defined puzzle instead of building the puzzle itself.”
www.reddit.com/r/MachineLea...
“Working with LLMs doesn’t feel the same. It’s like fitting pieces into a pre-defined puzzle instead of building the puzzle itself.”
www.reddit.com/r/MachineLea...
From the MachineLearning community on Reddit
Explore this post and more from the MachineLearning community
www.reddit.com
December 6, 2024 at 5:06 PM
I've been around the block a few times. When deep learning first became hot, many older colleagues bemoaned it as just tinkering + chain rule, and not intellectually satisfying. Then came SSL, equivariance, VAEs, GANs, neural ODEs, transformers, diffusion, etc. The richness was staggering.
🧵👇
🧵👇
Yesterday, @erikjbekkers.bsky.social presented his vision on equivariance to IvI (at UvA), showcasing recent work on geometry-grounded representation learning - addressing fundamental limitations in geometric reasoning of current AI systems. 🤖
Exciting times ahead for geometric deep learning! 🌐 🤩
Exciting times ahead for geometric deep learning! 🌐 🤩
November 19, 2024 at 4:11 PM
Yesterday, @erikjbekkers.bsky.social presented his vision on equivariance to IvI (at UvA), showcasing recent work on geometry-grounded representation learning - addressing fundamental limitations in geometric reasoning of current AI systems. 🤖
Exciting times ahead for geometric deep learning! 🌐 🤩
Exciting times ahead for geometric deep learning! 🌐 🤩
But did you mention translation equivariance? That would have changed everything for her!
August 12, 2026 at 6:32 PM
But did you mention translation equivariance? That would have changed everything for her!
The snow is gently falling outside the window, the models are training, what could be better? Two articles cool to read:
Does Equivariance matter at scale? (@johannbrehmer.bsky.social et al.) arxiv.org/abs/2410.23179
Denoising Diffusion Bridge Models (Linqi Zhou et al.) arxiv.org/pdf/2309.16948
Does Equivariance matter at scale? (@johannbrehmer.bsky.social et al.) arxiv.org/abs/2410.23179
Denoising Diffusion Bridge Models (Linqi Zhou et al.) arxiv.org/pdf/2309.16948
November 21, 2024 at 12:48 PM
The snow is gently falling outside the window, the models are training, what could be better? Two articles cool to read:
Does Equivariance matter at scale? (@johannbrehmer.bsky.social et al.) arxiv.org/abs/2410.23179
Denoising Diffusion Bridge Models (Linqi Zhou et al.) arxiv.org/pdf/2309.16948
Does Equivariance matter at scale? (@johannbrehmer.bsky.social et al.) arxiv.org/abs/2410.23179
Denoising Diffusion Bridge Models (Linqi Zhou et al.) arxiv.org/pdf/2309.16948
We're having a fun time with equivariance and modern vision architectures.
Want stronger Vision Transformers? Use octic-equivariant layers (arxiv.org/abs/2505.15441).
TLDR; We extend @bokmangeorg.bsky.social's reflection-equivariant ViTs to the (octic) group of 90-degree rotations and reflections and... it just works... (DINOv2+DeiT)
Code: github.com/davnords/octic-vits
TLDR; We extend @bokmangeorg.bsky.social's reflection-equivariant ViTs to the (octic) group of 90-degree rotations and reflections and... it just works... (DINOv2+DeiT)
Code: github.com/davnords/octic-vits
May 23, 2025 at 4:17 PM
We're having a fun time with equivariance and modern vision architectures.
“Dear mathematician, can you explain to me what mass is?”
“Very simple, dear: the mass of a dynamical system is the cohomology class of the Galilean group representing lack of equivariance of the moment map on the symplectic manifold that is the phase space of the system.”
“Very simple, dear: the mass of a dynamical system is the cohomology class of the Galilean group representing lack of equivariance of the moment map on the symplectic manifold that is the phase space of the system.”
November 10, 2025 at 1:40 PM
“Dear mathematician, can you explain to me what mass is?”
“Very simple, dear: the mass of a dynamical system is the cohomology class of the Galilean group representing lack of equivariance of the moment map on the symplectic manifold that is the phase space of the system.”
“Very simple, dear: the mass of a dynamical system is the cohomology class of the Galilean group representing lack of equivariance of the moment map on the symplectic manifold that is the phase space of the system.”
Equivariance, damn you autocorrect!!
April 17, 2026 at 9:01 PM
Equivariance, damn you autocorrect!!
Take-away: Don't give up 🙃
December 14, 2024 at 7:44 PM
Take-away: Don't give up 🙃
Lawful evil commutative diagram
March 6, 2025 at 7:08 PM
Lawful evil commutative diagram
Tomáš Karella presents their work on "Harmformer: Harmonic Networks Meet Transformers for Continuous Roto-Translation Equivariance"!
December 14, 2024 at 7:26 PM
Tomáš Karella presents their work on "Harmformer: Harmonic Networks Meet Transformers for Continuous Roto-Translation Equivariance"!
Nice study on scaling and equivariance. arxiv.org/abs/2510.09768
Scaling Laws and Symmetry, Evidence from Neural Force Fields
We present an empirical study in the geometric task of learning interatomic potentials, which shows equivariance matters even more at larger scales; we show a clear power-law scaling behaviour with re...
arxiv.org
October 14, 2025 at 9:20 AM
Nice study on scaling and equivariance. arxiv.org/abs/2510.09768
"Relaxed equivariance" "E(3)" "message passing" "SO(2) symmetry breaking" You jolt awake. You are Larry Roberts and the year is 1969. The future can not come to pass. ARPANET must be destroyed.
March 2, 2025 at 6:05 PM
"Relaxed equivariance" "E(3)" "message passing" "SO(2) symmetry breaking" You jolt awake. You are Larry Roberts and the year is 1969. The future can not come to pass. ARPANET must be destroyed.
In our recent preprint, we show in more general settings how identifiability of a network means that equivariance implies layerwise equivariance. arxiv.org/pdf/2601.21645
April 1, 2026 at 11:45 AM
In our recent preprint, we show in more general settings how identifiability of a network means that equivariance implies layerwise equivariance. arxiv.org/pdf/2601.21645
After a long hiatus, I've started blogging again! My first post was a difficult one to write, because I don't want to keep repeating what's already in papers.
chaitjo.substack.com/p/transforme...
chaitjo.substack.com/p/transforme...
Equivariance is dead, long live equivariance?
When should you bake symmetries into your architecture versus just scaling up — an attempt at a nuanced take.
chaitjo.substack.com
June 1, 2025 at 10:57 AM
After a long hiatus, I've started blogging again! My first post was a difficult one to write, because I don't want to keep repeating what's already in papers.
chaitjo.substack.com/p/transforme...
chaitjo.substack.com/p/transforme...
Equivariance is dead - long live equivariance? Scaling pure Transformers is the future of molecular generative models?
I think its a lot more nuanced -- I tried to give some perspectives in my spotlight talk at #ICLR2025 AI4Mat Workshop last week
Full video here: youtu.be/NiY4NLzemnU?...
I think its a lot more nuanced -- I tried to give some perspectives in my spotlight talk at #ICLR2025 AI4Mat Workshop last week
Full video here: youtu.be/NiY4NLzemnU?...
All-atom Diffusion Transformers - Chaitanya K. Joshi - ICLR 2025 AI4Mat Spotlight
YouTube video by Chaitanya K. Joshi
youtu.be
May 5, 2025 at 9:44 AM
Equivariance is dead - long live equivariance? Scaling pure Transformers is the future of molecular generative models?
I think its a lot more nuanced -- I tried to give some perspectives in my spotlight talk at #ICLR2025 AI4Mat Workshop last week
Full video here: youtu.be/NiY4NLzemnU?...
I think its a lot more nuanced -- I tried to give some perspectives in my spotlight talk at #ICLR2025 AI4Mat Workshop last week
Full video here: youtu.be/NiY4NLzemnU?...