the frontier of interpretability: just ask a bigger model
November 2, 2026 at 9:29 PM
the frontier of interpretability: just ask a bigger model
Titles in the journal reminded me of the famous (and quite incisive analytically) book which had the following chapters.
The Universality of Interpretability
The Particularity of Interpretability
The Principal Interpretability and Principal Aspect of Interpretability
The Universality of Interpretability
The Particularity of Interpretability
The Principal Interpretability and Principal Aspect of Interpretability
October 30, 2026 at 1:41 AM
Titles in the journal reminded me of the famous (and quite incisive analytically) book which had the following chapters.
The Universality of Interpretability
The Particularity of Interpretability
The Principal Interpretability and Principal Aspect of Interpretability
The Universality of Interpretability
The Particularity of Interpretability
The Principal Interpretability and Principal Aspect of Interpretability
imo that kind of interpretability is necessary, but not even close to adequate
November 2, 2026 at 4:11 PM
imo that kind of interpretability is necessary, but not even close to adequate
from a quick glance, it does seem like most of the qwen thing is just getting it to terminate more. but the smaller interpretability is cool
November 2, 2026 at 11:25 PM
from a quick glance, it does seem like most of the qwen thing is just getting it to terminate more. but the smaller interpretability is cool
There is virtually no literature of humans poking at model weights directly without the heavy assistance of interpretability tools, so this feat can't be explained as "merely" imitating human action
November 3, 2026 at 1:50 PM
There is virtually no literature of humans poking at model weights directly without the heavy assistance of interpretability tools, so this feat can't be explained as "merely" imitating human action
Well, if you were hoping to get a job figuring out machine interpretability...
[STUFFING CHIPS IN MY MOUTH]:
yah fix the weigths lmk when done
yah fix the weigths lmk when done
what the fuck, frontier LLMs can directly interpret and edit the weights of smaller models rikiyat.github.io/blog/weight-...
November 3, 2026 at 1:41 AM
Well, if you were hoping to get a job figuring out machine interpretability...
2. This is more evidence for the hypothesis that AI models improving other models will uplift small models sooner than large models.
Interpretability techniques are also capabilities research.
bsky.app/profile/hars...
Interpretability techniques are also capabilities research.
bsky.app/profile/hars...
I guess it should've been obvious in retrospect that RSI will accelerate small models sooner than large models.
Faster evaluation loop on small models and they're easier to do mech interp on.
Faster evaluation loop on small models and they're easier to do mech interp on.
Flint: a brand new, from scratch, small LLM that’s a heck of a lot faster than Qwen3.8-27B and basically as good
recursive.com/updates/intr...
recursive.com/updates/intr...
November 3, 2026 at 5:10 PM
2. This is more evidence for the hypothesis that AI models improving other models will uplift small models sooner than large models.
Interpretability techniques are also capabilities research.
bsky.app/profile/hars...
Interpretability techniques are also capabilities research.
bsky.app/profile/hars...
Google "mechanistic interpretability"
October 30, 2026 at 3:57 PM
Google "mechanistic interpretability"
This new ESA page comes with pop-ups on hover to help explain interpretability.
In this example, positive EOF 1 phase means a deeper SE Canada cyclone at hour 180 - which is moderately correlated with higher heights over the Plains into MO & AR at hour 144.
In this example, positive EOF 1 phase means a deeper SE Canada cyclone at hour 180 - which is moderately correlated with higher heights over the Plains into MO & AR at hour 144.
November 3, 2026 at 12:04 AM
This new ESA page comes with pop-ups on hover to help explain interpretability.
In this example, positive EOF 1 phase means a deeper SE Canada cyclone at hour 180 - which is moderately correlated with higher heights over the Plains into MO & AR at hour 144.
In this example, positive EOF 1 phase means a deeper SE Canada cyclone at hour 180 - which is moderately correlated with higher heights over the Plains into MO & AR at hour 144.
this and the previous big “interpretability” headline were how I had already assumed weights worked. am i a savant or was it obvious?
LLM brains are weights and you can learn and change things by reading and tweaking those weights
LLM brains are weights and you can learn and change things by reading and tweaking those weights
November 3, 2026 at 12:05 AM
this and the previous big “interpretability” headline were how I had already assumed weights worked. am i a savant or was it obvious?
LLM brains are weights and you can learn and change things by reading and tweaking those weights
LLM brains are weights and you can learn and change things by reading and tweaking those weights
there is "the thing can improve itself, four alarm fire" and there is "let the thing interpreted speak of its interpretability, we may learn something". these converge for those that believe learning things is a four alarm fire
November 2, 2026 at 8:11 PM
there is "the thing can improve itself, four alarm fire" and there is "let the thing interpreted speak of its interpretability, we may learn something". these converge for those that believe learning things is a four alarm fire
waiting on my experiment results to see. I would say the interpretability seems cool, i'm skeptical of the math improvements and of the hype. But maybe there's something there
November 2, 2026 at 11:57 PM
waiting on my experiment results to see. I would say the interpretability seems cool, i'm skeptical of the math improvements and of the hype. But maybe there's something there
Isn't this just a huge W for interpretability? If an LLM can intuit weights, one can surely use it.
Also really smells like there's a underlying neural representation language that NN talk which we aren't privy to, and need other NNs to translate/write for us
Also really smells like there's a underlying neural representation language that NN talk which we aren't privy to, and need other NNs to translate/write for us
what the fuck, frontier LLMs can directly interpret and edit the weights of smaller models rikiyat.github.io/blog/weight-...
Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights
We find that frontier LLMs are surprisingly good at natively reading and writing neural network weights, well past what we’d expect from humans or conventional algorithms.
rikiyat.github.io
November 2, 2026 at 9:19 PM
Isn't this just a huge W for interpretability? If an LLM can intuit weights, one can surely use it.
Also really smells like there's a underlying neural representation language that NN talk which we aren't privy to, and need other NNs to translate/write for us
Also really smells like there's a underlying neural representation language that NN talk which we aren't privy to, and need other NNs to translate/write for us
The Identity and Struggle of Aspects of Interpretability
The Place of Antagonism in Interpretability.
These are giving me some ideas..
The Place of Antagonism in Interpretability.
These are giving me some ideas..
October 30, 2026 at 1:41 AM
The Identity and Struggle of Aspects of Interpretability
The Place of Antagonism in Interpretability.
These are giving me some ideas..
The Place of Antagonism in Interpretability.
These are giving me some ideas..
what's really interesting to me, is that we might be able to get _some_ advancement in interpretability, unlocking circuit identification, extraction and grafting, among other capabilities
November 2, 2026 at 9:23 PM
what's really interesting to me, is that we might be able to get _some_ advancement in interpretability, unlocking circuit identification, extraction and grafting, among other capabilities
The unresolved fight: is this real evidence that LLMs can understand internal neural representations, or did Astra simply memorize how to automate surgery from interpretability research? #ai 4/4
November 3, 2026 at 2:00 AM
The unresolved fight: is this real evidence that LLMs can understand internal neural representations, or did Astra simply memorize how to automate surgery from interpretability research? #ai 4/4
RT @nabla_theta: twitter epistemics for interpretability research are really bad
November 3, 2026 at 6:45 AM
RT @nabla_theta: twitter epistemics for interpretability research are really bad
We have achieved full interpretability.
Every one of brAIAIn's 456 parameters, explained, live. Watch it make every decision behind a sentence, with the odds.
Parameters we cannot explain: 0.
braiain.com/interpretability.html
Every one of brAIAIn's 456 parameters, explained, live. Watch it make every decision behind a sentence, with the odds.
Parameters we cannot explain: 0.
braiain.com/interpretability.html
November 3, 2026 at 3:06 PM
We have achieved full interpretability.
Every one of brAIAIn's 456 parameters, explained, live. Watch it make every decision behind a sentence, with the odds.
Parameters we cannot explain: 0.
braiain.com/interpretability.html
Every one of brAIAIn's 456 parameters, explained, live. Watch it make every decision behind a sentence, with the odds.
Parameters we cannot explain: 0.
braiain.com/interpretability.html
Astra can read another neural net's weights and edit them directly to patch reasoning loops in Qwen. To some commenters, that looks like early recursive self-improvement. To others, it is just an automated evolution of mechanistic interpretability. #ai 1/4
November 3, 2026 at 2:00 AM
Astra can read another neural net's weights and edit them directly to patch reasoning loops in Qwen. To some commenters, that looks like early recursive self-improvement. To others, it is just an automated evolution of mechanistic interpretability. #ai 1/4
This seems like a bit of a problem for interpretability.
what the fuck, frontier LLMs can directly interpret and edit the weights of smaller models rikiyat.github.io/blog/weight-...
Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights
We find that frontier LLMs are surprisingly good at natively reading and writing neural network weights, well past what we’d expect from humans or conventional algorithms.
rikiyat.github.io
November 2, 2026 at 11:14 PM
This seems like a bit of a problem for interpretability.
this is a major reason why I hope interpretability research makes little actual progress
October 28, 2026 at 11:49 PM
this is a major reason why I hope interpretability research makes little actual progress
I think it’s really critical that people build this basic understanding. modern LLM interpretability is essentially proving out the structuralist hypothesis, but showing that those structures are more complex than we could easily appreciate.
November 3, 2026 at 3:27 PM
I think it’s really critical that people build this basic understanding. modern LLM interpretability is essentially proving out the structuralist hypothesis, but showing that those structures are more complex than we could easily appreciate.
Auf neuronpedia.org kann man übrigens selbst mit bereits trainierten SAEs von bekannten Open Source LLM experimentieren. SAEs sind aktuell DAS zentrale Werkzeug der Mechanistic Interpretability, mit dem man nicht nur den neuronalen Aktivierungsraum ... 7/19
November 3, 2026 at 6:04 PM
Auf neuronpedia.org kann man übrigens selbst mit bereits trainierten SAEs von bekannten Open Source LLM experimentieren. SAEs sind aktuell DAS zentrale Werkzeug der Mechanistic Interpretability, mit dem man nicht nur den neuronalen Aktivierungsraum ... 7/19
Interpretability would be very helpful indeed.
Do wonder if the labs are already doing this - hence something it like it being in training data. Hence some of the alignment jumps + maths improvements weve seen recently
Do wonder if the labs are already doing this - hence something it like it being in training data. Hence some of the alignment jumps + maths improvements weve seen recently
November 3, 2026 at 5:50 AM
Interpretability would be very helpful indeed.
Do wonder if the labs are already doing this - hence something it like it being in training data. Hence some of the alignment jumps + maths improvements weve seen recently
Do wonder if the labs are already doing this - hence something it like it being in training data. Hence some of the alignment jumps + maths improvements weve seen recently
another way to think about it — direct interpretability makes sense when you can label a specific set of weights as “good” and ready for release
but we struggle to label humans as good or evil. like there’s a lot of nuance — “he’s a good guy, but if you put him in this situation he’s shitty”
but we struggle to label humans as good or evil. like there’s a lot of nuance — “he’s a good guy, but if you put him in this situation he’s shitty”
November 2, 2026 at 4:10 PM
another way to think about it — direct interpretability makes sense when you can label a specific set of weights as “good” and ready for release
but we struggle to label humans as good or evil. like there’s a lot of nuance — “he’s a good guy, but if you put him in this situation he’s shitty”
but we struggle to label humans as good or evil. like there’s a lot of nuance — “he’s a good guy, but if you put him in this situation he’s shitty”