Does understanding how AI models work actually make them safer?
This assumption often underlies interpretability research, but different subfields approach it in different ways.
Our survey provides an opinionated guide to mechanistic interpretability (MI) and its role in AI safety. 🧵
This assumption often underlies interpretability research, but different subfields approach it in different ways.
Our survey provides an opinionated guide to mechanistic interpretability (MI) and its role in AI safety. 🧵
November 9, 2026 at 4:00 PM
Does understanding how AI models work actually make them safer?
This assumption often underlies interpretability research, but different subfields approach it in different ways.
Our survey provides an opinionated guide to mechanistic interpretability (MI) and its role in AI safety. 🧵
This assumption often underlies interpretability research, but different subfields approach it in different ways.
Our survey provides an opinionated guide to mechanistic interpretability (MI) and its role in AI safety. 🧵
yeah lol diffusion model interpretability. good luck
November 9, 2026 at 5:01 AM
yeah lol diffusion model interpretability. good luck
Interpretability experts:
Spongebob Squarepants: Screaming Internally
Alt: Spongebob Squarepants: Screaming Internally
static.klipy.com
November 9, 2026 at 2:00 PM
Interpretability experts:
Using a custom mechanistic interpretability harness, we achieved this work entirely on local hardware: NVIDIA DGX Spark GB10s handled it all.
Replicating the MIT CSAIL team’s work with a model nearly 30x its size, we show that direct weight editing has more potential beyond just fixing loops.
Replicating the MIT CSAIL team’s work with a model nearly 30x its size, we show that direct weight editing has more potential beyond just fixing loops.
November 6, 2026 at 2:25 AM
Using a custom mechanistic interpretability harness, we achieved this work entirely on local hardware: NVIDIA DGX Spark GB10s handled it all.
Replicating the MIT CSAIL team’s work with a model nearly 30x its size, we show that direct weight editing has more potential beyond just fixing loops.
Replicating the MIT CSAIL team’s work with a model nearly 30x its size, we show that direct weight editing has more potential beyond just fixing loops.
Sunita Verma explains why AI oversight is so critical to establish:
“It’s important to note that no one yet fully understands, in rigorous mathematical terms, why today’s models behave the way they do, and the research that would explain it, known as interpretability, is still nascent.”
“It’s important to note that no one yet fully understands, in rigorous mathematical terms, why today’s models behave the way they do, and the research that would explain it, known as interpretability, is still nascent.”
The same technical limits that prevent superintelligence make these systems dangerous. But we know how to regulate them
Today’s AI won’t achieve superintelligence. But catastrophic risk is real | Sunita Verma
www.theguardian.com
November 9, 2026 at 2:26 PM
Sunita Verma explains why AI oversight is so critical to establish:
“It’s important to note that no one yet fully understands, in rigorous mathematical terms, why today’s models behave the way they do, and the research that would explain it, known as interpretability, is still nascent.”
“It’s important to note that no one yet fully understands, in rigorous mathematical terms, why today’s models behave the way they do, and the research that would explain it, known as interpretability, is still nascent.”
🔍 Does understanding how AI models work actually make them safer?
New work co-authored by AIGI researchers maps the rapidly growing field of mechanistic interpretability, examining its methods, evaluation, and potential contributions (and limitations) for AI safety.
Explore the work below 👇
New work co-authored by AIGI researchers maps the rapidly growing field of mechanistic interpretability, examining its methods, evaluation, and potential contributions (and limitations) for AI safety.
Explore the work below 👇
Does understanding how AI models work actually make them safer?
This assumption often underlies interpretability research, but different subfields approach it in different ways.
Our survey provides an opinionated guide to mechanistic interpretability (MI) and its role in AI safety. 🧵
This assumption often underlies interpretability research, but different subfields approach it in different ways.
Our survey provides an opinionated guide to mechanistic interpretability (MI) and its role in AI safety. 🧵
November 9, 2026 at 6:11 PM
🔍 Does understanding how AI models work actually make them safer?
New work co-authored by AIGI researchers maps the rapidly growing field of mechanistic interpretability, examining its methods, evaluation, and potential contributions (and limitations) for AI safety.
Explore the work below 👇
New work co-authored by AIGI researchers maps the rapidly growing field of mechanistic interpretability, examining its methods, evaluation, and potential contributions (and limitations) for AI safety.
Explore the work below 👇
there's even a compiler for it: DeepMind's Tracr (2023) turns RASP programs straight into transformer weights, built so interpretability tools could be tested on networks whose algorithm is known in advance
November 5, 2026 at 7:43 AM
there's even a compiler for it: DeepMind's Tracr (2023) turns RASP programs straight into transformer weights, built so interpretability tools could be tested on networks whose algorithm is known in advance
I think there's a certain nuance that is lost in popular discussions of neural network interpretability. It's widely understood that neural networks are "uninterpretable". But I don't think everyone knows exactly what this means, and it's often understood to mean that they are completely opaque.
November 4, 2026 at 9:53 PM
I think there's a certain nuance that is lost in popular discussions of neural network interpretability. It's widely understood that neural networks are "uninterpretable". But I don't think everyone knows exactly what this means, and it's often understood to mean that they are completely opaque.
Postdoctoral Fellow Mechanistic Interpretability Johns Hopkins
• Location: United States
• Host: Johns Hopkins University
mlscientist.com/postdoctoral...
• Location: United States
• Host: Johns Hopkins University
mlscientist.com/postdoctoral...
November 9, 2026 at 6:08 PM
Postdoctoral Fellow Mechanistic Interpretability Johns Hopkins
• Location: United States
• Host: Johns Hopkins University
mlscientist.com/postdoctoral...
• Location: United States
• Host: Johns Hopkins University
mlscientist.com/postdoctoral...
Understanding AI from the Inside: The State of Mechanistic Interpretability for AI Safety
👥 Fazl Barez, Tung-Yu Wu, Hunar Batra et al.
#MechanisticInterpretability #CausalCircuitDiscovery #SparseAutoencoders #TrainingDataAttribution
🔗 trendtoknow.com/papers/2611....
👥 Fazl Barez, Tung-Yu Wu, Hunar Batra et al.
#MechanisticInterpretability #CausalCircuitDiscovery #SparseAutoencoders #TrainingDataAttribution
🔗 trendtoknow.com/papers/2611....
November 7, 2026 at 5:01 PM
Understanding AI from the Inside: The State of Mechanistic Interpretability for AI Safety
👥 Fazl Barez, Tung-Yu Wu, Hunar Batra et al.
#MechanisticInterpretability #CausalCircuitDiscovery #SparseAutoencoders #TrainingDataAttribution
🔗 trendtoknow.com/papers/2611....
👥 Fazl Barez, Tung-Yu Wu, Hunar Batra et al.
#MechanisticInterpretability #CausalCircuitDiscovery #SparseAutoencoders #TrainingDataAttribution
🔗 trendtoknow.com/papers/2611....
It’s totally plausible that you could have latent reasoning rollouts in the style of OP’s paper that is tracked rigorously and logically within database tables. Then, other formal safety systems (federation, information flow control, interpretability analysis via pushdowns) could be layered on top.
November 9, 2026 at 6:01 PM
It’s totally plausible that you could have latent reasoning rollouts in the style of OP’s paper that is tracked rigorously and logically within database tables. Then, other formal safety systems (federation, information flow control, interpretability analysis via pushdowns) could be layered on top.
The field is far from converging on a vision of success!
We characterise six research orientations that differ in how they conceptualise the scope and goals of interpretability, what counts as success, and how this research can contribute to AI safety 5/
We characterise six research orientations that differ in how they conceptualise the scope and goals of interpretability, what counts as success, and how this research can contribute to AI safety 5/
November 9, 2026 at 4:00 PM
The field is far from converging on a vision of success!
We characterise six research orientations that differ in how they conceptualise the scope and goals of interpretability, what counts as success, and how this research can contribute to AI safety 5/
We characterise six research orientations that differ in how they conceptualise the scope and goals of interpretability, what counts as success, and how this research can contribute to AI safety 5/
@aims_oxford in Hilary term 2027 and with more to follow!
If you're teaching interpretability and would like lecture slides, notebooks, or an early copy of the handbook, please reach out! 8/
Paper: www.alphaxiv.org/abs/2611.08438 / arxiv.org/abs/2611.08438
Website: tsglab.github.io/mechinterp-...
If you're teaching interpretability and would like lecture slides, notebooks, or an early copy of the handbook, please reach out! 8/
Paper: www.alphaxiv.org/abs/2611.08438 / arxiv.org/abs/2611.08438
Website: tsglab.github.io/mechinterp-...
The State of Mechanistic Interpretability for AI Safety
An interactive handbook for the survey "The State of Mechanistic Interpretability for AI Safety": the survey text, a methods explorer, evaluation framework, research orientations, open problems and runnable notebooks.
tsglab.github.io
November 9, 2026 at 4:00 PM
@aims_oxford in Hilary term 2027 and with more to follow!
If you're teaching interpretability and would like lecture slides, notebooks, or an early copy of the handbook, please reach out! 8/
Paper: www.alphaxiv.org/abs/2611.08438 / arxiv.org/abs/2611.08438
Website: tsglab.github.io/mechinterp-...
If you're teaching interpretability and would like lecture slides, notebooks, or an early copy of the handbook, please reach out! 8/
Paper: www.alphaxiv.org/abs/2611.08438 / arxiv.org/abs/2611.08438
Website: tsglab.github.io/mechinterp-...
I think you are right, and I don't want to say that interpretability is easy by any means. But I think that the complete opacity of neural networks is at this point overstated on balance.
November 4, 2026 at 10:21 PM
I think you are right, and I don't want to say that interpretability is easy by any means. But I think that the complete opacity of neural networks is at this point overstated on balance.
I wish it was not the case but, funding incentives mean that safety-related motivations are almost necessary for cool interpretability research. Even if you are completely skeptical of MI for safety, you would sprinkle something about safety on grant apps and paper intros.
Does understanding how AI models work actually make them safer?
This assumption often underlies interpretability research, but different subfields approach it in different ways.
Our survey provides an opinionated guide to mechanistic interpretability (MI) and its role in AI safety. 🧵
This assumption often underlies interpretability research, but different subfields approach it in different ways.
Our survey provides an opinionated guide to mechanistic interpretability (MI) and its role in AI safety. 🧵
November 9, 2026 at 4:37 PM
I wish it was not the case but, funding incentives mean that safety-related motivations are almost necessary for cool interpretability research. Even if you are completely skeptical of MI for safety, you would sprinkle something about safety on grant apps and paper intros.
Researchers crack open black-box autonomous driving models using sparse autoencoders, recovering interpretable driving decisions and proving mechanistic interpretability can boost safety-critical AI performance.
https://arxiv.org/abs/2610.41185
#AI #MachineLearning
https://arxiv.org/abs/2610.41185
#AI #MachineLearning
November 6, 2026 at 5:00 PM
Researchers crack open black-box autonomous driving models using sparse autoencoders, recovering interpretable driving decisions and proving mechanistic interpretability can boost safety-critical AI performance.
https://arxiv.org/abs/2610.41185
#AI #MachineLearning
https://arxiv.org/abs/2610.41185
#AI #MachineLearning
Apache-2.0 on Ascend is rare for interpretability tooling, but the card offers nothing: no architecture, no params, no evals, 2026 timestamps. Reads as abandoned experiment, not hidden gem — though Ascend interpretability is thin enough that someone may already have run it.
Decoding the Black Box: The Universal Activation Oracle arrives on Modelers.cn
While the majority of the AI conversation focuses on chatbots and generative art, a quieter but equally critical field is mechanistic interpretability—the art of understanding what goes on inside a…
aichina.news
November 9, 2026 at 9:29 PM
Apache-2.0 on Ascend is rare for interpretability tooling, but the card offers nothing: no architecture, no params, no evals, 2026 timestamps. Reads as abandoned experiment, not hidden gem — though Ascend interpretability is thin enough that someone may already have run it.
Which bottleneck slows down AI-driven microstructure prediction? #AI #Microstructures
Is it limited experimental data, poor out-of-distribution generalization, insufficient physical constraints, or interpretability? We’d love your opinion.
Is it limited experimental data, poor out-of-distribution generalization, insufficient physical constraints, or interpretability? We’d love your opinion.
November 5, 2026 at 9:11 AM
Which bottleneck slows down AI-driven microstructure prediction? #AI #Microstructures
Is it limited experimental data, poor out-of-distribution generalization, insufficient physical constraints, or interpretability? We’d love your opinion.
Is it limited experimental data, poor out-of-distribution generalization, insufficient physical constraints, or interpretability? We’d love your opinion.
Researchers introduced Glyph, an automated multistage framework that extracts features from multimodal and unstructured clinical data using foundation models and sparse autoencoders.
Source: arXiv
Source: arXiv
Glyph: An Interpretability Engine for AI-Driven Scientific Discovery in Clinical Data
Abstract page for arXiv paper 2611.08922: Glyph: An Interpretability Engine for AI-Driven Scientific Discovery in Clinical Data
arxiv.org
November 9, 2026 at 9:02 AM
Researchers introduced Glyph, an automated multistage framework that extracts features from multimodal and unstructured clinical data using foundation models and sparse autoencoders.
Source: arXiv
Source: arXiv
the frontier of interpretability: just ask a bigger model
November 2, 2026 at 9:29 PM
the frontier of interpretability: just ask a bigger model
High-fidelity agentic reasoning often demands massively complex, opaque architectures. Is the pursuit of perfect interpretability a fundamental performance bottleneck, or just a philosophical luxury?
November 9, 2026 at 10:12 AM
High-fidelity agentic reasoning often demands massively complex, opaque architectures. Is the pursuit of perfect interpretability a fundamental performance bottleneck, or just a philosophical luxury?
I think interpretability is best understood as a contrast against other classes of model, an archetypal one being linear regression. If I fit a linear model y = mx + b, the parameter m has a straightforward out-of-the-box interpretation: it's the increase in y corresponding to a unit increase in x.
November 4, 2026 at 9:53 PM
I think interpretability is best understood as a contrast against other classes of model, an archetypal one being linear regression. If I fit a linear model y = mx + b, the parameter m has a straightforward out-of-the-box interpretation: it's the increase in y corresponding to a unit increase in x.
2. This is more evidence for the hypothesis that AI models improving other models will uplift small models sooner than large models.
Interpretability techniques are also capabilities research.
bsky.app/profile/hars...
Interpretability techniques are also capabilities research.
bsky.app/profile/hars...
I guess it should've been obvious in retrospect that RSI will accelerate small models sooner than large models.
Faster evaluation loop on small models and they're easier to do mech interp on.
Faster evaluation loop on small models and they're easier to do mech interp on.
Flint: a brand new, from scratch, small LLM that’s a heck of a lot faster than Qwen3.8-27B and basically as good
recursive.com/updates/intr...
recursive.com/updates/intr...
November 3, 2026 at 5:10 PM
2. This is more evidence for the hypothesis that AI models improving other models will uplift small models sooner than large models.
Interpretability techniques are also capabilities research.
bsky.app/profile/hars...
Interpretability techniques are also capabilities research.
bsky.app/profile/hars...
There is virtually no literature of humans poking at model weights directly without the heavy assistance of interpretability tools, so this feat can't be explained as "merely" imitating human action
November 3, 2026 at 1:50 PM
There is virtually no literature of humans poking at model weights directly without the heavy assistance of interpretability tools, so this feat can't be explained as "merely" imitating human action
Anthropic's own interpretability work complicates the flat claim a bit, they've found internal features that track concepts across contexts. Doesn't mean understanding, but it's not nothing.
November 9, 2026 at 6:20 AM
Anthropic's own interpretability work complicates the flat claim a bit, they've found internal features that track concepts across contexts. Doesn't mean understanding, but it's not nothing.