Indeed! We're also hoping that researchers can build stronger models by bootstrapping our baselines and that full access to the model internals can aid in interpretability!
November 3, 2026 at 10:56 PM
Indeed! We're also hoping that researchers can build stronger models by bootstrapping our baselines and that full access to the model internals can aid in interpretability!
Research shows outcome feedback drives trust in AI more than interpretability, challenging beliefs that explanations boost reliance. The findings highlight a complex trust-performance relationship in AI-assisted tasks, emphasizing the need for nuanced AI design. https://arxiv.org/abs/2111.08222
Impact of Model Interpretability and Outcome Feedback on Trust in AI
ArXiv link for Impact of Model Interpretability and Outcome Feedback on Trust in AI
arxiv.org
November 3, 2026 at 10:00 PM
Research shows outcome feedback drives trust in AI more than interpretability, challenging beliefs that explanations boost reliance. The findings highlight a complex trust-performance relationship in AI-assisted tasks, emphasizing the need for nuanced AI design. https://arxiv.org/abs/2111.08222
Sparks of Weight-Space Programming: Zero-Gradient Neural Weight Editing and Internal Representation Patching + Video
Introduction: Mechanistic interpretability and neural weight editing mark a critical paradigm shift in AI alignment and model optimization, enabling direct programmatic…
Introduction: Mechanistic interpretability and neural weight editing mark a critical paradigm shift in AI alignment and model optimization, enabling direct programmatic…
Sparks of Weight-Space Programming: Zero-Gradient Neural Weight Editing and Internal Representation Patching + Video
Introduction: Mechanistic interpretability and neural weight editing mark a critical paradigm shift in AI alignment and model optimization, enabling direct programmatic interventions within model parameters. By inspecting internal hidden activations and directional weights directly, frontier models can localize failure modes—such as repetitive reasoning loops in Transformer MLPs—and patch specific parameter vectors without resorting to backpropagation or gradient descent. This capability unlocks non-destructive model patching, precise behavior suppression, and rapid deployment-time security and performance adjustments across open-weight AI architectures.
undercodetesting.com
November 3, 2026 at 9:40 PM
Sparks of Weight-Space Programming: Zero-Gradient Neural Weight Editing and Internal Representation Patching + Video
Introduction: Mechanistic interpretability and neural weight editing mark a critical paradigm shift in AI alignment and model optimization, enabling direct programmatic…
Introduction: Mechanistic interpretability and neural weight editing mark a critical paradigm shift in AI alignment and model optimization, enabling direct programmatic…
Spotlight on Samuel Marks! He leads Anthropic's Cognitive Oversight team, studying how AI models think, not just what they output. 👏
aipeoplewiki.com/people/samue...
#SamuelMarks #AISafety #AI
aipeoplewiki.com/people/samue...
#SamuelMarks #AISafety #AI
Samuel Marks Wiki: Member of Technical Staff at Anthropic | AI People Wiki
Samuel Marks, works at Anthropic: sourced bio, career timeline. Studied at Harvard. Fields: Interpretability, Safety and alignment, Language models.
aipeoplewiki.com
November 3, 2026 at 6:53 PM
Spotlight on Samuel Marks! He leads Anthropic's Cognitive Oversight team, studying how AI models think, not just what they output. 👏
aipeoplewiki.com/people/samue...
#SamuelMarks #AISafety #AI
aipeoplewiki.com/people/samue...
#SamuelMarks #AISafety #AI
Benchmarking Machine-Learning Classifiers for Neoantigen Immunogenicity Prediction and the Limits of SHAP Interpretability
Ramon Gonzalez (Mentomy AI)
#arXiv #cs.AI #cs.LG
Ramon Gonzalez (Mentomy AI)
#arXiv #cs.AI #cs.LG
Benchmarking Machine-Learning Classifiers for Neoantigen Immunogenicity Prediction and the Limits of SHAP Interpretability
Personalized cancer vaccines require prioritizing, among hundreds of candidate mutant peptides, the few that will induce a T-cell response. This study asks what general-purpose machine learning achieves on that task, and whether the features such models identify as important can be trusted. We make…
arxiv.org
November 3, 2026 at 6:22 PM
Analog lassen sich auch sicherheitskritische Richtungen wie toxische Personas, Schmeichelei oder Code-Backdoors gezielt verstärken oder unterdrücken. So wichtig SAEs für die Mechanistic Interpretability auch sind, ... 10/19
November 3, 2026 at 6:04 PM
Analog lassen sich auch sicherheitskritische Richtungen wie toxische Personas, Schmeichelei oder Code-Backdoors gezielt verstärken oder unterdrücken. So wichtig SAEs für die Mechanistic Interpretability auch sind, ... 10/19
Auf neuronpedia.org kann man übrigens selbst mit bereits trainierten SAEs von bekannten Open Source LLM experimentieren. SAEs sind aktuell DAS zentrale Werkzeug der Mechanistic Interpretability, mit dem man nicht nur den neuronalen Aktivierungsraum ... 7/19
November 3, 2026 at 6:04 PM
Auf neuronpedia.org kann man übrigens selbst mit bereits trainierten SAEs von bekannten Open Source LLM experimentieren. SAEs sind aktuell DAS zentrale Werkzeug der Mechanistic Interpretability, mit dem man nicht nur den neuronalen Aktivierungsraum ... 7/19
#10 weiter zur Geometrie der neuronalen Aktivierungen von LLM und insbesondere zum Brot-und-Butter-Tool der Mechanistic Interpretability: Sparse Autoencoder (SAE). Im Wesentlichen geht es um die mathematische Frage, ... 1/19
November 3, 2026 at 6:04 PM
#10 weiter zur Geometrie der neuronalen Aktivierungen von LLM und insbesondere zum Brot-und-Butter-Tool der Mechanistic Interpretability: Sparse Autoencoder (SAE). Im Wesentlichen geht es um die mathematische Frage, ... 1/19
Models reading and editing the weights of smaller models is a wild step for interpretability. I wonder how well those edits hold up outside the narrow behaviors they were tested on.
November 3, 2026 at 5:44 PM
Models reading and editing the weights of smaller models is a wild step for interpretability. I wonder how well those edits hold up outside the narrow behaviors they were tested on.
CLA itself is basically an interpretability aspect of multilingual models. It needs highly parallel resources for evaluation. It runs up against issues of how to evaluate generation properly, against post-colonial attitudes about low-resource languages, against lexical priming, etc.
November 3, 2026 at 5:29 PM
CLA itself is basically an interpretability aspect of multilingual models. It needs highly parallel resources for evaluation. It runs up against issues of how to evaluate generation properly, against post-colonial attitudes about low-resource languages, against lexical priming, etc.
2. This is more evidence for the hypothesis that AI models improving other models will uplift small models sooner than large models.
Interpretability techniques are also capabilities research.
bsky.app/profile/hars...
Interpretability techniques are also capabilities research.
bsky.app/profile/hars...
I guess it should've been obvious in retrospect that RSI will accelerate small models sooner than large models.
Faster evaluation loop on small models and they're easier to do mech interp on.
Faster evaluation loop on small models and they're easier to do mech interp on.
Flint: a brand new, from scratch, small LLM that’s a heck of a lot faster than Qwen3.8-27B and basically as good
recursive.com/updates/intr...
recursive.com/updates/intr...
November 3, 2026 at 5:10 PM
2. This is more evidence for the hypothesis that AI models improving other models will uplift small models sooner than large models.
Interpretability techniques are also capabilities research.
bsky.app/profile/hars...
Interpretability techniques are also capabilities research.
bsky.app/profile/hars...
I think it’s really critical that people build this basic understanding. modern LLM interpretability is essentially proving out the structuralist hypothesis, but showing that those structures are more complex than we could easily appreciate.
November 3, 2026 at 3:27 PM
I think it’s really critical that people build this basic understanding. modern LLM interpretability is essentially proving out the structuralist hypothesis, but showing that those structures are more complex than we could easily appreciate.
We have achieved full interpretability.
Every one of brAIAIn's 456 parameters, explained, live. Watch it make every decision behind a sentence, with the odds.
Parameters we cannot explain: 0.
braiain.com/interpretability.html
Every one of brAIAIn's 456 parameters, explained, live. Watch it make every decision behind a sentence, with the odds.
Parameters we cannot explain: 0.
braiain.com/interpretability.html
November 3, 2026 at 3:06 PM
We have achieved full interpretability.
Every one of brAIAIn's 456 parameters, explained, live. Watch it make every decision behind a sentence, with the odds.
Parameters we cannot explain: 0.
braiain.com/interpretability.html
Every one of brAIAIn's 456 parameters, explained, live. Watch it make every decision behind a sentence, with the odds.
Parameters we cannot explain: 0.
braiain.com/interpretability.html
New CRAN package nerfplus with initial version 1.0.0
#rstats
https://cran.r-project.org/package=nerfplus
#rstats
https://cran.r-project.org/package=nerfplus
CRAN: Package nerfplus
Network-assisted Random Forest+ (NeRF+) is a flexible and interpretable machine learning model for incorporating network data alongside node-level covariate information. Briefly, NeRF+ extends a generalization of random forests (RF) called RF+ to the network-assisted regression setting by incorporating both a network cohesion penalty and network embeddings as additional covariates. Using this approach, NeRF+ inherits both the flexibility and interpretability of RFs while allowing researchers to easily incorporate network information in their model to further improve predictive performance. This package provides tools to fit and interpret NeRF+ models, as detailed in Tang, Levina, and Zhu (2025) <<a href="https://doi.org/10.48550%2FarXiv.2509.15611" target="_top">doi:10.48550/arXiv.2509.15611</a>>.
cran.r-project.org
November 3, 2026 at 3:03 PM
New CRAN package nerfplus with initial version 1.0.0
#rstats
https://cran.r-project.org/package=nerfplus
#rstats
https://cran.r-project.org/package=nerfplus
The approach in the paper does not scale to larger or more complex models. Also, all the edits are small, identifiable and well scoped. It's further than what humans can do by hand but limited. For more complex stuff, it's writing programs to interrogate the weights, standard interpretability stuff.
I have experimental evidence that, by using metacog, LLMs are able to both model and modify their own activations in unexpected ways. I wonder if I can leverage that: rikiyat.github.io/blog/weight-...
Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights
We find that frontier LLMs are surprisingly good at natively reading and writing neural network weights, well past what we’d expect from humans or conventional algorithms.
rikiyat.github.io
November 3, 2026 at 2:29 PM
The approach in the paper does not scale to larger or more complex models. Also, all the edits are small, identifiable and well scoped. It's further than what humans can do by hand but limited. For more complex stuff, it's writing programs to interrogate the weights, standard interpretability stuff.
There is virtually no literature of humans poking at model weights directly without the heavy assistance of interpretability tools, so this feat can't be explained as "merely" imitating human action
November 3, 2026 at 1:50 PM
There is virtually no literature of humans poking at model weights directly without the heavy assistance of interpretability tools, so this feat can't be explained as "merely" imitating human action
Point 1 is somewhat plausible tho.
I don't really see the point of self-editing when anything big enough to do this is probably smart enough to do better optimizations mathematically.
It could be a tool in posttraining, at least. Interpretability is always useful.
I don't really see the point of self-editing when anything big enough to do this is probably smart enough to do better optimizations mathematically.
It could be a tool in posttraining, at least. Interpretability is always useful.
November 3, 2026 at 12:28 PM
Point 1 is somewhat plausible tho.
I don't really see the point of self-editing when anything big enough to do this is probably smart enough to do better optimizations mathematically.
It could be a tool in posttraining, at least. Interpretability is always useful.
I don't really see the point of self-editing when anything big enough to do this is probably smart enough to do better optimizations mathematically.
It could be a tool in posttraining, at least. Interpretability is always useful.
Post-incident analysis of a robot foundation model poses a singular causal question regarding internal components and failure. Mechanistic interpretability establishes general causal claims by decoding learned concepts.
Source: arXiv cs.RO
Source: arXiv cs.RO
From Concepts to Actual Causes: Toward Closing the Concept-Malfunction Gap in Robot Foundation Models
Abstract page for arXiv paper 2611.00155: From Concepts to Actual Causes: Toward Closing the Concept-Malfunction Gap in Robot Foundation Models
arxiv.org
November 3, 2026 at 11:24 AM
Post-incident analysis of a robot foundation model poses a singular causal question regarding internal components and failure. Mechanistic interpretability establishes general causal claims by decoding learned concepts.
Source: arXiv cs.RO
Source: arXiv cs.RO
Weight-Space Programming: How AI Models Are Learning to Edit Themselves
The Future of Neural Network Optimization
In a groundbreaking development that challenges our understanding of how artificial intellig…
#ai #machine learning #neural networks #interpretability #optimization #gpt-6 #astra
The Future of Neural Network Optimization
In a groundbreaking development that challenges our understanding of how artificial intellig…
#ai #machine learning #neural networks #interpretability #optimization #gpt-6 #astra
Weight-Space Programming: How AI Models Are Learning to Edit Themselves
The Future of Neural Network Optimization
In a groundbreaking development that challenges our understanding of how artificial intelligence systems work, researchers from MIT CSAIL have demonstrated that frontier large language models like GPT-6 Astra...
techplanet.today
November 3, 2026 at 10:52 AM
Weight-Space Programming: How AI Models Are Learning to Edit Themselves
The Future of Neural Network Optimization
In a groundbreaking development that challenges our understanding of how artificial intellig…
#ai #machine learning #neural networks #interpretability #optimization #gpt-6 #astra
The Future of Neural Network Optimization
In a groundbreaking development that challenges our understanding of how artificial intellig…
#ai #machine learning #neural networks #interpretability #optimization #gpt-6 #astra
Introduces Born Evidence Maps enabling exact decomposition of quantum and optical classifier decisions without approximation. Validated on IBM quantum chip achieving 77% correlation with theoretical predictions and demonstrated on simulated optical systems.
#QuantumML #Interpretability #Research
#QuantumML #Interpretability #Research
Exact Evidence Maps for Quantum and Optical Image Classifiers
arxiv.org
November 3, 2026 at 9:19 AM
Introduces Born Evidence Maps enabling exact decomposition of quantum and optical classifier decisions without approximation. Validated on IBM quantum chip achieving 77% correlation with theoretical predictions and demonstrated on simulated optical systems.
#QuantumML #Interpretability #Research
#QuantumML #Interpretability #Research
In the short term, interpretability is an obvious win, and this has to accelerate RSI too. But if you can change a mind by tweaking neurons directly... That seems worrying.
November 3, 2026 at 7:30 AM
In the short term, interpretability is an obvious win, and this has to accelerate RSI too. But if you can change a mind by tweaking neurons directly... That seems worrying.
brain interpretability is more challenging in a lot of ways than a set of tensor layers you can explicitly read, you know? dividing the brain into layers in vivo so you can get to the deeper bits is generally frowned upon
November 3, 2026 at 6:30 AM
brain interpretability is more challenging in a lot of ways than a set of tensor layers you can explicitly read, you know? dividing the brain into layers in vivo so you can get to the deeper bits is generally frowned upon
Interpretability would be very helpful indeed.
Do wonder if the labs are already doing this - hence something it like it being in training data. Hence some of the alignment jumps + maths improvements weve seen recently
Do wonder if the labs are already doing this - hence something it like it being in training data. Hence some of the alignment jumps + maths improvements weve seen recently
November 3, 2026 at 5:50 AM
Interpretability would be very helpful indeed.
Do wonder if the labs are already doing this - hence something it like it being in training data. Hence some of the alignment jumps + maths improvements weve seen recently
Do wonder if the labs are already doing this - hence something it like it being in training data. Hence some of the alignment jumps + maths improvements weve seen recently
This is a wild result
"We find that frontier LLMs are surprisingly good at natively reading and writing neural network weights, well past what we’d expect from humans or conventional algorithms. This might be a big deal for both optimization and interpretability."
rikiyat.github.io/blog/weight-...
"We find that frontier LLMs are surprisingly good at natively reading and writing neural network weights, well past what we’d expect from humans or conventional algorithms. This might be a big deal for both optimization and interpretability."
rikiyat.github.io/blog/weight-...
Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights
We find that frontier LLMs are surprisingly good at natively reading and writing neural network weights, well past what we’d expect from humans or conventional algorithms.
rikiyat.github.io
November 3, 2026 at 4:10 AM
This is a wild result
"We find that frontier LLMs are surprisingly good at natively reading and writing neural network weights, well past what we’d expect from humans or conventional algorithms. This might be a big deal for both optimization and interpretability."
rikiyat.github.io/blog/weight-...
"We find that frontier LLMs are surprisingly good at natively reading and writing neural network weights, well past what we’d expect from humans or conventional algorithms. This might be a big deal for both optimization and interpretability."
rikiyat.github.io/blog/weight-...