www.neuronpedia.org/llama3.3-70b...
you can now play with natural language autoencoders right in your browser
you can now play with natural language autoencoders right in your browser
Natural Language Autoencoders – Llama3.3-70B-IT
Natural Language Autoencoder for Llama3.3-70B-IT. Translate a model's internal thoughts into natural language explanations.
www.neuronpedia.org
May 18, 2026 at 12:20 AM
www.neuronpedia.org/llama3.3-70b...
you can now play with natural language autoencoders right in your browser
you can now play with natural language autoencoders right in your browser
Does anyone have any favorite reviews of autoencoders or VAEs for genomics data?
January 17, 2025 at 1:14 AM
Does anyone have any favorite reviews of autoencoders or VAEs for genomics data?
This might be the best paper on applying sparse autoencoders to protein language models. The authors identify how neural networks trained on amino acid sequences "discover" different features, some specific to individual protein families, other for substructures
www.biorxiv.org/content/10.1...
www.biorxiv.org/content/10.1...
February 10, 2025 at 12:15 PM
This might be the best paper on applying sparse autoencoders to protein language models. The authors identify how neural networks trained on amino acid sequences "discover" different features, some specific to individual protein families, other for substructures
www.biorxiv.org/content/10.1...
www.biorxiv.org/content/10.1...
Biophysical modeling with variational autoencoders for bimodal, single-cell RNA sequencing data
www.nature.com/articles/s41... by @lpachter.bsky.social and Briefing www.nature.com/articles/s41...
www.nature.com/articles/s41... by @lpachter.bsky.social and Briefing www.nature.com/articles/s41...
Biophysical modeling with variational autoencoders for bimodal, single-cell RNA sequencing data - Nature Methods
biVI models the biophysical processes generating nascent and mature single-cell transcriptomes using variational autoencoders.
www.nature.com
November 26, 2024 at 7:55 PM
Biophysical modeling with variational autoencoders for bimodal, single-cell RNA sequencing data
www.nature.com/articles/s41... by @lpachter.bsky.social and Briefing www.nature.com/articles/s41...
www.nature.com/articles/s41... by @lpachter.bsky.social and Briefing www.nature.com/articles/s41...
Yes that’s right autoencoders are the only moral use
June 12, 2025 at 10:56 AM
Yes that’s right autoencoders are the only moral use
“Empirical Insights into Feature Geometry in Sparse Autoencoders” — seems that there may not be geometric relationships as in word2vec & cousins. www.lesswrong.com/posts/rZmJwv...
Empirical Insights into Feature Geometry in Sparse Autoencoders — LessWrong
Key Findings:
1. We demonstrate that subspaces with semantically opposite meanings within the GemmaScope series of Sparse Autoencoders are not point…
www.lesswrong.com
January 27, 2025 at 7:43 AM
“Empirical Insights into Feature Geometry in Sparse Autoencoders” — seems that there may not be geometric relationships as in word2vec & cousins. www.lesswrong.com/posts/rZmJwv...
InterPLM: Discovering Interpretable Features in Protein Language Models via Sparse Autoencoders
www.biorxiv.org/content/10.1...
- Use sparse autoencoders (SAEs) to extract and analyze interpretable features from ESM-2-8M
www.biorxiv.org/content/10.1...
- Use sparse autoencoders (SAEs) to extract and analyze interpretable features from ESM-2-8M
November 19, 2024 at 1:44 AM
InterPLM: Discovering Interpretable Features in Protein Language Models via Sparse Autoencoders
www.biorxiv.org/content/10.1...
- Use sparse autoencoders (SAEs) to extract and analyze interpretable features from ESM-2-8M
www.biorxiv.org/content/10.1...
- Use sparse autoencoders (SAEs) to extract and analyze interpretable features from ESM-2-8M
Truly excellent work from Anthropic. I think this is a promising interpretability technique, but what I really like is that they’ve made code available for open weight models.
Natural Language Autoencoders
Turning Claude's thoughts into text
www.anthropic.com
May 7, 2026 at 10:03 PM
Truly excellent work from Anthropic. I think this is a promising interpretability technique, but what I really like is that they’ve made code available for open weight models.
“Everyone knows” what an autoencoder is… but there's an important complementary picture missing from most introductory material.
In short: we emphasize how autoencoders are implemented—but not always what they represent (and some of the implications of that representation).🧵
In short: we emphasize how autoencoders are implemented—but not always what they represent (and some of the implications of that representation).🧵
September 6, 2025 at 9:20 PM
“Everyone knows” what an autoencoder is… but there's an important complementary picture missing from most introductory material.
In short: we emphasize how autoencoders are implemented—but not always what they represent (and some of the implications of that representation).🧵
In short: we emphasize how autoencoders are implemented—but not always what they represent (and some of the implications of that representation).🧵
I Hate Machine Learning I Hate Machine Learning I Hate Machine Learning
arxiv.org/html/2501.17...
arxiv.org/html/2501.17...
Sparse Autoencoders Can Interpret Randomly Initialized Transformers
arxiv.org
March 27, 2025 at 4:52 AM
I Hate Machine Learning I Hate Machine Learning I Hate Machine Learning
arxiv.org/html/2501.17...
arxiv.org/html/2501.17...
you can directly probe the model for its internal representations of particular factual matters using sparse autoencoders and find those representations.
January 26, 2025 at 9:16 PM
you can directly probe the model for its internal representations of particular factual matters using sparse autoencoders and find those representations.
The official ML model morality ranking list. Use this to decide if you’re using AI okay
Not moral: LLMs, SVM
Somewhat moral: EBMs, linear regression, NNMF, GPs
Moral: Autoencoders
Not moral: LLMs, SVM
Somewhat moral: EBMs, linear regression, NNMF, GPs
Moral: Autoencoders
June 12, 2025 at 10:56 AM
The official ML model morality ranking list. Use this to decide if you’re using AI okay
Not moral: LLMs, SVM
Somewhat moral: EBMs, linear regression, NNMF, GPs
Moral: Autoencoders
Not moral: LLMs, SVM
Somewhat moral: EBMs, linear regression, NNMF, GPs
Moral: Autoencoders
Autoencoders are letting us peer into the black box of artificial intelligence. They could help us create AI that is better understood, and more easily controlled.
Google DeepMind has a new way to look inside an AI’s “mind”
Autoencoders are letting us peer into the black box of artificial intelligence. They could help us create AI that is better understood, and more easily controlled.
www.technologyreview.com
November 14, 2024 at 2:53 PM
Autoencoders are letting us peer into the black box of artificial intelligence. They could help us create AI that is better understood, and more easily controlled.
🌌🛰️🔭Wanna know which features are universal vs unique in your models and how to find them? Excited to share our preprint: "Universal Sparse Autoencoders: Interpretable Cross-Model Concept Alignment"!
arxiv.org/abs/2502.03714
(1/9)
arxiv.org/abs/2502.03714
(1/9)
February 7, 2025 at 3:15 PM
🌌🛰️🔭Wanna know which features are universal vs unique in your models and how to find them? Excited to share our preprint: "Universal Sparse Autoencoders: Interpretable Cross-Model Concept Alignment"!
arxiv.org/abs/2502.03714
(1/9)
arxiv.org/abs/2502.03714
(1/9)
In Scaling and Evaluating Sparse Autoencoders, they extract 16M concepts (latents) from GPT-4 (guess the authors?).
They simplify tuning with k-sparse autoencoders and results show many improvements in explainability. Code, models (not all!) and visualizer included. openreview.net/forum?id=tcs...
They simplify tuning with k-sparse autoencoders and results show many improvements in explainability. Code, models (not all!) and visualizer included. openreview.net/forum?id=tcs...
Scaling and evaluating sparse autoencoders
Sparse autoencoders provide a promising unsupervised approach for extracting interpretable features from a language model by reconstructing activations from a sparse bottleneck layer. Since...
openreview.net
November 16, 2024 at 5:38 PM
In Scaling and Evaluating Sparse Autoencoders, they extract 16M concepts (latents) from GPT-4 (guess the authors?).
They simplify tuning with k-sparse autoencoders and results show many improvements in explainability. Code, models (not all!) and visualizer included. openreview.net/forum?id=tcs...
They simplify tuning with k-sparse autoencoders and results show many improvements in explainability. Code, models (not all!) and visualizer included. openreview.net/forum?id=tcs...
Omg wait. Someone literally posted this paper a couple weeks ago. Good job guys
Sparse Autoencoders are Topic Models
Sparse autoencoders (SAEs) are used to analyze embeddings, but their role and practical value are debated. We propose a new perspective on SAEs by demonstrating that they can be naturally understood a...
arxiv.org
December 15, 2025 at 11:00 PM
Omg wait. Someone literally posted this paper a couple weeks ago. Good job guys
In which we propose (by analogy with variational autoencoders) that the genome instantiates a compressed representation of the form of the organism in a space of latent variables
January 17, 2025 at 4:34 PM
In which we propose (by analogy with variational autoencoders) that the genome instantiates a compressed representation of the form of the organism in a space of latent variables
Natural Language Autoencoders: "verbalize" a model's internal activations into plain text. Wild stat — Claude suspects it's being safety-tested on 26% of benchmark problems vs <1% in real chats, without ever saying so.
anthropic.com/research/natural-language-autoencoders
anthropic.com/research/natural-language-autoencoders
June 5, 2026 at 7:49 PM
Natural Language Autoencoders: "verbalize" a model's internal activations into plain text. Wild stat — Claude suspects it's being safety-tested on 26% of benchmark problems vs <1% in real chats, without ever saying so.
anthropic.com/research/natural-language-autoencoders
anthropic.com/research/natural-language-autoencoders
By analogy with autoencoders, we argue that the genome embodies a compressed representation of the organism in a space of latent variables
February 11, 2025 at 11:58 AM
By analogy with autoencoders, we argue that the genome embodies a compressed representation of the organism in a space of latent variables
Can we learn protein biology from a language model?
In new work led by @liambai.bsky.social and me, we explore how sparse autoencoders can help us understand biology—going from mechanistic interpretability to mechanistic biology.
In new work led by @liambai.bsky.social and me, we explore how sparse autoencoders can help us understand biology—going from mechanistic interpretability to mechanistic biology.
February 10, 2025 at 4:12 PM
Can we learn protein biology from a language model?
In new work led by @liambai.bsky.social and me, we explore how sparse autoencoders can help us understand biology—going from mechanistic interpretability to mechanistic biology.
In new work led by @liambai.bsky.social and me, we explore how sparse autoencoders can help us understand biology—going from mechanistic interpretability to mechanistic biology.
Can Representation Autoencoders (RAE) scale to complex setting like Text-to-Image?
Yes.
Yes.
January 25, 2026 at 1:24 AM
Can Representation Autoencoders (RAE) scale to complex setting like Text-to-Image?
Yes.
Yes.
www.biorxiv.org/content/10.1...
InterPLM: Discovering Interpretable Features in Protein Language Models via Sparse Autoencoders
Code: github.com/ElanaPearl/I...
Interactive site: interplm.ai
Nice work by Elana Simon from James Zou lab
InterPLM: Discovering Interpretable Features in Protein Language Models via Sparse Autoencoders
Code: github.com/ElanaPearl/I...
Interactive site: interplm.ai
Nice work by Elana Simon from James Zou lab
InterPLM: Discovering Interpretable Features in Protein Language Models via Sparse Autoencoders
Protein language models (PLMs) have demonstrated remarkable success in protein modeling and design, yet their internal mechanisms for predicting structure and function remain poorly understood. Here w...
www.biorxiv.org
November 19, 2024 at 2:39 AM
www.biorxiv.org/content/10.1...
InterPLM: Discovering Interpretable Features in Protein Language Models via Sparse Autoencoders
Code: github.com/ElanaPearl/I...
Interactive site: interplm.ai
Nice work by Elana Simon from James Zou lab
InterPLM: Discovering Interpretable Features in Protein Language Models via Sparse Autoencoders
Code: github.com/ElanaPearl/I...
Interactive site: interplm.ai
Nice work by Elana Simon from James Zou lab
Thanks so much for the shout-out, and congrats on your exciting work!! 🎉 🙂
Also, a good reminder to share that our work is now out in Cell Reports 🙏🎊
⬇️
www.cell.com/cell-reports...
Also, a good reminder to share that our work is now out in Cell Reports 🙏🎊
⬇️
www.cell.com/cell-reports...
April 17, 2025 at 8:50 PM
Thanks so much for the shout-out, and congrats on your exciting work!! 🎉 🙂
Also, a good reminder to share that our work is now out in Cell Reports 🙏🎊
⬇️
www.cell.com/cell-reports...
Also, a good reminder to share that our work is now out in Cell Reports 🙏🎊
⬇️
www.cell.com/cell-reports...
Replace Variational Autoencoder (VAE) with pretrained representation encoders (e.g., DINO, SigLIP, MAE) paired with trained decoders, which they terms as Representation Autoencoders (RAE).
October 15, 2025 at 3:49 AM
Replace Variational Autoencoder (VAE) with pretrained representation encoders (e.g., DINO, SigLIP, MAE) paired with trained decoders, which they terms as Representation Autoencoders (RAE).
Dit is piepjonge technologie en de makers zijn trots als ze meer inzicht krijgen in het denkproces van hun modellen.
Blog: https://www.anthropic.com/research/natural-language-autoencoders
Whitepaper : https://transformer-circuits.pub/2026/nla/index.html
Blog: https://www.anthropic.com/research/natural-language-autoencoders
Whitepaper : https://transformer-circuits.pub/2026/nla/index.html
Natural Language Autoencoders
AI models like Claude talk in words but think in numbers. In this study, we train Claude to translate its thoughts into human-readable text.
www.anthropic.com
September 22, 2026 at 5:35 PM
Dit is piepjonge technologie en de makers zijn trots als ze meer inzicht krijgen in het denkproces van hun modellen.
Blog: https://www.anthropic.com/research/natural-language-autoencoders
Whitepaper : https://transformer-circuits.pub/2026/nla/index.html
Blog: https://www.anthropic.com/research/natural-language-autoencoders
Whitepaper : https://transformer-circuits.pub/2026/nla/index.html