What's the difference between explainability and interpretability? Does the machine learning community have an agreed-upon definition?
No.
There's a great overview, which is from this paper: arxiv.org/abs/2211.08943
My take: I prefer interpretability since the term explainability is too strong.
No.
There's a great overview, which is from this paper: arxiv.org/abs/2211.08943
My take: I prefer interpretability since the term explainability is too strong.
November 26, 2024 at 3:41 PM
What's the difference between explainability and interpretability? Does the machine learning community have an agreed-upon definition?
No.
There's a great overview, which is from this paper: arxiv.org/abs/2211.08943
My take: I prefer interpretability since the term explainability is too strong.
No.
There's a great overview, which is from this paper: arxiv.org/abs/2211.08943
My take: I prefer interpretability since the term explainability is too strong.
‘85% of IT leaders say traceability and explainability gaps have delayed or stopped AI projects from reaching production.‘
I love the term ‘traceability and explainability gaps’ as a way of saying ‘We have no way of knowing why our LLM bots do things and we’ve suddenly realized that’s a problem’.
I love the term ‘traceability and explainability gaps’ as a way of saying ‘We have no way of knowing why our LLM bots do things and we’ve suddenly realized that’s a problem’.
March 11, 2026 at 10:03 AM
‘85% of IT leaders say traceability and explainability gaps have delayed or stopped AI projects from reaching production.‘
I love the term ‘traceability and explainability gaps’ as a way of saying ‘We have no way of knowing why our LLM bots do things and we’ve suddenly realized that’s a problem’.
I love the term ‘traceability and explainability gaps’ as a way of saying ‘We have no way of knowing why our LLM bots do things and we’ve suddenly realized that’s a problem’.
Fun afternoon talking about proof, evidence, explainability and trust at a sold out Hackney Empire.
January 26, 2025 at 7:26 PM
Fun afternoon talking about proof, evidence, explainability and trust at a sold out Hackney Empire.
What problem is explainability/interpretability research trying to solve in ML, and do you have a favorite paper articulating what that problem is?
October 8, 2025 at 7:27 PM
What problem is explainability/interpretability research trying to solve in ML, and do you have a favorite paper articulating what that problem is?
Explainability is a scam like alignment and guardrails bsky.app/profile/iris...
If you need to do a lot of work to make model explainable, you didn't make a (good) model
June 20, 2026 at 6:19 PM
Explainability is a scam like alignment and guardrails bsky.app/profile/iris...
i'm glad you brought up the automated proof example because it shows how relative concepts like "explainability" and "interpretability" are
July 11, 2025 at 3:23 PM
i'm glad you brought up the automated proof example because it shows how relative concepts like "explainability" and "interpretability" are
A starter pack of people working on interpretability / explainability of all kinds, using theoretical and/or empirical approaches.
Reply or DM if you want to be added, and help me reach others!
go.bsky.app/DZv6TSS
Reply or DM if you want to be added, and help me reach others!
go.bsky.app/DZv6TSS
November 14, 2024 at 5:00 PM
A starter pack of people working on interpretability / explainability of all kinds, using theoretical and/or empirical approaches.
Reply or DM if you want to be added, and help me reach others!
go.bsky.app/DZv6TSS
Reply or DM if you want to be added, and help me reach others!
go.bsky.app/DZv6TSS
This has been partially out for a while now but it's finally complete: Our IJDH special issue on "Reproducibility & Explainability" in DH, with contributions by a host of wonderful colleagues. Check out our introduction below and then read fantastic pieces by link.springer.com/collections/... 1/2
Reproducibility and Explainability in Digital Humanities
“Reproducibility” and “explainability” are important methodological considerations in the Sciences, and are increasingly relevant in Digital Humanities. The ...
link.springer.com
January 16, 2024 at 5:54 PM
This has been partially out for a while now but it's finally complete: Our IJDH special issue on "Reproducibility & Explainability" in DH, with contributions by a host of wonderful colleagues. Check out our introduction below and then read fantastic pieces by link.springer.com/collections/... 1/2
Excited to share our paper: "Chain-of-Thought Is Not Explainability"! We unpack a critical misconception in AI: models explaining their steps (CoT) aren't necessarily revealing their true reasoning. Spoiler: the transparency can be an illusion. (1/9) 🧵
July 1, 2025 at 3:41 PM
Excited to share our paper: "Chain-of-Thought Is Not Explainability"! We unpack a critical misconception in AI: models explaining their steps (CoT) aren't necessarily revealing their true reasoning. Spoiler: the transparency can be an illusion. (1/9) 🧵
Fantastic new paper casting doubt on explainability of explainable AI. To explain complex machine learning algorithms you need reproducibility of the explanation at a minimum academic.oup.com/ehjdh/articl... #machinelearning #Statistics #StatsSky @maartenvsmeden.bsky.social
Signal or noise? Evaluating commonly used attribution methods for explaining deep neural networks in electrocardiogram classification
AbstractAims. Attribution-based explainability methods are widely used in electrocardiogram (ECG) analysis to interpret predictions from ‘black-box’ deep n
academic.oup.com
March 17, 2026 at 12:39 PM
Fantastic new paper casting doubt on explainability of explainable AI. To explain complex machine learning algorithms you need reproducibility of the explanation at a minimum academic.oup.com/ehjdh/articl... #machinelearning #Statistics #StatsSky @maartenvsmeden.bsky.social
Thank you for replying so thoroughly Alondra! Algorithmic agnotology is such an apt concept. It makes me realise the poverty of the standard convo on explainability in philosophy. Sure we can’t expect total explainability of any tool but AI companies have powerful incentives to keep it to minimum
August 6, 2026 at 1:33 PM
Thank you for replying so thoroughly Alondra! Algorithmic agnotology is such an apt concept. It makes me realise the poverty of the standard convo on explainability in philosophy. Sure we can’t expect total explainability of any tool but AI companies have powerful incentives to keep it to minimum
one of the frustrating things about explainability work is that it will never explain things such as, e.g., "why did the computer draw that based on what i told it to." somewhere in an alien high-dimensional geometry, spaghetti is adjacent to this weird goblin.
February 18, 2025 at 4:25 AM
one of the frustrating things about explainability work is that it will never explain things such as, e.g., "why did the computer draw that based on what i told it to." somewhere in an alien high-dimensional geometry, spaghetti is adjacent to this weird goblin.
But if you _do_ want some AI help, Graze has an LLM you can prompt to build you a feed as well!
Graze's LLM & feed builder also provides some explainability since you can look at the blocks that the LLM generates for you.
Graze's LLM & feed builder also provides some explainability since you can look at the blocks that the LLM generates for you.
🤔 But what if I want a custom feed NOW?
As it turns out, graze.social has the technology. Creating feeds can be as easy as pie. Give it a try!
No AI required. Just I.
As it turns out, graze.social has the technology. Creating feeds can be as easy as pie. Give it a try!
No AI required. Just I.
Bluesky Custom Feeds - Built By You
Design, deploy, and grow custom feeds of any complexity on Bluesky with Graze.social.
graze.social
March 30, 2026 at 1:42 PM
But if you _do_ want some AI help, Graze has an LLM you can prompt to build you a feed as well!
Graze's LLM & feed builder also provides some explainability since you can look at the blocks that the LLM generates for you.
Graze's LLM & feed builder also provides some explainability since you can look at the blocks that the LLM generates for you.
Giving a talk tomorrow at #NeurIPS2024 on the exciting topic of explainability!
At NeurIPS? Come by the 2nd workshop on Attributing Model Behavior at Scale (ATTRIB)!
Meeting Rm 205-207 @ 9am - amazing talks by @surbhigoel.bsky.social @sanmikoyejo.bsky.social Baharan Mirzasoleiman, Robert Geirhos, @coallaoh.bsky.social + exciting contributed talks!
Details: attrib-workshop.cc
Meeting Rm 205-207 @ 9am - amazing talks by @surbhigoel.bsky.social @sanmikoyejo.bsky.social Baharan Mirzasoleiman, Robert Geirhos, @coallaoh.bsky.social + exciting contributed talks!
Details: attrib-workshop.cc
ATTRIB 2024 WorkshopConference Schedule
attrib-workshop.cc
December 14, 2024 at 1:56 AM
Giving a talk tomorrow at #NeurIPS2024 on the exciting topic of explainability!
I like explainability, understandability, reliability, mathematical logic, rather than empirical black box voodoo with no guarantees or easy way to predict when it will and won't work.
March 30, 2026 at 3:09 PM
I like explainability, understandability, reliability, mathematical logic, rather than empirical black box voodoo with no guarantees or easy way to predict when it will and won't work.
Regardless of what explainability/mech interp in AI is actually after, and whether or not they know what they’re searching for, we can confidently say they’re pursuing what systems neuroscience has pursued for decades, with very similar puzzles and confusions.
What problem is explainability/interpretability research trying to solve in ML, and do you have a favorite paper articulating what that problem is?
October 8, 2025 at 8:17 PM
Regardless of what explainability/mech interp in AI is actually after, and whether or not they know what they’re searching for, we can confidently say they’re pursuing what systems neuroscience has pursued for decades, with very similar puzzles and confusions.
🌍🌐 Registration for Course 3 of our ML training series under the EU DestinE initiative is now open. Explore explainability, foundation models, hybrid modelling, downscaling, coupled Earth systems and AI forecasting pipelines. 📅 5 October | Online
Register now: learning.ecmwf.int/course/view....
Register now: learning.ecmwf.int/course/view....
September 25, 2026 at 8:57 AM
🌍🌐 Registration for Course 3 of our ML training series under the EU DestinE initiative is now open. Explore explainability, foundation models, hybrid modelling, downscaling, coupled Earth systems and AI forecasting pipelines. 📅 5 October | Online
Register now: learning.ecmwf.int/course/view....
Register now: learning.ecmwf.int/course/view....
3) Explainability
Model explainability (e.g. Shap values) is becoming increasingly popular to open the "black-box" of complex analytical models. Unfortunately, opening black-boxes often reveals things of limited interest to readers and may even mislead
doi.org/10.1016/S2589-7500(21)00208-9
Model explainability (e.g. Shap values) is becoming increasingly popular to open the "black-box" of complex analytical models. Unfortunately, opening black-boxes often reveals things of limited interest to readers and may even mislead
doi.org/10.1016/S2589-7500(21)00208-9
The false hope of current approaches to explainable artificial intelligence in health care
The black-box nature of current artificial intelligence (AI) has caused some to question whether AI must be explainable to be used in high-stakes scenarios such as medicine. It has been argued that explainable AI will engender trust with the health-care workforce, provide transparency into the AI decision making process, and potentially mitigate various kinds of bias. In this Viewpoint, we argue that this argument represents a false hope for explainable AI and that current explainability methods are unlikely to achieve these goals for patient-level decision support.
www.thelancet.com
December 23, 2024 at 10:36 AM
3) Explainability
Model explainability (e.g. Shap values) is becoming increasingly popular to open the "black-box" of complex analytical models. Unfortunately, opening black-boxes often reveals things of limited interest to readers and may even mislead
doi.org/10.1016/S2589-7500(21)00208-9
Model explainability (e.g. Shap values) is becoming increasingly popular to open the "black-box" of complex analytical models. Unfortunately, opening black-boxes often reveals things of limited interest to readers and may even mislead
doi.org/10.1016/S2589-7500(21)00208-9
Ya'll were warned about the negative feedback loop problem a year ago. Unless someone comes up with an explainability solution, LLM tools are going to keep getting worse.
We ran a randomized controlled trial to see how much AI coding tools speed up experienced open-source developers.
The results surprised us: Developers thought they were 20% faster with AI tools, but they were actually 19% slower when they had access to AI than when they didn't.
The results surprised us: Developers thought they were 20% faster with AI tools, but they were actually 19% slower when they had access to AI than when they didn't.
July 10, 2025 at 10:44 PM
Ya'll were warned about the negative feedback loop problem a year ago. Unless someone comes up with an explainability solution, LLM tools are going to keep getting worse.
Explainability without mechanism is an antipattern.
September 23, 2026 at 8:54 AM
Explainability without mechanism is an antipattern.
Very happy to be at #FAccT2025 in Athens, where I presented our work "Gender Bias in Explainability: Investigating Performance Disparity in Post-hoc Methods"
📄Paper: dl.acm.org/doi/10.1145/...
At #FAccT2025? Let's connect if you're interested in improving the usability of explainability methods!
📄Paper: dl.acm.org/doi/10.1145/...
At #FAccT2025? Let's connect if you're interested in improving the usability of explainability methods!
June 26, 2025 at 5:25 AM
Very happy to be at #FAccT2025 in Athens, where I presented our work "Gender Bias in Explainability: Investigating Performance Disparity in Post-hoc Methods"
📄Paper: dl.acm.org/doi/10.1145/...
At #FAccT2025? Let's connect if you're interested in improving the usability of explainability methods!
📄Paper: dl.acm.org/doi/10.1145/...
At #FAccT2025? Let's connect if you're interested in improving the usability of explainability methods!
Code explainability for massive diffs: www.coderabbit.ai/change-stack
CodeRabbit Change Stack | Code explainability for massive diffs
Turn massive diffs into an understandable, layered review with CodeRabbit Change Stack.
www.coderabbit.ai
September 24, 2026 at 2:30 PM
Code explainability for massive diffs: www.coderabbit.ai/change-stack
*Automatically Interpreting Millions of Features in LLMs*
by @norabelrose.bsky.social et al.
An open-source pipeline for finding interpretable features in LLMs with sparse autoencoders and automated explainability methods from @eleutherai.bsky.social.
arxiv.org/abs/2410.13928
by @norabelrose.bsky.social et al.
An open-source pipeline for finding interpretable features in LLMs with sparse autoencoders and automated explainability methods from @eleutherai.bsky.social.
arxiv.org/abs/2410.13928
November 27, 2024 at 2:58 PM
*Automatically Interpreting Millions of Features in LLMs*
by @norabelrose.bsky.social et al.
An open-source pipeline for finding interpretable features in LLMs with sparse autoencoders and automated explainability methods from @eleutherai.bsky.social.
arxiv.org/abs/2410.13928
by @norabelrose.bsky.social et al.
An open-source pipeline for finding interpretable features in LLMs with sparse autoencoders and automated explainability methods from @eleutherai.bsky.social.
arxiv.org/abs/2410.13928
New piece on failures of explainable #AI in medical #diagnostics! As Laura Gorrieri, @paultrautt.bsky.social & I argue, Explanations should not focus on AI per se but on how AI enhances the ability to act (drawing on "understanding virtuoso" @henkderegt.bsky.social)
doi.org/10.1017/cfc....
doi.org/10.1017/cfc....
Between explaining and understanding: Rethinking AI explainability in medical diagnostics | Cambridge Forum on AI: Culture and Society | Cambridge Core
Between explaining and understanding: Rethinking AI explainability in medical diagnostics - Volume 2
doi.org
May 29, 2026 at 12:54 PM
New piece on failures of explainable #AI in medical #diagnostics! As Laura Gorrieri, @paultrautt.bsky.social & I argue, Explanations should not focus on AI per se but on how AI enhances the ability to act (drawing on "understanding virtuoso" @henkderegt.bsky.social)
doi.org/10.1017/cfc....
doi.org/10.1017/cfc....
Thrilled to report that @sarahwiegreffe.bsky.social has accepted an assistant professor position at the University of Maryland. Looking forward to more great NLP explainability and interpretability work.
A bit late to announce, but I’m excited to share that I'll be starting as an assistant professor at UMD CS @univofmaryland.bsky.social this August.
I'll be recruiting PhD students this upcoming cycle for fall 2026. (And if you're a UMD grad student, sign up for my fall seminar!)
I'll be recruiting PhD students this upcoming cycle for fall 2026. (And if you're a UMD grad student, sign up for my fall seminar!)
June 16, 2025 at 6:23 PM
Thrilled to report that @sarahwiegreffe.bsky.social has accepted an assistant professor position at the University of Maryland. Looking forward to more great NLP explainability and interpretability work.