Finally out in TACL:
🌎EWoK (Elements of World Knowledge)🌎: A cognition-inspired framework for evaluating basic world knowledge in language models
tl;dr: LLMs learn basic social concepts way easier than physical&spatial concepts
Paper: direct.mit.edu/tacl/article...
Website: ewok-core.github.io
🌎EWoK (Elements of World Knowledge)🌎: A cognition-inspired framework for evaluating basic world knowledge in language models
tl;dr: LLMs learn basic social concepts way easier than physical&spatial concepts
Paper: direct.mit.edu/tacl/article...
Website: ewok-core.github.io
October 20, 2025 at 5:36 PM
Finally out in TACL:
🌎EWoK (Elements of World Knowledge)🌎: A cognition-inspired framework for evaluating basic world knowledge in language models
tl;dr: LLMs learn basic social concepts way easier than physical&spatial concepts
Paper: direct.mit.edu/tacl/article...
Website: ewok-core.github.io
🌎EWoK (Elements of World Knowledge)🌎: A cognition-inspired framework for evaluating basic world knowledge in language models
tl;dr: LLMs learn basic social concepts way easier than physical&spatial concepts
Paper: direct.mit.edu/tacl/article...
Website: ewok-core.github.io
TACL ❤️
Academics: What's a journal in your field that's great? I feel like they are a dwindling breed. Criteria:
(1) No APC. Supports green or diamond OA.
(2) High-quality and relatively fast peer review.
(3) Editors who provide real curational value and exercise purposeful agency in the decision process.
(1) No APC. Supports green or diamond OA.
(2) High-quality and relatively fast peer review.
(3) Editors who provide real curational value and exercise purposeful agency in the decision process.
July 23, 2026 at 6:18 PM
TACL ❤️
Our TACL paper "Let Guidelines Guide You: A Prescriptive Guideline-Centered Data Annotation Methodology" is finally out, only six months after acceptance! 😅🎉
direct.mit.edu/tacl/article...
direct.mit.edu/tacl/article...
September 22, 2026 at 2:45 PM
Our TACL paper "Let Guidelines Guide You: A Prescriptive Guideline-Centered Data Annotation Methodology" is finally out, only six months after acceptance! 😅🎉
direct.mit.edu/tacl/article...
direct.mit.edu/tacl/article...
The TArgeted Cohesin Loader (TACL) paper was just published. Happy that we were able to contribute to this really exciting project!
If you want to learn how targeting cohesin to defined loci in the genome affects the local chromatin environment and transcription, look no further!
rdcu.be/eLiT5
If you want to learn how targeting cohesin to defined loci in the genome affects the local chromatin environment and transcription, look no further!
rdcu.be/eLiT5
Characterization of induced cohesin loop extrusion trajectories in living cells
Nature Genetics - This study introduces a system called TArgeted Cohesin Loader (TACL) that recruits cohesin complexes at defined genomic regions and induces loop extrusion events in living cells,...
rdcu.be
October 16, 2025 at 8:17 PM
The TArgeted Cohesin Loader (TACL) paper was just published. Happy that we were able to contribute to this really exciting project!
If you want to learn how targeting cohesin to defined loci in the genome affects the local chromatin environment and transcription, look no further!
rdcu.be/eLiT5
If you want to learn how targeting cohesin to defined loci in the genome affects the local chromatin environment and transcription, look no further!
rdcu.be/eLiT5
Three for you
Computational Linguistics
TACL
NEJLT
Computational Linguistics
TACL
NEJLT
Academics: What's a journal in your field that's great? I feel like they are a dwindling breed. Criteria:
(1) No APC. Supports green or diamond OA.
(2) High-quality and relatively fast peer review.
(3) Editors who provide real curational value and exercise purposeful agency in the decision process.
(1) No APC. Supports green or diamond OA.
(2) High-quality and relatively fast peer review.
(3) Editors who provide real curational value and exercise purposeful agency in the decision process.
July 23, 2026 at 10:53 AM
Three for you
Computational Linguistics
TACL
NEJLT
Computational Linguistics
TACL
NEJLT
I am becoming an editor (EiC) at TACL (Transactions of ACL) journal. Not what I expected to be doing in retirement, but I am a strong believer in journals as the best way to present research, and I want to help TACL become an even better venue for publishing NLP research.
June 29, 2026 at 8:31 AM
I am becoming an editor (EiC) at TACL (Transactions of ACL) journal. Not what I expected to be doing in retirement, but I am a strong believer in journals as the best way to present research, and I want to help TACL become an even better venue for publishing NLP research.
Prof. Vivian Yun-Nung Chen has been appointed as the fourth TACL EiC 🎉
January 25, 2026 at 12:03 AM
Prof. Vivian Yun-Nung Chen has been appointed as the fourth TACL EiC 🎉
The final version of our survey on conformal prediction for NLP has now been published in TACL: direct.mit.edu/tacl/article...
Conformal Prediction for Natural Language Processing: A Survey
Abstract. The rapid proliferation of large language models and natural language processing (NLP) applications creates a crucial need for uncertainty quantification to mitigate risks such as Hallucinat...
direct.mit.edu
December 6, 2024 at 9:05 PM
The final version of our survey on conformal prediction for NLP has now been published in TACL: direct.mit.edu/tacl/article...
Tacl da gan Alfie Merritt.
1-0
1-0
October 6, 2026 at 6:44 PM
Tacl da gan Alfie Merritt.
1-0
1-0
There’s plenty of evidence for political bias in LLMs, but very few evals reflect realistic LLM use cases — which is where bias actually matters.
IssueBench, our attempt to fix this, is accepted at TACL, and I will be at #EMNLP2025 next week to talk about it!
New results 🧵
IssueBench, our attempt to fix this, is accepted at TACL, and I will be at #EMNLP2025 next week to talk about it!
New results 🧵
Are LLMs biased when they write about political issues?
We just released IssueBench – the largest, most realistic benchmark of its kind – to answer this question more robustly than ever before.
Long 🧵with spicy results 👇
We just released IssueBench – the largest, most realistic benchmark of its kind – to answer this question more robustly than ever before.
Long 🧵with spicy results 👇
October 29, 2025 at 4:12 PM
There’s plenty of evidence for political bias in LLMs, but very few evals reflect realistic LLM use cases — which is where bias actually matters.
IssueBench, our attempt to fix this, is accepted at TACL, and I will be at #EMNLP2025 next week to talk about it!
New results 🧵
IssueBench, our attempt to fix this, is accepted at TACL, and I will be at #EMNLP2025 next week to talk about it!
New results 🧵
🤔Do Vision and Language Models Share Concepts? 🚀
We present an empirical evaluation and find that language models partially converge towards representations isomorphic to those of vision models. #EMNLP
📃 direct.mit.edu/tacl/article...
We present an empirical evaluation and find that language models partially converge towards representations isomorphic to those of vision models. #EMNLP
📃 direct.mit.edu/tacl/article...
November 19, 2024 at 12:48 PM
🤔Do Vision and Language Models Share Concepts? 🚀
We present an empirical evaluation and find that language models partially converge towards representations isomorphic to those of vision models. #EMNLP
📃 direct.mit.edu/tacl/article...
We present an empirical evaluation and find that language models partially converge towards representations isomorphic to those of vision models. #EMNLP
📃 direct.mit.edu/tacl/article...
Champion's Tokon Assemble is just him 💀. See more #MarvelTokon on our YouTube: www.youtube.com/watch?v=tAcl... #FGC
August 6, 2026 at 11:08 PM
Champion's Tokon Assemble is just him 💀. See more #MarvelTokon on our YouTube: www.youtube.com/watch?v=tAcl... #FGC
Excited that the final version of our paper on filtered corpus training is out now in TACL! (Original thread is on the bad site, but the paper's title is a succinct summary.)
doi.org/10.1162/tacl...
Always a pleasure to work with Abhinav Patil and @jumelet.bsky.social , who deserve the real credit
doi.org/10.1162/tacl...
Always a pleasure to work with Abhinav Patil and @jumelet.bsky.social , who deserve the real credit
Filtered Corpus Training (FiCT) Shows that Language Models Can Generalize from Indirect Evidence
Abstract. This paper introduces Filtered Corpus Training, a method that trains language models (LMs) on corpora with certain linguistic constructions filtered out from the training data, and uses it t...
doi.org
December 2, 2024 at 7:10 PM
Excited that the final version of our paper on filtered corpus training is out now in TACL! (Original thread is on the bad site, but the paper's title is a succinct summary.)
doi.org/10.1162/tacl...
Always a pleasure to work with Abhinav Patil and @jumelet.bsky.social , who deserve the real credit
doi.org/10.1162/tacl...
Always a pleasure to work with Abhinav Patil and @jumelet.bsky.social , who deserve the real credit
New work to appear @ TACL!
Language models (LMs) are remarkably good at generating novel well-formed sentences, leading to claims that they have mastered grammar.
Yet they often assign higher probability to ungrammatical strings than to grammatical strings.
How can both things be true? 🧵👇
Language models (LMs) are remarkably good at generating novel well-formed sentences, leading to claims that they have mastered grammar.
Yet they often assign higher probability to ungrammatical strings than to grammatical strings.
How can both things be true? 🧵👇
November 10, 2025 at 10:11 PM
New work to appear @ TACL!
Language models (LMs) are remarkably good at generating novel well-formed sentences, leading to claims that they have mastered grammar.
Yet they often assign higher probability to ungrammatical strings than to grammatical strings.
How can both things be true? 🧵👇
Language models (LMs) are remarkably good at generating novel well-formed sentences, leading to claims that they have mastered grammar.
Yet they often assign higher probability to ungrammatical strings than to grammatical strings.
How can both things be true? 🧵👇
🚨 Two Papers @aclmeeting.bsky.social! 🚨
🎉 1×ACL Main + 1×TACL
🎥 What Is That Talk About? A Video-to-Text Summarization Dataset for Scientific Presentations (ACL)
🧠 Explanatory Summarization with Discourse-Driven Planning (TACL)
Thanks to my amazing co-authors! 🙏
#LLMs #NLProc #TACL #ACL #ACL2025
🎉 1×ACL Main + 1×TACL
🎥 What Is That Talk About? A Video-to-Text Summarization Dataset for Scientific Presentations (ACL)
🧠 Explanatory Summarization with Discourse-Driven Planning (TACL)
Thanks to my amazing co-authors! 🙏
#LLMs #NLProc #TACL #ACL #ACL2025
May 21, 2025 at 8:37 AM
@sarapapi.bsky.social presented her TACL paper: “How real is your real-time simultaneous speech-to-text translation system?”
👉 aclanthology.org/2025.tacl-1.14/
(2/6)
👉 aclanthology.org/2025.tacl-1.14/
(2/6)
August 2, 2025 at 4:32 PM
@sarapapi.bsky.social presented her TACL paper: “How real is your real-time simultaneous speech-to-text translation system?”
👉 aclanthology.org/2025.tacl-1.14/
(2/6)
👉 aclanthology.org/2025.tacl-1.14/
(2/6)
Getting back to my South Florida roots at #EMNLP2024 this week, say hi. Tues: @kanishka.bsky.social on how LLMs learn a rare construction and a posters on his work on property inheritance. Wednesday: Ask-a-Philosopher! @harveylederman.bsky.social and I will be presenting our TACL philosophy paper.
November 10, 2024 at 11:26 PM
Getting back to my South Florida roots at #EMNLP2024 this week, say hi. Tues: @kanishka.bsky.social on how LLMs learn a rare construction and a posters on his work on property inheritance. Wednesday: Ask-a-Philosopher! @harveylederman.bsky.social and I will be presenting our TACL philosophy paper.
Wrapping up my oral presentations today with our TACL paper "QE4PE: Quality Estimation for Human Post-editing" at the Interpretability morning session #EMNLP2025 (Room A104, 11:45 China time)!
Paper: arxiv.org/abs/2503.03044
Slides/video/poster: underline.io/lecture/1315...
Paper: arxiv.org/abs/2503.03044
Slides/video/poster: underline.io/lecture/1315...
November 7, 2025 at 2:50 AM
Wrapping up my oral presentations today with our TACL paper "QE4PE: Quality Estimation for Human Post-editing" at the Interpretability morning session #EMNLP2025 (Room A104, 11:45 China time)!
Paper: arxiv.org/abs/2503.03044
Slides/video/poster: underline.io/lecture/1315...
Paper: arxiv.org/abs/2503.03044
Slides/video/poster: underline.io/lecture/1315...
Working on RAG? Check out our ClapNQ benchmark (accepted to TACL) to test the full RAG pipeline!
github.com/primeqa/clapnq
arxiv.org/abs/2404.02103
github.com/primeqa/clapnq
arxiv.org/abs/2404.02103
GitHub - primeqa/clapnq
Contribute to primeqa/clapnq development by creating an account on GitHub.
github.com
November 19, 2024 at 2:49 AM
Working on RAG? Check out our ClapNQ benchmark (accepted to TACL) to test the full RAG pipeline!
github.com/primeqa/clapnq
arxiv.org/abs/2404.02103
github.com/primeqa/clapnq
arxiv.org/abs/2404.02103
With only a week left for #EMNLP2025, we are happy to announce all the works we 🐮 will present 🥳 - come and say "hi" to our posters and presentations during the Main and the co-located events (*SEM and workshops) See you in Suzhou ✈️
October 27, 2025 at 11:54 AM
With only a week left for #EMNLP2025, we are happy to announce all the works we 🐮 will present 🥳 - come and say "hi" to our posters and presentations during the Main and the co-located events (*SEM and workshops) See you in Suzhou ✈️
Great thread!
Was telling my 17 year old son who’s been testing the coding waters about how I used/programmed TACL, VFP and SQL, C++, etc (he joked from the 1900’s, mom 😜), and I asked (dared) him to create a simple decision tree. It was hilarious, and now he’s looking at old school tutorials. 🙂
Was telling my 17 year old son who’s been testing the coding waters about how I used/programmed TACL, VFP and SQL, C++, etc (he joked from the 1900’s, mom 😜), and I asked (dared) him to create a simple decision tree. It was hilarious, and now he’s looking at old school tutorials. 🙂
November 25, 2025 at 6:09 AM
Great thread!
Was telling my 17 year old son who’s been testing the coding waters about how I used/programmed TACL, VFP and SQL, C++, etc (he joked from the 1900’s, mom 😜), and I asked (dared) him to create a simple decision tree. It was hilarious, and now he’s looking at old school tutorials. 🙂
Was telling my 17 year old son who’s been testing the coding waters about how I used/programmed TACL, VFP and SQL, C++, etc (he joked from the 1900’s, mom 😜), and I asked (dared) him to create a simple decision tree. It was hilarious, and now he’s looking at old school tutorials. 🙂
issues w preference LM benchmarks:
🐡data contains cases where the "bad" response is just as good as chosen one
🐟model rankings can feel off (claude ranks lower than expected)
led by @cmalaviya.bsky.social, we study underspecified queries & detrimental effect on model evals; accepted to TACL 2025
🐡data contains cases where the "bad" response is just as good as chosen one
🐟model rankings can feel off (claude ranks lower than expected)
led by @cmalaviya.bsky.social, we study underspecified queries & detrimental effect on model evals; accepted to TACL 2025
In our new paper, “Contextualized Evaluations: Judging Language Model Responses to Underspecified Queries,” we find that adding just a bit of missing context can reorder model leaderboards—and surface hidden biases. 🧵👇
July 22, 2025 at 5:02 PM
issues w preference LM benchmarks:
🐡data contains cases where the "bad" response is just as good as chosen one
🐟model rankings can feel off (claude ranks lower than expected)
led by @cmalaviya.bsky.social, we study underspecified queries & detrimental effect on model evals; accepted to TACL 2025
🐡data contains cases where the "bad" response is just as good as chosen one
🐟model rankings can feel off (claude ranks lower than expected)
led by @cmalaviya.bsky.social, we study underspecified queries & detrimental effect on model evals; accepted to TACL 2025
LMs that "know more" about toxicity are less toxic!
Our #TACL 📄 connects behavior and internals:
💠 LMs amplify toxicity beyond humans
💠 Information about toxicity peaks in lower layers
💠 Bypassing these layers increases toxicity
More details👇 #NLProc #interpretability (1/🧵)
Our #TACL 📄 connects behavior and internals:
💠 LMs amplify toxicity beyond humans
💠 Information about toxicity peaks in lower layers
💠 Bypassing these layers increases toxicity
More details👇 #NLProc #interpretability (1/🧵)
January 27, 2026 at 1:01 PM
LMs that "know more" about toxicity are less toxic!
Our #TACL 📄 connects behavior and internals:
💠 LMs amplify toxicity beyond humans
💠 Information about toxicity peaks in lower layers
💠 Bypassing these layers increases toxicity
More details👇 #NLProc #interpretability (1/🧵)
Our #TACL 📄 connects behavior and internals:
💠 LMs amplify toxicity beyond humans
💠 Information about toxicity peaks in lower layers
💠 Bypassing these layers increases toxicity
More details👇 #NLProc #interpretability (1/🧵)
Belated, but still happy to see our paper (with @drhanjones.bsky.social) on fleeting memory transformers is out in TACL!
We find that giving language models human-like memory decay *improves* language learning, while, unexpectedly, impairing human reading time prediction
Follow up results soon!
We find that giving language models human-like memory decay *improves* language learning, while, unexpectedly, impairing human reading time prediction
Follow up results soon!
July 20, 2026 at 9:41 AM
Belated, but still happy to see our paper (with @drhanjones.bsky.social) on fleeting memory transformers is out in TACL!
We find that giving language models human-like memory decay *improves* language learning, while, unexpectedly, impairing human reading time prediction
Follow up results soon!
We find that giving language models human-like memory decay *improves* language learning, while, unexpectedly, impairing human reading time prediction
Follow up results soon!
