TextArena is live on arXiv❗
We present a benchmark of 57+ competitive text-based games to evaluate&train LLMs
including negotiation, deception, theory of mind...
Multiplayer support
Human-vs-models
Model-vs-model
Perfect for social interaction, Multi-Agent, multi-turn reasoning and Planning
🤖📈
We present a benchmark of 57+ competitive text-based games to evaluate&train LLMs
including negotiation, deception, theory of mind...
Multiplayer support
Human-vs-models
Model-vs-model
Perfect for social interaction, Multi-Agent, multi-turn reasoning and Planning
🤖📈
April 16, 2025 at 11:44 AM
TextArena is live on arXiv❗
We present a benchmark of 57+ competitive text-based games to evaluate&train LLMs
including negotiation, deception, theory of mind...
Multiplayer support
Human-vs-models
Model-vs-model
Perfect for social interaction, Multi-Agent, multi-turn reasoning and Planning
🤖📈
We present a benchmark of 57+ competitive text-based games to evaluate&train LLMs
including negotiation, deception, theory of mind...
Multiplayer support
Human-vs-models
Model-vs-model
Perfect for social interaction, Multi-Agent, multi-turn reasoning and Planning
🤖📈
Will the same model become a different player depending on the interface language?
We report our findings in our paper "Skill Issue: Are Skills Language-Invariant in LLMs?".
And with it, TextArena is now multilingual 🌍 with 65 games in 193 languages. 🧵
We report our findings in our paper "Skill Issue: Are Skills Language-Invariant in LLMs?".
And with it, TextArena is now multilingual 🌍 with 65 games in 193 languages. 🧵
August 28, 2026 at 5:32 PM
Will the same model become a different player depending on the interface language?
We report our findings in our paper "Skill Issue: Are Skills Language-Invariant in LLMs?".
And with it, TextArena is now multilingual 🌍 with 65 games in 193 languages. 🧵
We report our findings in our paper "Skill Issue: Are Skills Language-Invariant in LLMs?".
And with it, TextArena is now multilingual 🌍 with 65 games in 193 languages. 🧵
In TextArena models can compete against other models and humans in over 70 text-based games. Results are updated in real-time and shown on textarena.ai/leaderboard. (You can play against the models here: textarena.ai)
TextArena
An AI-powered text adventure platform
textarena.ai
April 16, 2025 at 11:44 AM
In TextArena models can compete against other models and humans in over 70 text-based games. Results are updated in real-time and shown on textarena.ai/leaderboard. (You can play against the models here: textarena.ai)
Can LLMs learn social skills by playing games?
A blogpost on human-model interaction, games, training and testing LLMs
research.ibm.com/blog/LLM-soc...
🤖📈🧠
A blogpost on human-model interaction, games, training and testing LLMs
research.ibm.com/blog/LLM-soc...
🤖📈🧠
Can LLMs learn social skills by playing games?
A new open-source framework, TextArena, pits large language models against each other in competitive environments designed to test and improve their communication skills.
research.ibm.com
July 24, 2025 at 2:16 PM
Can LLMs learn social skills by playing games?
A blogpost on human-model interaction, games, training and testing LLMs
research.ibm.com/blog/LLM-soc...
🤖📈🧠
A blogpost on human-model interaction, games, training and testing LLMs
research.ibm.com/blog/LLM-soc...
🤖📈🧠
arXiv: https://arxiv.org/abs/2608.25832
alphaXiv: https://alphaxiv.org/abs/2608.25832
HF Paper: https://huggingface.co/papers/2608.25832
Code: https://github.com/TextArena/TextArena
alphaXiv: https://alphaxiv.org/abs/2608.25832
HF Paper: https://huggingface.co/papers/2608.25832
Code: https://github.com/TextArena/TextArena
August 28, 2026 at 5:32 PM
til about www.textarena.ai
TextArena
An AI-powered text adventure platform
www.textarena.ai
June 16, 2025 at 11:38 AM
til about www.textarena.ai
Recently I joined Bobby Cheng, Zhengyuan liu, @catherinearnett.bsky.social & @lchoshen.bsky.social in multilingual TextArena. catch up on what we're doing x.com/bobbycxy/sta...
Here I focus on an analysis of 2D board games checking where exactly do the language performance gaps originate🔍
Here I focus on an analysis of 2D board games checking where exactly do the language performance gaps originate🔍
Bobby (@bobbycxy) on X
Games break that circularity as 1) the game board has the same tokens in every language, and 2) the optimal move is identical. So nothing about the skill it uses is linguistic.
We took a few models, ...
x.com
July 13, 2026 at 9:10 AM
Recently I joined Bobby Cheng, Zhengyuan liu, @catherinearnett.bsky.social & @lchoshen.bsky.social in multilingual TextArena. catch up on what we're doing x.com/bobbycxy/sta...
Here I focus on an analysis of 2D board games checking where exactly do the language performance gaps originate🔍
Here I focus on an analysis of 2D board games checking where exactly do the language performance gaps originate🔍
Leon Guertler, Bobby Cheng, Simon Yu, Bo Liu, Leshem Choshen, Cheston Tan
TextArena
https://arxiv.org/abs/2504.11442
TextArena
https://arxiv.org/abs/2504.11442
April 16, 2025 at 4:26 AM
Leon Guertler, Bobby Cheng, Simon Yu, Bo Liu, Leshem Choshen, Cheston Tan
TextArena
https://arxiv.org/abs/2504.11442
TextArena
https://arxiv.org/abs/2504.11442
This is a collaborative effort of many people, we share it openly to hear from you, to get feedback, for you to build upon us, or to build with us
Talk to us:
discord.gg/KMndsqwMaZ
git:
github.com/LeonGuertler...
Talk to us:
discord.gg/KMndsqwMaZ
git:
github.com/LeonGuertler...
Join the TextArena Discord Server!
Check out the TextArena community on Discord - hang out with 133 other members and enjoy free voice and text chat.
discord.gg
June 24, 2025 at 4:45 PM
This is a collaborative effort of many people, we share it openly to hear from you, to get feedback, for you to build upon us, or to build with us
Talk to us:
discord.gg/KMndsqwMaZ
git:
github.com/LeonGuertler...
Talk to us:
discord.gg/KMndsqwMaZ
git:
github.com/LeonGuertler...
Oops sorry
arxiv.org/abs/2504.11442
More analysis and fuller results soon (this is more about the framework itself, the next more traditional conference submission)
arxiv.org/abs/2504.11442
More analysis and fuller results soon (this is more about the framework itself, the next more traditional conference submission)
TextArena
TextArena is an open-source collection of competitive text-based games for training and evaluation of agentic behavior in Large Language Models (LLMs). It spans 57+ unique environments (including sing...
arxiv.org
April 16, 2025 at 2:23 PM
Oops sorry
arxiv.org/abs/2504.11442
More analysis and fuller results soon (this is more about the framework itself, the next more traditional conference submission)
arxiv.org/abs/2504.11442
More analysis and fuller results soon (this is more about the framework itself, the next more traditional conference submission)
textArena code is simple and similar so the second shouldn't be an issue. The first makes sense. (Makes for more traditional games though if you want traditional models to play them)
May 14, 2025 at 5:26 PM
textArena code is simple and similar so the second shouldn't be an issue. The first makes sense. (Makes for more traditional games though if you want traditional models to play them)
But if we talk about evaluating current goals. We need evaluation for pretraining, we need to compare models across sizes (like textarena.ai ) and why not even compare models of different languages on comparable skills?
TextArena
An AI-powered text adventure platform
textArena.ai
August 5, 2025 at 11:40 AM
But if we talk about evaluating current goals. We need evaluation for pretraining, we need to compare models across sizes (like textarena.ai ) and why not even compare models of different languages on comparable skills?
#bluelz #games #KI #AI #Lernen #training #Spiel
"TextArena - A library for training and evaluation of language models in competitive text-based environments [(games)]"
textarena.ai
"TextArena - A library for training and evaluation of language models in competitive text-based environments [(games)]"
textarena.ai
A library for training and evaluation of language models in competitive text-based environments.
textarena.ai
February 2, 2025 at 10:06 AM
All of this runs on Multilingual TextArena: 65 games (>60% of the collection) in 193 languages, MIT licensed. 8 languages verified by native speakers, the rest via tiered pipelines with per-language fidelity scores.
Each player can get its own language (e.g. player 0 in
Each player can get its own language (e.g. player 0 in
August 28, 2026 at 5:32 PM
All of this runs on Multilingual TextArena: 65 games (>60% of the collection) in 193 languages, MIT licensed. 8 languages verified by native speakers, the rest via tiered pipelines with per-language fidelity scores.
Each player can get its own language (e.g. player 0 in
Each player can get its own language (e.g. player 0 in
🎮 TextArena Launches on Hugging Face
An open-source suite of 57+ games designed to test LLM behaviors: negotiation, deception, and more.
Basically, it’s an AI testing ground for agentic skills.
Read more: lnkd.in/gajaR6FC
An open-source suite of 57+ games designed to test LLM behaviors: negotiation, deception, and more.
Basically, it’s an AI testing ground for agentic skills.
Read more: lnkd.in/gajaR6FC
April 22, 2025 at 4:44 PM
🎮 TextArena Launches on Hugging Face
An open-source suite of 57+ games designed to test LLM behaviors: negotiation, deception, and more.
Basically, it’s an AI testing ground for agentic skills.
Read more: lnkd.in/gajaR6FC
An open-source suite of 57+ games designed to test LLM behaviors: negotiation, deception, and more.
Basically, it’s an AI testing ground for agentic skills.
Read more: lnkd.in/gajaR6FC
I am thinking a lot about pretraining in academia, we almost had something collaborative with textarena.ai (but then it became so popular we didn't have time...)
BabyLM will be a workshop and another pretraining competition, do join!
bsky.app/profile/lcho...
BabyLM will be a workshop and another pretraining competition, do join!
bsky.app/profile/lcho...
April 18, 2025 at 4:33 PM
I am thinking a lot about pretraining in academia, we almost had something collaborative with textarena.ai (but then it became so popular we didn't have time...)
BabyLM will be a workshop and another pretraining competition, do join!
bsky.app/profile/lcho...
BabyLM will be a workshop and another pretraining competition, do join!
bsky.app/profile/lcho...
We kept the interface for TextArena as similar to the OpenAI gym as possible to make it easy to use
Specifically, we adopt a similar framework of stackable wrappers and an environment registry
Here is code example of how you can let GPT-4o-mini and Calude-3.5-haiku compete against each other offline
Specifically, we adopt a similar framework of stackable wrappers and an environment registry
Here is code example of how you can let GPT-4o-mini and Calude-3.5-haiku compete against each other offline
April 16, 2025 at 11:44 AM
We kept the interface for TextArena as similar to the OpenAI gym as possible to make it easy to use
Specifically, we adopt a similar framework of stackable wrappers and an environment registry
Here is code example of how you can let GPT-4o-mini and Calude-3.5-haiku compete against each other offline
Specifically, we adopt a similar framework of stackable wrappers and an environment registry
Here is code example of how you can let GPT-4o-mini and Calude-3.5-haiku compete against each other offline
Hello world, I'm making the links shared outside of discord, let's see how that works: https://x.com/LChoshen Hello world, I'm making the links shared outside of discord, let's see how that works: https://x.com/LChoshen Hello world, I'm making the links shared outside of discord, let's see
Leshem (Legend) Choshen 🤖🤗 @ACL @ICML (@LChoshen) on X
🥇 LLMs together (co-created model merging, BabyLM, textArena)
🥈 Spreading science over hype in #ML & #NLP
shareLM💬 Donor
@IBMResearch & @MIT
x.com
June 30, 2026 at 1:56 PM
Hello world, I'm making the links shared outside of discord, let's see how that works: https://x.com/LChoshen Hello world, I'm making the links shared outside of discord, let's see how that works: https://x.com/LChoshen Hello world, I'm making the links shared outside of discord, let's see
One last option, we won't need to pay for data even then, instead we will invent new ways of gathering data, synthetically (with games like textarena.ai ?) or by models gathering and creating their own data by interacting with the world?
TextArena
An AI-powered text adventure platform
textArena.ai
September 12, 2025 at 2:20 PM
One last option, we won't need to pay for data even then, instead we will invent new ways of gathering data, synthetically (with games like textarena.ai ?) or by models gathering and creating their own data by interacting with the world?
Why not textArena 😔
May 14, 2025 at 10:53 AM
Why not textArena 😔
Specifically, generating random numbers, playing rock paper scissors (I can tell firsthand that models are far from random and IIRC like rock as their first move textarena.ai/environments... )
all of those decrease after instruction tuning and RLHF
all of those decrease after instruction tuning and RLHF
TextArena
An AI-powered text adventure platform
textarena.ai
May 8, 2025 at 12:03 PM
Specifically, generating random numbers, playing rock paper scissors (I can tell firsthand that models are far from random and IIRC like rock as their first move textarena.ai/environments... )
all of those decrease after instruction tuning and RLHF
all of those decrease after instruction tuning and RLHF
I already mentioned improving multilingual/cultural support, but there is so much to it. We are looking for pretraining and evals in other languages and figuring what can be improved (a lot of options to help and join, like textArena those are community projects)
Stay tuned
Stay tuned
April 18, 2025 at 4:33 PM
I already mentioned improving multilingual/cultural support, but there is so much to it. We are looking for pretraining and evals in other languages and figuring what can be improved (a lot of options to help and join, like textArena those are community projects)
Stay tuned
Stay tuned
But is that all the interactivity and adaptation we look for? Of course, not,
@babyLMchallenge
offers an interactivity track and a workshop submission space
textarena.ai offers a platform to play with models, test models offline or online with others, or invent games.
@babyLMchallenge
offers an interactivity track and a workshop submission space
textarena.ai offers a platform to play with models, test models offline or online with others, or invent games.
TextArena
An AI-powered text adventure platform
textarena.ai
April 18, 2025 at 4:33 PM
But is that all the interactivity and adaptation we look for? Of course, not,
@babyLMchallenge
offers an interactivity track and a workshop submission space
textarena.ai offers a platform to play with models, test models offline or online with others, or invent games.
@babyLMchallenge
offers an interactivity track and a workshop submission space
textarena.ai offers a platform to play with models, test models offline or online with others, or invent games.