ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real-World User-AI Conversation #MachineLearning #AI

He never saw it coming x)
#LeagueOfLegends #LoLCommunity #StreamerLife #GamingClips #TwitchMoments #ToxicChat #StreamerWin #LoLMemes #FunnyGaming

Zi Lin, Zihan Wang, Yongqi Tong, Yangkun Wang, Yuxin Guo, Yujia Wang, Jingbo Shang
ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real-World User-AI Conversation. (arXiv:2310.17389v1 [cs.CL])
http://arxiv.org/abs/2310.17389

activating steering primarily for harmful inputs. Experiments using safety benchmarks like ToxicChat & In-The-Wild Jailbreak Prompts demonstrate that our weighted steering controller significantly increases refusal rates compared to the base LLM, [5/7 of https://arxiv.org/abs/2505.20309v1]

Lots of people evaluated gpt-oss-safeguard as well, here's the code and research notes from David Gros trying to replicate its performance on the ToxicChat dataset

github.com/roostorg/mod...