Researchers at Allen AI have dug into the internals of hybrid language models to understand where they outperform pure transformers — and where they don't. The findings reveal surprising patterns in how different token types are handled across architectures.