New arXiv paper introduces Batch Cost Lookahead (BCL), a coordination-free data-loading policy that balances per-rank batch costs in synchronous data-parallel training. Across 13 NeMo-RL SFT corpora on eight…

#GPUs #DistributedTraining #AIInfrastructure #NeMoRL
https://arxiv.org/abs/2610.21162