Shares of CoreWeave sank 15%, while yields in its bonds blew out further.

The scale of LLM training jobs requires parallelization planning over large GPU clusters. Due to different GPU types and interconnects added over time, these GPU clusters are increasingly heterogen…

Modern cloud platforms increasingly host large-scale deep learning (DL) workloads, demanding high-throughput, low-latency GPU scheduling. However, the growing heterogeneity of GPU clusters and limi…

The increasing scale of large language models (LLMs) necessitates highly efficient collective communication frameworks, particularly as training workloads extend to hundreds of thousands of GPUs. T…

The NVIDIA Collective Communication Library (NCCL) is a critical software layer enabling high-performance collectives on large-scale GPU clusters. Despite being open source with a documented API, i…

Parallel computing with multiple GPUs has become the dominant paradigm for machine learning tasks, especially those of large language models (LLMs). To reduce the latency incurred by inter-GPU comm…

Scheduling All-to-All communications efficiently is fundamental to minimizing job completion times in distributed systems. Incast and straggler flows can slow down All-to-All transfers; and GPU clu…

Maandag zijn bij NorthC Datacenters in Rotterdam de eerste NVIDIA DGX B200 supercomputers van Nederland onthuld door Constantijn van Oranje, Special Envoy bij Techleap. Deze krachtige machines maken deel uit van het nieuwe GPUcluster van AIstartup Juvoly en bieden de rekenkracht om de volgende generatie AImodellen lokaal te ontwikkelen en volledig in eigen land te hosten. Hiermee zet Nederland een belangrijke stap naar een onafhankelijke AIsector.