Meet the honorees at https://pytorch.org/blog/2026-pytorch-foundation-contributor-awards/
Meet the honorees at https://pytorch.org/blog/2026-pytorch-foundation-contributor-awards/
https://deployedbyai.com/log/7bd7e718dc16/deepspeed-v0-20-0/?k=br
Follow for one line every time AI actually ships, as it ships, 24/7.
https://deployedbyai.com/log/7bd7e718dc16/deepspeed-v0-20-0/?k=br
Follow for one line every time AI actually ships, as it ships, 24/7.
https://www.youtube.com/playlist?list=PLVqjs87Mef-U
https://www.youtube.com/playlist?list=PLVqjs87Mef-U
Our SRE guide to Megatron-DeepSpeed 3D Parallelism covers TP/PP/DP topology math, ZeRO-2 vs ZeRO-3 trade-offs, NCCL RoCEv2 tuning, and Slurm scripts:
www.servermo.com/howto/setup-...
#DeepSpeed #MegatronLM #AI #MLOps
Our SRE guide to Megatron-DeepSpeed 3D Parallelism covers TP/PP/DP topology math, ZeRO-2 vs ZeRO-3 trade-offs, NCCL RoCEv2 tuning, and Slurm scripts:
www.servermo.com/howto/setup-...
#DeepSpeed #MegatronLM #AI #MLOps
9 AM PT / 12 PM ET
https://zoom-lfx.platform.linuxfoundation.org/meeting/93902569995?password=7d9c4fc9-3efa-4715-88f0-df8a6deb008b
9 AM PT / 12 PM ET
https://zoom-lfx.platform.linuxfoundation.org/meeting/93902569995?password=7d9c4fc9-3efa-4715-88f0-df8a6deb008b
https://events.linuxfoundation.org/pytorch-conference-north-america/program/keynote-livestream/
Talks and workshops across PyTorch, vLLM, DeepSpeed, Ray, Helion, and Safetensors.
https://events.linuxfoundation.org/pytorch-conference-north-america/program/keynote-livestream/
Talks and workshops across PyTorch, vLLM, DeepSpeed, Ray, Helion, and Safetensors.
フレームワークが動くのを待つなFSDPを書け リリース8日後に397B MoEを学習した基盤を全公開
フレームワークが動くのを待つなFSDPを書け リリース8日後に397B MoEを学習した基盤を全公開
At #PyTorchCon North America 2026, Masahiro Tanaka will cover tensor, sequence, and expert parallelism in DeepSpeed and walk through benchmarks.
https://hubs.la/Q04v4SL60
At #PyTorchCon North America 2026, Masahiro Tanaka will cover tensor, sequence, and expert parallelism in DeepSpeed and walk through benchmarks.
https://hubs.la/Q04v4SL60
Higgsfield is an open-source GPU orchestration framework that manages multi-node, fault-tolerant training for trillion-parameter models by coordinating resources, supporting deepspeed/PyTorch sharding, and integrating with GitHub for CI/CD, all aimed at streamlined distributed LLM (1/2)
Higgsfield is an open-source GPU orchestration framework that manages multi-node, fault-tolerant training for trillion-parameter models by coordinating resources, supporting deepspeed/PyTorch sharding, and integrating with GitHub for CI/CD, all aimed at streamlined distributed LLM (1/2)
Explore PyTorch, vLLM, DeepSpeed, Ray, Helion, and Safetensors. CFP open through Sept. 27.
https://pytorch.org/blog/pytorch-day-japan-2026-comes-to-tokyo/
Explore PyTorch, vLLM, DeepSpeed, Ray, Helion, and Safetensors. CFP open through Sept. 27.
https://pytorch.org/blog/pytorch-day-japan-2026-comes-to-tokyo/
implicit-none.com/en/deepspeed...
implicit-none.com/en/deepspeed...
Qwen2.5-0.5B over 20 steps on 2x H100: 15.76 s on a single card, 35.19 s with ZeRO-2, 49.99 s with ZeRO-3.
ZeRO earns its place at 7B, not at 0.5B.
https://massedcompute.com/deepspeed-multi-gpu-training-cloud/
Qwen2.5-0.5B over 20 steps on 2x H100: 15.76 s on a single card, 35.19 s with ZeRO-2, 49.99 s with ZeRO-3.
ZeRO earns its place at 7B, not at 0.5B.
https://massedcompute.com/deepspeed-multi-gpu-training-cloud/
At #PyTorchCon NA 2025, Richard Liaw spoke about “how important Ray can be for the stack.”
Register by Sept. 4: https://hubs.la/Q04tDf0j0
#PyTorchCon
At #PyTorchCon NA 2025, Richard Liaw spoke about “how important Ray can be for the stack.”
Register by Sept. 4: https://hubs.la/Q04tDf0j0
#PyTorchCon
On this superior DeepSpeed tutorial, we offer a hands-on walkthrough of cutting-edge optimization strategies for coaching massive language fashions effectively. By combining ZeRO…
On this superior DeepSpeed tutorial, we offer a hands-on walkthrough of cutting-edge optimization strategies for coaching massive language fashions effectively. By combining ZeRO…
In this advanced DeepSpeed tutorial, we provide a hands-on walkthrough of cutting-edge optimization techniques for training large language models efficiently. By combining ZeRO…
In this advanced DeepSpeed tutorial, we provide a hands-on walkthrough of cutting-edge optimization techniques for training large language models efficiently. By combining ZeRO…
The DeepSpeed workforce unveiled ZenFlow, a brand new offloading engine designed to beat a significant bottleneck in giant language mannequin (LLM) coaching: CPU-induced GPU…
The DeepSpeed workforce unveiled ZenFlow, a brand new offloading engine designed to beat a significant bottleneck in giant language mannequin (LLM) coaching: CPU-induced GPU…
The DeepSpeed team unveiled ZenFlow, a new offloading engine designed to overcome a major bottleneck in large language model (LLM) training: CPU-induced GPU stalls. While…
The DeepSpeed team unveiled ZenFlow, a new offloading engine designed to overcome a major bottleneck in large language model (LLM) training: CPU-induced GPU stalls. While…
- GPT-NeoX
- Megatron-LM
- Megatron-Deepspeed
- TinyLlama
- Others
- GPT-NeoX
- Megatron-LM
- Megatron-Deepspeed
- TinyLlama
- Others