#FSDP
FsDP
November 11, 2026 at 12:32 AM
"O sobrenome, do qual se orgulha, também já trouxe questões bem sérias por conta da polarização que foi a última eleição."

Polarização seus fsdp?
November 12, 2026 at 5:48 PM
Transformers v5.23.0 adds Inkling 2 Small, a multimodal MoE model featuring 48 decoder layers routing tokens across 384 experts for 594B total parameters. PyTorch also shipped FSDP fixes for double gradient division in singleton groups during unreplicated expert reductions.
Release Release v5.23.0 · huggingface/transformers
Release v5.23.0 New Model additions Inkling2 Inkling 2 Small is a multimodal model from Thinking Machines Lab that accepts text and image inputs and generates text outputs, and it is intended for ...
github.com
November 13, 2026 at 2:39 PM
Training LLM agents with RL stalls when multi-turn rollouts block on slow environments and GPUs sit idle. AgentRL runs a fully asynchronous GRPO pipeline, with rollout, actor and reference worker pools on Ray and NCCL weight streaming to rollout workers. It builds on SGLang, verl and FSDP,...
November 11, 2026 at 8:44 AM
DMuon transforms matrix optimization in AI training by assigning ownership and reducing communication overhead. This innovation reaches 2.1x throughput of current methods, enhancing efficiency for data-parallel runtimes like PyTorch FSDP and speeding training. https://arxiv.org/abs/2611.07084
DMuon: Scaling Matrix Optimizers in Data Parallelism with Topology-Aware Matrix Ownership and Communication
ArXiv link for DMuon: Scaling Matrix Optimizers in Data Parallelism with Topology-Aware Matrix Ownership and Communication
arxiv.org
November 7, 2026 at 11:00 AM
Want to train PyTorch models on Cloud TPUs without rewriting your code for another framework? PyTorch/XLA from is a Python package that plugs the XLA compiler in as a PyTorch backend. It supports single-process, multi-process and SPMD training, DDP and FSDP, TorchDynamo, and custom kernels via...
November 5, 2026 at 1:56 AM
Cara quem quer apostar nessa merda hoje, se o Copom amanhã reduz a Selic o crédito privado brasileiro vai explodir.

Fsdp estão nivelando Selic com risco².
November 3, 2026 at 11:20 PM
🤖 **Distributed training with Ray Train DLCs and EFA on EKS**

The Ray Train Deep Learning Container ships the full CUDA-through-EFA stack, so you build no image. This post walks through a reproducible sample that fine-tunes Qwen2.5-1.5B with PyTorch FSDP acro...

📰 Source: Artificial […]
Original post on igeek.gamer-geek-news.com
igeek.gamer-geek-news.com
October 30, 2026 at 6:30 PM
Wang, Zhao, Ren, Zhou, Wang, Guan, Zhang, Li, Mo, Ma, Xiong, Guo, Liao, Song: The Endgame of FSDP Communication: Layout-Aware Collectives on Copy Engines, RDMA, and NVSwitch https://arxiv.org/abs/2610.38228 https://arxiv.org/pdf/2610.38228 https://arxiv.org/html/2610.38228
October 30, 2026 at 6:40 AM
Regression-test existing workloads before upgrading.

Transformers v5.20.0 removes cache methods, requires kernels 0.17.2+ and flags expert-parallel slowdowns. LiteLLM v1.107.0-dev.3 fixes cost accounting and streaming; validate proxy behavior before production.

Read more on Kimbodo:
Which AI Library Releases Need Regression Testing Before You Upgrade?
What Happened Transformers v5.20.0 adds seven models, broader quantized export support across Dynamo, ONNX, OpenVINO and ExecuTorch, and checkpoint portability across FSDP and tensor-parallel configurations. It also removes deprecated cache…
kimbodo.com
October 29, 2026 at 7:40 PM
Finetuning an open LLM usually means digging through a tangled framework just to change one training detail. LitGPT from gives you from-scratch implementations of 20+ LLMs, with recipes to pretrain, finetune, evaluate and serve them. It supports LoRA, QLoRA, 4-bit quantization, FSDP and Flash...
October 29, 2026 at 12:27 AM
📰 New article by Sunita Koppar, Kevin Wang, Sukhmeet Singh

Distributed training with Ray Train DLCs and EFA on EKS

#AWS #AI #MachineLearning
Distributed training with Ray Train DLCs and EFA on EKS
The Ray Train Deep Learning Container ships the full CUDA-through-EFA stack, so you build no image. This post walks through a reproducible sample that fine-tunes Qwen2.5-1.5B with PyTorch FSDP across eight GPUs on two EFA-enabled g6.12xlarge instances on Amazon EKS, using the container exactly as published.
aws.amazon.com
October 28, 2026 at 6:21 PM
🤖 **Distributed training with Ray Train DLCs and EFA on EKS**

The Ray Train Deep Learning Container ships the full CUDA-through-EFA stack, so you build no image. This post walks through a reproducible sample that fine-tunes Qwen2.5-1.5B with PyTorch FSDP acro...

📰 Source: Artificial […]
Original post on igeek.gamer-geek-news.com
igeek.gamer-geek-news.com
October 28, 2026 at 6:28 PM
Foda-se a pacificação fsdp da Globo.
October 26, 2026 at 4:32 PM
As fucking 20 horas seus fsdp!
A primeira virada da história! VIVA LULA!
October 25, 2026 at 11:54 PM
Força FsdP!
Dilma Rousseff's Expressive Moment
ALT: Dilma Rousseff's Expressive Moment
static.klipy.com
October 25, 2026 at 10:54 PM
Off-the-shelf embeddings often miss the similarity signal in your own domain data. AnglE, from, is a Python library for training and running BERT- and LLM-based sentence embeddings with angle-optimized loss. It supports contrastive, CoSENT and Espresso losses, single or multi-GPU (FSDP)...
October 23, 2026 at 1:18 PM
#後で読む 用メモです→
フレームワークが動くのを待つなFSDPを書け リリース8日後に397B MoEを学習した基盤を全公開
フレームワークが動くのを待つな、FSDPを書け — リリース8日後に397B MoEを学習した基盤を全公開 - Qiita
はじめに:フレームワークが動けば、たしかに簡単。動けば。 LLMのファインチューニングというと、verl・ms-swift・axolotl・DeepSpeedといった学習フレームワークを使うのが定石とされています。 しかし、日本の共有計算資源(Slurmクラスタ、固定され...
qiita.com
October 17, 2026 at 9:54 PM
NVIDIA Resiliency Extension (NVRx) with Amazon EKS optimizes PyTorch training by reducing checkpoint blocking and fault recovery time. Async checkpointing reached 99%+ efficiency across 16-64 GPUs, while NVRx in-process restart recovered in just 10 seconds per fault, a significant improvement over
Fault tolerant distributed training on Amazon EKS using NVRx | Amazon Web Services
Integrate NVIDIA Resiliency Extension (NVRx) into PyTorch FSDP training on Amazon EKS to overlap checkpoint I/O with training and recover from GPU faults in seconds. This post covers async checkpointing, in-process restart, and ft_launcher in-job restart, with H100 benchmarks at 2 to 8 nodes showing 99%+ training efficiency and second-scale recovery.
aws.amazon.com
October 15, 2026 at 12:39 AM