Early-Signal Detector

method · first seen 28 Oct 2026

fully sharded data parallelism

Also: FSDP, ZeRO, FSDP2, sharded data parallelism

Distributed training that shards model parameters, gradients, and optimizer states across devices.

Mentions per week

0219 May11 Aug10 NovPapersCodeDiscussion

PapersCodeDiscussion

Earliest sources

Latest sources

Mentioned together with

Muon optimizerNVSwitch multicastcopy engines