trends.barnas.net

method · first seen 13 Oct 2026

DiLoCo

Also: distributed low-communication training

Local-update distributed training method that synchronises model replicas infrequently to reduce communication.

Mentions per week

0119 May11 Aug10 NovPapersCodeDiscussion

PapersCodeDiscussion

Earliest sources

Latest sources

Mentioned together with

Muon optimizerwhitening-based preconditioningcommunication-computation overlap