Gunjan Dhanuka — Learning Notes
Search
Search
Dark mode
Light mode
Reader mode
Explorer
deepspeed
2 items with this tag.
Sep 02, 2026
ZeRO (Zero Redundancy Optimizer)
gpu
distributed-training
zero
deepspeed
sharded-data-parallelism
optimizer-state
memory
offloading
Sep 02, 2026
Learn: ZeRO & FSDP
gpu
distributed-training
zero
fsdp
fsdp2
deepspeed
sharded-data-parallelism
mixed-precision
optimizer-state
offloading
collectives