Fix zero stage2 cpu_offload bug when some model trainable parameters skipped in training#2

Open

ghosthamlet wants to merge 19 commits intomasterfrom ghosthamlet-stage2-bug

+1,805-608

Commits on Mar 18, 2021

consistent checkpoint filenaming (microsoft#865 )

stas00
and
tjruwase
authored
[doc] launcher (microsoft#868 )

stas00
and
jeffra
authored

Commits on Mar 24, 2021

[doc] pipeline (microsoft#888 )
stas00
authored

Commits on Mar 25, 2021

[debug utils] see_memory_usage fixes (microsoft#890 )
stas00
authored

Commits on Mar 26, 2021

full fp32 weights reconstruction for zero 2+3 (microsoft#892 )
stas00
authored

Commits on Mar 27, 2021