Skip to content

Latest commit

 

History

History
executable file
·
28 lines (21 loc) · 792 Bytes

optimizers.rst

File metadata and controls

executable file
·
28 lines (21 loc) · 792 Bytes

Optimizers

DeepSpeed offers high-performance implementations of Adam optimizer on CPU; FusedAdam, FusedLamb, OnebitAdam, OnebitLamb optimizers on GPU.

Adam (CPU)

.. autoclass:: deepspeed.ops.adam.DeepSpeedCPUAdam

FusedAdam (GPU)

.. autoclass:: deepspeed.ops.adam.FusedAdam

FusedLamb (GPU)

.. autoclass:: deepspeed.ops.lamb.FusedLamb

OneBitAdam (GPU)

.. autoclass:: deepspeed.runtime.fp16.onebit.adam.OnebitAdam

ZeroOneAdam (GPU)

.. autoclass:: deepspeed.runtime.fp16.onebit.zoadam.ZeroOneAdam

OnebitLamb (GPU)

.. autoclass:: deepspeed.runtime.fp16.onebit.lamb.OnebitLamb