Tensor-Parallelism general support#1512

Merged

jeffra merged 17 commits intomasterfrom inference/TP-general-support

Nov 12, 2021

+177-24

Commits on Oct 27, 2021

fixing the softmax masking when using triangular masking
Reza Yazdani
committed

Commits on Oct 29, 2021

Merge branch 'master' of github.com:microsoft/DeepSpeed
Reza Yazdani
committed

Commits on Nov 2, 2021

Merge branch 'master' of github.com:microsoft/DeepSpeed
Reza Yazdani
committed
add model-parallelism support independent of kernel-injection
Reza Yazdani
committed
remove unnecessary changes
Reza Yazdani
committed
add gpt-2 TP support without using policy
Reza Yazdani
committed
Merge branch 'master' into inference/TP-general-support
tjruwase
authored

Commits on Nov 3, 2021

fix some issues with the Tensor-Parallelism support
Reza Yazdani
committed
Merge branch 'inference/TP-general-support' of github.com:microsoft/DeepSpeed into inference/TP-general-support
Reza Yazdani
committed

Commits on Nov 4, 2021

Merge branch 'master' into inference/TP-general-support
RezaYazdaniAminabadi
authored
fix the replace-module for TP-inference support
Reza Yazdani
committed
Merge branch 'inference/TP-general-support' of github.com:microsoft/DeepSpeed into inference/TP-general-support
Reza Yazdani
committed
fix formatting
Reza Yazdani
committed

Commits on Nov 8, 2021

Merge branch 'master' into inference/TP-general-support
jeffra
authored

Commits on Nov 9, 2021

Merge branch 'master' into inference/TP-general-support
RezaYazdaniAminabadi
authored

Commits on Nov 11, 2021