Add generation server scripts using HF accelerate and DS-inference#328

Merged

mayank31398 merged 33 commits intomainfrom add-generation-server

Sep 1, 2022

+1,436

Commits on Aug 12, 2022

first step towards making libs

Mayank Mishra
authored and
Mayank Mishra
committed
HF accelerate model

Mayank Mishra
authored and
Mayank Mishra
committed

Commits on Aug 13, 2022

refactor accelerate

Mayank Mishra
authored and
Mayank Mishra
committed
refactor DS inference

Mayank Mishra
authored and
Mayank Mishra
committed
refactor DS ZeRO

Mayank Mishra
authored and
Mayank Mishra
committed
make inference library

Mayank Mishra
authored and
Mayank Mishra
committed

Commits on Aug 15, 2022

cli

Mayank Mishra
authored and
Mayank Mishra
committed
server

Mayank Mishra
authored and
Mayank Mishra
committed

Commits on Aug 16, 2022

request

Mayank Mishra
authored and
Mayank Mishra
committed
remove MaxTokensError

Mayank Mishra
authored and
Mayank Mishra
committed
fix batch size error with DS inference server

Mayank Mishra
authored and
Mayank Mishra
committed
type fix

Mayank Mishra
authored and
Mayank Mishra
committed
add latency

Mayank Mishra
authored and
Mayank Mishra
committed
add latency

Mayank Mishra
authored and
Mayank Mishra
committed

Commits on Aug 17, 2022

add min_length to default kwargs

Mayank Mishra
authored and
Mayank Mishra
committed
str kwargs

Mayank Mishra
authored and
Mayank Mishra
committed
str kwargs

Mayank Mishra
authored and
Mayank Mishra
committed

Commits on Aug 18, 2022

fix comma

Mayank Mishra
authored and
Mayank Mishra
committed
add old scripts back

Mayank Mishra
authored and
Mayank Mishra
committed

Commits on Aug 19, 2022

move scripts

Mayank Mishra
authored and
Mayank Mishra
committed

Commits on Aug 20, 2022

drop data

Mayank Mishra
authored and
Mayank Mishra
committed
minor changes + add README

Mayank Mishra
authored and
Mayank Mishra
committed
update README

Mayank Mishra
authored and
Mayank Mishra
committed
Merge branch 'main' into add-generation-server

Mayank Mishra
authored and
Mayank Mishra
committed

Commits on Aug 21, 2022

drop nccl

Mayank Mishra
authored and
Mayank Mishra
committed

Commits on Aug 24, 2022

fix

Mayank Mishra
authored and
Mayank Mishra
committed
default values

Mayank Mishra
authored and
Mayank Mishra
committed
resolve issues

Mayank Mishra
authored and
Mayank Mishra
committed

Commits on Aug 25, 2022

handle keyboard interrupt

Mayank Mishra
authored and
Mayank Mishra
committed

Commits on Aug 27, 2022

remove caching

Mayank Mishra
authored and
Mayank Mishra
committed

Commits on Aug 29, 2022

use snapshot_download

Mayank Mishra
authored and
Mayank Mishra
committed
make server class

Mayank Mishra
authored and
Mayank Mishra
committed

Commits on Sep 1, 2022

fix snapshot download

Mayank Mishra
authored and
Mayank Mishra
committed