Decompose sample_from_logits for clarity and further development#190

Merged

masahi merged 3 commits intooctoml:batch-servingfrom Deelvin:vc/tvm-model-refactor

Feb 2, 2024

+85-85

Commits on Feb 2, 2024

refactor sampling from logits, other small fixes
Valery Chernov
committed
use RequestType
Valery Chernov
committed
fix lint
Valery Chernov
committed