🛠️ #46 inited the least latency routing#70

Merged

roma-glushko merged 25 commits intodevelopfrom 46-routing-least-latency-strategy

Jan 14, 2024

+693-83

Commits on Jan 11, 2024

#46 inited the least latency routing
roma-glushko
committed

Commits on Jan 13, 2024

#46 Added the moving average implementation to keep track of latencies over time
roma-glushko
committed
#46 Added latency filed to langModels + protected movingAverage by mutex
roma-glushko
committed
#46 : Got the least latency routing logic defined with warm up phase
roma-glushko
committed

Commits on Jan 14, 2024