Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Dynamic KV cache allocation #1364

Merged
merged 39 commits into from
Dec 24, 2024
Merged

Dynamic KV cache allocation #1364

merged 39 commits into from
Dec 24, 2024

Conversation

popovaan
Copy link
Contributor

Dynamic KV cache allocation
Ticket: CVS-158409

@ilya-lavrenov ilya-lavrenov self-assigned this Dec 11, 2024
@popovaan popovaan marked this pull request as ready for review December 13, 2024 10:45
ilya-lavrenov
ilya-lavrenov reviewed Dec 16, 2024
View reviewed changes
src/cpp/src/cache_manager.hpp Outdated Show resolved Hide resolved
src/cpp/src/cache_manager.hpp Outdated Show resolved Hide resolved
src/cpp/src/scheduler.hpp Outdated Show resolved Hide resolved
src/cpp/src/utils/paged_attention_transformations.cpp Outdated Show resolved Hide resolved
src/cpp/src/cache_manager.hpp Outdated Show resolved Hide resolved
src/cpp/src/scheduler.hpp Outdated Show resolved Hide resolved
@ilya-lavrenov ilya-lavrenov added this to the 2025.0 milestone Dec 16, 2024
@github-actions github-actions bot added the category: samples GenAI samples label Dec 17, 2024
@github-actions github-actions bot added the category: speculative decoding Speculative decoding label Dec 20, 2024
ilya-lavrenov
ilya-lavrenov reviewed Dec 24, 2024
View reviewed changes
src/cpp/src/cache_manager.hpp Show resolved Hide resolved
src/cpp/src/continuous_batching_impl.hpp Outdated Show resolved Hide resolved
src/cpp/src/block_manager.hpp Outdated Show resolved Hide resolved
src/cpp/src/scheduler.hpp Outdated Show resolved Hide resolved
src/cpp/src/scheduler.hpp Outdated Show resolved Hide resolved
src/cpp/src/scheduler.hpp Outdated Show resolved Hide resolved
src/cpp/src/scheduler.hpp Outdated Show resolved Hide resolved
src/cpp/src/scheduler.hpp Outdated Show resolved Hide resolved
src/cpp/src/scheduler.hpp Outdated Show resolved Hide resolved
ilya-lavrenov
ilya-lavrenov reviewed Dec 24, 2024
View reviewed changes
src/cpp/src/scheduler.hpp Outdated Show resolved Hide resolved
src/cpp/src/scheduler.hpp Outdated Show resolved Hide resolved
popovaan and others added 2 commits December 24, 2024 14:30
@ilya-lavrenov ilya-lavrenov added this pull request to the merge queue Dec 24, 2024
Merged via the queue into openvinotoolkit:master with commit 021d880 Dec 24, 2024
59 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ilya-lavrenov ilya-lavrenov ilya-lavrenov approved these changes

@luo-cheng2021 luo-cheng2021 Awaiting requested review from luo-cheng2021

@iefode iefode Awaiting requested review from iefode

@Wovchena Wovchena Awaiting requested review from Wovchena

Assignees

@ilya-lavrenov ilya-lavrenov

Labels
category: continuous batching Continuous batching category: LLM LLM pipeline (stateful, static) category: samples GenAI samples category: speculative decoding Speculative decoding no-match-files
Projects
None yet
Milestone
2025.0
Development

Successfully merging this pull request may close these issues.
5 participants
@popovaan @ilya-lavrenov @Wovchena @iefode @luo-cheng2021