Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix logging msg for block manager #3701

Merged
merged 3 commits into from
Mar 28, 2024
Merged

fix logging msg for block manager #3701

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
e7130e4
fix logging msg for block manager
simon-mo Mar 28, 2024
06abb24
make sure we only log when needed
simon-mo Mar 28, 2024
4d18de5
fix flash attn msg
simon-mo Mar 28, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion vllm/attention/selector.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,8 @@ def _can_use_flash_attn(dtype: torch.dtype) -> bool:
try:
import flash_attn # noqa: F401
except ImportError:
logger.info("flash_attn is not found.")
logger.info(
"Cannot use FlashAttention because the package is not found. "
"Please install it for better performance.")
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@WoosukKwon is this true?

return False
return True
3 changes: 1 addition & 2 deletions vllm/core/block_manager_v1.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -230,13 +230,12 @@ def __init__(
self.watermark_blocks = int(watermark * num_gpu_blocks)

if self.enable_caching:
logger.info("enable automatic prefix caching")
logger.info("Automatic prefix caching is enabled.")
self.gpu_allocator = CachedBlockAllocator(Device.GPU, block_size,
num_gpu_blocks)
self.cpu_allocator = CachedBlockAllocator(Device.CPU, block_size,
num_cpu_blocks)
else:
logger.info("disable automatic prefix caching")
self.gpu_allocator = UncachedBlockAllocator(
Device.GPU, block_size, num_gpu_blocks)
self.cpu_allocator = UncachedBlockAllocator(
Expand Down
2 changes: 1 addition & 1 deletion vllm/model_executor/parallel_utils/pynccl_utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@
try:
from vllm.model_executor.parallel_utils.pynccl import (NCCLCommunicator,
ncclGetVersion)
logger.info(f"vLLM is using nccl=={ncclGetVersion()}")
except Exception as e:
# in non-NVIDIA environments, we can't import the nccl module
# e.g. when running on machines with AMD GPUs
Expand Down Expand Up @@ -39,6 +38,7 @@ def set_pynccl_stream(stream: torch.cuda.Stream):
def init_process_group(world_size: int, rank: int, init_method: str) -> None:
assert not is_initialized()
global comm
logger.info(f"vLLM is using nccl=={ncclGetVersion()}")
comm = NCCLCommunicator(init_method=init_method,
world_size=world_size,
rank=rank)
Expand Down
Loading