Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Misc] [CI/Build] Speed up block manager CPU-only unit tests ~10x by opting-out of GPU cleanup #3783

Merged
merged 1 commit into from
Apr 2, 2024

Conversation

cadedaniel
Copy link
Collaborator

@cadedaniel cadedaniel commented Apr 1, 2024

783 passed, 4 deselected in 93.71s (0:01:33)
783 passed, 4 deselected in 2.16s

#3631 adds a global cleanup which empties the torch cache among other things. For CPU-only unit tests, this is unnecessary and actually causes a big slowdown for CPU-only tests due to loading torch. My dev machine has a slow EBS connection so loading torch is like ~10x time of a small CPU test.

This PR adds a pytest fixture to disable global cleanup. In your conftest (in some subdirectory) you can opt-out of global cleanup by overriding it as follows:

@pytest.fixture()
def should_do_global_cleanup_after_test() -> bool:
    return False

@cadedaniel cadedaniel marked this pull request as ready for review April 2, 2024 00:06
@cadedaniel cadedaniel enabled auto-merge (squash) April 2, 2024 00:06
@cadedaniel cadedaniel merged commit eb69d68 into vllm-project:main Apr 2, 2024
33 of 34 checks passed
@cadedaniel cadedaniel deleted the speedup-cpu-unit-tests branch April 2, 2024 02:52
Temirulan pushed a commit to Temirulan/vllm-whisper that referenced this pull request Sep 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants