Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[torch.compile] rework test plans #9866

Merged
merged 19 commits into from
Nov 1, 2024
Merged

[torch.compile] rework test plans #9866

merged 19 commits into from
Nov 1, 2024

Conversation

youkaichao
Copy link
Member

added inductor tests by comparing logprobs ( it can change the output, so exact comparison is not doable).

also added encoding tests and tests for vision-language models.

also add compile for vision tower of language models.

@youkaichao
add more tests
a9076cb 
Signed-off-by: youkaichao <[email protected]>
@youkaichao
tolerate diff in encode
2743a5a 
Signed-off-by: youkaichao <[email protected]>
@youkaichao
Merge branch 'main' into compile_encode
9feb037
@youkaichao
change test signature for readability
73087f3 
Signed-off-by: youkaichao <[email protected]>
@youkaichao
fix 3 comparisons
ab4e9f8 
Signed-off-by: youkaichao <[email protected]>
@youkaichao
fix llava
f4d6680 
Signed-off-by: youkaichao <[email protected]>
@youkaichao
support phi3v
71d6e79 
Signed-off-by: youkaichao <[email protected]>
@youkaichao
fix chat template
16f7cf8 
Signed-off-by: youkaichao <[email protected]>
@youkaichao
add flags
7468d0f 
Signed-off-by: youkaichao <[email protected]>
@youkaichao
use greedy
e3ebeb6 
Signed-off-by: youkaichao <[email protected]>
@youkaichao
logprobs comparison
f32feb8 
Signed-off-by: youkaichao <[email protected]>
@youkaichao
fix
71a6856 
Signed-off-by: youkaichao <[email protected]>
@youkaichao
compile vision tower
52a3058 
Signed-off-by: youkaichao <[email protected]>
@youkaichao
add close comparison
3bbc3b4 
Signed-off-by: youkaichao <[email protected]>
@youkaichao
add checks
832e84e 
Signed-off-by: youkaichao <[email protected]>
@youkaichao
add tests
3e341dc 
Signed-off-by: youkaichao <[email protected]>
@youkaichao
use gptq for testing
7d4055a 
Signed-off-by: youkaichao <[email protected]>
@github-actions GitHub Actions
Copy link

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can do one of these:

  • Add ready label to the PR
  • Enable auto-merge.

🚀

@youkaichao youkaichao added the ready ONLY add when PR is ready to merge/full CI is needed label Oct 31, 2024
@youkaichao
fix
79481fe 
Signed-off-by: youkaichao <[email protected]>
@youkaichao
comment
092f135 
Signed-off-by: youkaichao <[email protected]>
DarkLight1337
DarkLight1337 approved these changes Oct 31, 2024
View reviewed changes
Copy link
Member

@DarkLight1337 DarkLight1337 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good, just a nit

tests/utils.py Show resolved Hide resolved
@youkaichao youkaichao merged commit 566cd27 into vllm-project:main Nov 1, 2024
59 checks passed
@youkaichao youkaichao deleted the compile_encode branch November 1, 2024 05:20
@CRZbulabula CRZbulabula mentioned this pull request Nov 2, 2024
Merged
lk-chen pushed a commit to lk-chen/vllm that referenced this pull request Nov 4, 2024
@youkaichao
[torch.compile] rework test plans (vllm-project#9866)
230ceb9 
Signed-off-by: youkaichao <[email protected]>
lk-chen pushed a commit to lk-chen/vllm that referenced this pull request Nov 4, 2024
@youkaichao
[torch.compile] rework test plans (vllm-project#9866)
0c1e4d1 
Signed-off-by: youkaichao <[email protected]>
Signed-off-by: Linkun Chen <[email protected]>
richardsliu pushed a commit to richardsliu/vllm that referenced this pull request Nov 4, 2024
@youkaichao @richardsliu
[torch.compile] rework test plans (vllm-project#9866)
2f80b13 
Signed-off-by: youkaichao <[email protected]>
Signed-off-by: Richard Liu <[email protected]>
bigPYJ1151 pushed a commit to bigPYJ1151/vllm that referenced this pull request Nov 5, 2024
@youkaichao @bigPYJ1151
[torch.compile] rework test plans (vllm-project#9866)
9f584d7 
Signed-off-by: youkaichao <[email protected]>
hissu-hyvarinen pushed a commit to ROCm/vllm that referenced this pull request Nov 6, 2024
@youkaichao @hissu-hyvarinen
[torch.compile] rework test plans (vllm-project#9866)
d278a68 
Signed-off-by: youkaichao <[email protected]>
JC1DA pushed a commit to JC1DA/vllm that referenced this pull request Nov 11, 2024
@youkaichao @JC1DA
[torch.compile] rework test plans (vllm-project#9866)
32e8d1e 
Signed-off-by: youkaichao <[email protected]>
Signed-off-by: Loc Huynh <[email protected]>
sumitd2 pushed a commit to sumitd2/vllm that referenced this pull request Nov 14, 2024
@youkaichao @sumitd2
[torch.compile] rework test plans (vllm-project#9866)
660f59a 
Signed-off-by: youkaichao <[email protected]>
Signed-off-by: Sumit Dubey <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@DarkLight1337 DarkLight1337 DarkLight1337 approved these changes

Assignees
No one assigned
Labels
ready ONLY add when PR is ready to merge/full CI is needed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@youkaichao @DarkLight1337