Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[misc] error early for old-style class #10304

Merged
merged 6 commits into from
Nov 14, 2024

Conversation

youkaichao
Copy link
Member

as we change to the vllm_config, out-of-tree registered models need to be updated.

we give a clear error message here for users to update.

Signed-off-by: youkaichao <[email protected]>
Copy link

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can do one of these:

  • Add ready label to the PR
  • Enable auto-merge.

🚀

Signed-off-by: youkaichao <[email protected]>
Copy link
Collaborator

@russellb russellb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't tested this, but the change lgtm.

Another way that might be nice for the future would be to create versioned base classes to inherit from, like ModuleV1, ModuleV2, ... where a new one is created to reflect a new API that is not backwards compatible.

In this code, instead of having to inspect the function signature, you could explicitly check for which version it inherits from.

Anyway, just an idea. This is more interesting in a post-v1 world where maintaining API compatibility is considered more important.

Signed-off-by: youkaichao <[email protected]>
Signed-off-by: youkaichao <[email protected]>
Signed-off-by: youkaichao <[email protected]>
Signed-off-by: youkaichao <[email protected]>
@mergify mergify bot added the documentation Improvements or additions to documentation label Nov 13, 2024
@youkaichao
Copy link
Member Author

@russellb thanks for the suggestion! I updated the doc to include some shim code example.

It's not just V1, V0 also has this change.

@russellb
Copy link
Collaborator

@russellb thanks for the suggestion! I updated the doc to include some shim code example.

It's not just V1, V0 also has this change.

Right, I know both have the change. I just mean that once V1 is more locked down, it may be worth being a bit more strict about API changes that impact external consumers, and we could do some things in the code to help with that.

@youkaichao
Copy link
Member Author

it may be worth being a bit more strict about API changes that impact external consumers

we don't guarantee any undocumented details for out-of-tree registered code. They need to catch up. For the new code, it is documented, and we will make that signature backward compatible in the future.

@russellb
Copy link
Collaborator

it may be worth being a bit more strict about API changes that impact external consumers

we don't guarantee any undocumented details for out-of-tree registered code. They need to catch up. For the new code, it is documented, and we will make that signature backward compatible in the future.

I think we're in agreement!

@youkaichao youkaichao added the ready ONLY add when PR is ready to merge/full CI is needed label Nov 14, 2024
@youkaichao youkaichao merged commit 504ac53 into vllm-project:main Nov 14, 2024
63 checks passed
@youkaichao youkaichao deleted the hint_type_error branch November 14, 2024 02:55
omer-dayan pushed a commit to omer-dayan/vllm that referenced this pull request Nov 14, 2024
sumitd2 pushed a commit to sumitd2/vllm that referenced this pull request Nov 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation ready ONLY add when PR is ready to merge/full CI is needed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants