-
-
Notifications
You must be signed in to change notification settings - Fork 4.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[misc] error early for old-style class #10304
Conversation
Signed-off-by: youkaichao <[email protected]>
👋 Hi! Thank you for contributing to the vLLM project. Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can do one of these:
🚀 |
Signed-off-by: youkaichao <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I haven't tested this, but the change lgtm.
Another way that might be nice for the future would be to create versioned base classes to inherit from, like ModuleV1
, ModuleV2
, ... where a new one is created to reflect a new API that is not backwards compatible.
In this code, instead of having to inspect the function signature, you could explicitly check for which version it inherits from.
Anyway, just an idea. This is more interesting in a post-v1 world where maintaining API compatibility is considered more important.
Signed-off-by: youkaichao <[email protected]>
Signed-off-by: youkaichao <[email protected]>
Signed-off-by: youkaichao <[email protected]>
Signed-off-by: youkaichao <[email protected]>
@russellb thanks for the suggestion! I updated the doc to include some shim code example. It's not just V1, V0 also has this change. |
Right, I know both have the change. I just mean that once V1 is more locked down, it may be worth being a bit more strict about API changes that impact external consumers, and we could do some things in the code to help with that. |
we don't guarantee any undocumented details for out-of-tree registered code. They need to catch up. For the new code, it is documented, and we will make that signature backward compatible in the future. |
I think we're in agreement! |
Signed-off-by: youkaichao <[email protected]> Signed-off-by: OmerD <[email protected]>
Signed-off-by: youkaichao <[email protected]> Signed-off-by: Sumit Dubey <[email protected]>
as we change to the
vllm_config
, out-of-tree registered models need to be updated.we give a clear error message here for users to update.