-
Notifications
You must be signed in to change notification settings - Fork 27.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix jetmoe model #31279
Fix jetmoe model #31279
Conversation
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for fixing, @Cyrilvallez!
Could you also remove the skips from the model tests which were added in #31266 as a patch?
I think the changes look OK, but let's have a second opinion from @ArthurZucker, as I'm not very familiar with the recent cache changes
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we merged this for other models as well so LGTM.
* Fix jetmoe model * Remove skip-tests
* Fix jetmoe model * Remove skip-tests
* Fix jetmoe model * Remove skip-tests
* Fix jetmoe model * Remove skip-tests
* Fix jetmoe model * Remove skip-tests
What does this PR do?
Fixes #31266
cc @ArthurZucker @amyeroberts
I am sorry I did not notice that
JetMoePreTrainedModel
got_supports_cache_class = True
support while working on #30536.To avoid the same bug in the future, any model getting
_supports_cache_class = True
in the future should change the line:to
in
prepare_inputs_for_generation()
because checkingpast_key_values is None
is no longer correct as empty but initializedDynamicCache
can be passed.Also to minimize code complexity, the if-else that checks
if isinstance(past_key_values, Cache):
(still inprepare_inputs_for_generation()
) can be safely removed in favor of the case when this is True when_supports_cache_class = True
asgenerate()
will only use properCache
classes in this case.Some similar if-else could also be removed in upstream model architectures classes as well.