Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Scatter update variable in TF optimizer #19281

Closed
shkarupa-alex opened this issue Mar 11, 2024 · 1 comment · Fixed by #19313
Closed

Scatter update variable in TF optimizer #19281

shkarupa-alex opened this issue Mar 11, 2024 · 1 comment · Fixed by #19313
Assignees
@hertschuh @sachinprasadhs
Labels
type:feature The user is asking for a new feature.

Comments

@shkarupa-alex
Copy link
Contributor

In TensorFlow there is a cool (fast) variable update operation - scatter_update (like "assign" for dense variables).
It would be cool if you override assign operation for such cases (i think it should looks like https://github.com/keras-team/keras/blob/master/keras/backend/tensorflow/optimizer.py#L45 )

P.S.
Found such case during migration of Keras v2 custom optimizer.
@sachinprasadhs sachinprasadhs added type:feature The user is asking for a new feature. keras-team-review-pending Pending review by a Keras team member. labels Mar 12, 2024
@mattdangerw mattdangerw removed the keras-team-review-pending Pending review by a Keras team member. label Mar 14, 2024
hertschuh added a commit to hertschuh/keras that referenced this issue Mar 14, 2024
@hertschuh
Add Tensorflow support for variable scatter_udpate in optimizers.
c7a01b9 
This should have been added as part of keras-team#18692 as it was added in the super class: https://github.com/keras-team/keras/blob/master/keras/optimizers/base_optimizer.py#L222

Fixes keras-team#19281
@hertschuh hertschuh mentioned this issue Mar 14, 2024
Merged
hertschuh added a commit to hertschuh/keras that referenced this issue Mar 14, 2024
@hertschuh
Add Tensorflow support for variable scatter_update in optimizers.
bb9551b 
This should have been added along with `scatter_add` and `scatter_sub` as part of keras-team#18692 as it was added in the super class: https://github.com/keras-team/keras/blob/master/keras/optimizers/base_optimizer.py#L222

Fixes keras-team#19281
fchollet pushed a commit that referenced this issue Mar 14, 2024
@hertschuh
Add Tensorflow support for variable scatter_update in optimizers. (#…
3fcb38c 
…19313)

This should have been added along with `scatter_add` and `scatter_sub` as part of #18692 as it was added in the super class: https://github.com/keras-team/keras/blob/master/keras/optimizers/base_optimizer.py#L222

Fixes #19281
@hertschuh
Copy link
Collaborator

@shkarupa-alex , thanks for the report. This was an omission. The fix is in keras-nightly, you can now just do self.assign(variable, update) within the optimizer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@hertschuh hertschuh

@sachinprasadhs sachinprasadhs

Labels
type:feature The user is asking for a new feature.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add Tensorflow support for variable scatter_update in optimizers. hertschuh/keras
4 participants
@hertschuh @shkarupa-alex @mattdangerw @sachinprasadhs