Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Pipeline warnings and checkpoint portability #588

Merged
merged 3 commits into from
Dec 8, 2020
Merged

Pipeline warnings and checkpoint portability #588

merged 3 commits into from
Dec 8, 2020

Conversation

ShadenSmith
Copy link
Contributor

This PR does two things:

  1. Drops deprecated allreduce_gradients argument to backward()
  2. Fixes a portability bug in pipeline checkpointing. Previously, checkpoints could not be renamed due to this.

@ShadenSmith ShadenSmith added the bug Something isn't working label Dec 8, 2020
jeffra
jeffra approved these changes Dec 8, 2020
View reviewed changes
Copy link
Collaborator

@jeffra jeffra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, but can we add a test case that covers the checkpoint bug?

@ShadenSmith
Copy link
Contributor Author

Good point. Our pipeline checkpoint suite already covers this in a way. This change that returns None instead of a file path would trigger the same bug; we'd just fail to open None instead of a non-existent (or incorrect) file. Does that seem like a reasonable test?

@ShadenSmith ShadenSmith merged commit 2f62697 into microsoft:master Dec 8, 2020
@ShadenSmith ShadenSmith deleted the pp-portable-ckpt branch December 8, 2020 17:42
@bobisapotato bobisapotato mentioned this pull request Jan 24, 2021
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jeffra jeffra jeffra approved these changes

@arashashari arashashari Awaiting requested review from arashashari

@awan-10 awan-10 Awaiting requested review from awan-10

@cli99 cli99 Awaiting requested review from cli99

@conglongli conglongli Awaiting requested review from conglongli

@eltonzheng eltonzheng Awaiting requested review from eltonzheng

@minjiaz minjiaz Awaiting requested review from minjiaz

@niumanar niumanar Awaiting requested review from niumanar

@RezaYazdaniAminabadi RezaYazdaniAminabadi Awaiting requested review from RezaYazdaniAminabadi

@samyam samyam Awaiting requested review from samyam

@tjruwase tjruwase Awaiting requested review from tjruwase tjruwase is a code owner

Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@ShadenSmith @jeffra