Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

convert decay_rate to stop_lr from old inputs #949

Merged
merged 2 commits into from
Aug 12, 2021

Conversation

njzjz
Copy link
Member

@njzjz njzjz commented Aug 11, 2021

close #919.

@codecov-commenter
Copy link

codecov-commenter commented Aug 11, 2021

Codecov Report

Merging #949 (d9b0a65) into devel (e8a9101) will increase coverage by 0.04%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##            devel     #949      +/-   ##
==========================================
+ Coverage   75.47%   75.51%   +0.04%     
==========================================
  Files          85       85              
  Lines        6784     6796      +12     
==========================================
+ Hits         5120     5132      +12     
  Misses       1664     1664              
Impacted Files Coverage Δ
deepmd/utils/compat.py 84.55% <100.00%> (+1.49%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e8a9101...d9b0a65. Read the comment docs.

@njzjz
Copy link
Member Author

njzjz commented Aug 11, 2021

By the way, I'm confused -- does decay_rate really work in v1.2 and v1.3? I found it would be overridden here

def build(self, global_step, stop_batch = None) :
if stop_batch is None:
self.decay_steps_ = self.cd['decay_steps'] if self.cd['decay_steps'] is not None else 5000
self.decay_rate_ = self.cd['decay_rate'] if self.cd['decay_rate'] is not None else 0.95
else:
self.stop_lr_ = self.cd['stop_lr'] if self.cd['stop_lr'] is not None else 5e-8
default_ds = 100 if stop_batch // 10 > 100 else stop_batch // 100 + 1
self.decay_steps_ = self.cd['decay_steps'] if self.cd['decay_steps'] is not None else default_ds
if self.decay_steps_ >= stop_batch:
self.decay_steps_ = default_ds
self.decay_rate_ = np.exp(np.log(self.stop_lr_ / self.start_lr_) / (stop_batch / self.decay_steps_))

@njzjz njzjz force-pushed the convert_decay_rate branch from 65d0053 to d9b0a65 Compare August 11, 2021 23:48
@amcadmus
Copy link
Member

By the way, I'm confused -- does decay_rate really work in v1.2 and v1.3? I found it would be overridden here

def build(self, global_step, stop_batch = None) :
if stop_batch is None:
self.decay_steps_ = self.cd['decay_steps'] if self.cd['decay_steps'] is not None else 5000
self.decay_rate_ = self.cd['decay_rate'] if self.cd['decay_rate'] is not None else 0.95
else:
self.stop_lr_ = self.cd['stop_lr'] if self.cd['stop_lr'] is not None else 5e-8
default_ds = 100 if stop_batch // 10 > 100 else stop_batch // 100 + 1
self.decay_steps_ = self.cd['decay_steps'] if self.cd['decay_steps'] is not None else default_ds
if self.decay_steps_ >= stop_batch:
self.decay_steps_ = default_ds
self.decay_rate_ = np.exp(np.log(self.stop_lr_ / self.start_lr_) / (stop_batch / self.decay_steps_))

Yes the decay_rate will be overwritten by a rate computed from the the given stop_lr

@amcadmus amcadmus merged commit 5529a97 into deepmodeling:devel Aug 12, 2021
@njzjz njzjz deleted the convert_decay_rate branch August 12, 2021 22:19
gzq942560379 pushed a commit to HPC-AI-Team/deepmd-kit that referenced this pull request Sep 2, 2021
* convert `decay_rate` to `stop_lr` from old inputs

close deepmodeling#919.

* fix tests
njzjz added a commit to njzjz/deepmd-kit that referenced this pull request Sep 21, 2023
For historical reasons, it stays in the wrong directory.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants