Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] restart file from v2.0.0.b4 cannot freeze models in v2.0.0 #1053

Closed
njzjz opened this issue Aug 28, 2021 · 3 comments · Fixed by #1055
Closed

[BUG] restart file from v2.0.0.b4 cannot freeze models in v2.0.0 #1053

njzjz opened this issue Aug 28, 2021 · 3 comments · Fixed by #1055
Labels

Comments

@njzjz
Copy link
Member

njzjz commented Aug 28, 2021

Summary

Continue training from a checkpoint file in v2.0.0.b4 cannot freeze models in v2.0.0. #1000 breaks it. It worked after I removed model_type, train_attr/min_nbor_dist, and train_attr/training_script.

The following nodes will be frozen: ['model_type', 'descrpt_attr/rcut', 'descrpt_attr/ntypes', 'model_attr/tmap', 'model_attr/model_type', 'model_attr/model_version', 'train_attr/min_nbor_dist', 'train_attr/training_script', 'o_energy', 'o_force', 'o_virial', 'o_atom_energy', 'o_atom_virial', 'fitting_attr/dfparam', 'fitting_attr/daparam']
Traceback (most recent call last):
  File "/home/jz748/ambertools-dp/bin/dp", line 10, in <module>
    sys.exit(main())
  File "/home/jz748/ambertools-dp/lib/python3.9/site-packages/deepmd/entrypoints/main.py", line 439, in main
    freeze(**dict_args)
  File "/home/jz748/ambertools-dp/lib/python3.9/site-packages/deepmd/entrypoints/freeze.py", line 183, in freeze
    output_graph_def = tf.graph_util.convert_variables_to_constants(
  File "/home/jz748/ambertools-dp/lib/python3.9/site-packages/tensorflow/python/util/deprecation.py", line 337, in new_func
    return func(*args, **kwargs)
  File "/home/jz748/ambertools-dp/lib/python3.9/site-packages/tensorflow/python/framework/graph_util_impl.py", line 276, in convert_variables_to_constants
    ret = convert_to_constants.convert_variables_to_constants_from_session_graph(
  File "/home/jz748/ambertools-dp/lib/python3.9/site-packages/tensorflow/python/framework/convert_to_constants.py", line 1160, in convert_variables_to_constants_from_session_graph
    converter_data=_SessionConverterData(
  File "/home/jz748/ambertools-dp/lib/python3.9/site-packages/tensorflow/python/framework/convert_to_constants.py", line 857, in __init__
    graph_def = graph_util.extract_sub_graph(graph_def, output_node_names)
  File "/home/jz748/ambertools-dp/lib/python3.9/site-packages/tensorflow/python/util/deprecation.py", line 337, in new_func
    return func(*args, **kwargs)
  File "/home/jz748/ambertools-dp/lib/python3.9/site-packages/tensorflow/python/framework/graph_util_impl.py", line 209, in extract_sub_graph
    _assert_nodes_are_present(name_to_node, dest_nodes)
  File "/home/jz748/ambertools-dp/lib/python3.9/site-packages/tensorflow/python/framework/graph_util_impl.py", line 164, in _assert_nodes_are_present
    assert d in name_to_node, "%s is not in graph" % d
AssertionError: model_type is not in graph

Deepmd-kit version, installation way, input file, running commands, error log, etc.

v2.0.0

Steps to Reproduce

Train a model in v2.0.0.b4 without freezing, and freeze in v2.0.0.

Further Information, Files, and Links

@njzjz njzjz added the bug label Aug 28, 2021
@amcadmus
Copy link
Member

This is expected, because to support compressed training, 2.0.0 breaks the model compatibility of 2.0.0.b4.

@amcadmus
Copy link
Member

we shouldn't have introduced such a change in beta version...

@njzjz
Copy link
Member Author

njzjz commented Aug 28, 2021

I don't find a easy way to provide compatibility..

njzjz added a commit to njzjz/deepmd-kit that referenced this issue Aug 28, 2021
@njzjz njzjz linked a pull request Aug 28, 2021 that will close this issue
amcadmus pushed a commit that referenced this issue Aug 29, 2021
@njzjz njzjz closed this as completed Aug 29, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants