Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Getting started-movielens 03-Training-with-HugeCTR.ipynb nb does not print out the acc and loss metrics #1521

Closed
rnyak opened this issue Apr 21, 2022 · 2 comments · Fixed by #1532
Labels
bug Something isn't working

Comments

@rnyak
Copy link
Contributor

rnyak commented Apr 21, 2022

Describe the bug
When I run the 03-Training-with-HugeCTR.ipynb notebook, the training step does not print out the accuracy and loss metrics.

We can avoid this

  1. Move the following commands to another cell:
model.compile()
model.summary()
model.fit(max_iter=2000, display=100, eval_interval=200, snapshot=1900)
model.graph_to_json(graph_config_file=MODEL_DIR + "1/movielens.json")

OR

  1. add the training commands to a .py file and run the following in another cell:
!python model.py

Second option might be problematic for the hugectr unit test because of !.

Steps/Code to reproduce bug
please run 03-Training-with-HugeCTR.ipynb

Expected behavior
A clear and concise description of what you expected to happen.

Environment details (please complete the following information):

  • Environment location: [Bare-metal, Docker, Cloud(specify cloud provider)]
  • Method of NVTabular install: [conda, Docker, or from source]
    • If method of install is [Docker], provide docker pull & docker run commands used

merlin-training:22.04 container is used

@rnyak rnyak added the bug Something isn't working label Apr 21, 2022
@rnyak
Copy link
Contributor Author

rnyak commented Apr 21, 2022

@benfred and @karlhigley can you please advice? I will create a PR accordingly. thanks.

@karlhigley
Copy link
Contributor

The first option sounds good to me

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
2 participants