Updating hyper-parameters on MLP Pytorch #20
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
I have been playing around with hyper-parameters, and with the current model on mlp_pytorch_kaparthy, I have: Validation Loss: 2.059125 Test Loss: 2.057981416583061 Meanwhile, mlp_pytorch_reyzenello has: Validation Loss: 2.053818 Test Loss: 2.0344724378415515 Maybe that could be a better configuration? What do you think? I have applied random search using 100 configurations to find the best configuration with hyper-parameters that are suitable for that model. If I have made any miscalculations or missed anything, feel free to add a comment. I'm open to feedback. In case of minor changes, I will update the pull request! Here is my Google Colab, which I have used to find out the best configuration (work in progress): https://colab.research.google.com/drive/1R0FF5qlB900CrxuxV_XW89Ds-Pup_S8e#scrollTo=PDzMw_o4IKyo
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I have been playing around with hyper-parameters, and with the current model on mlp_pytorch_kaparthy, I have:
Validation Loss: 2.059125
Test Loss: 2.057981416583061
Meanwhile, mlp_pytorch_reyzenello has:
Validation Loss: 2.053818
Test Loss: 2.0344724378415515
Maybe that could be a better configuration? What do you think? I have applied random search using 100 configurations to find the best configuration with hyper-parameters that are suitable for that model. If I have made any miscalculations or missed anything, feel free to add a comment. I'm open to feedback. In case of minor changes, I will update the pull request!
Here is my Google Colab, which I have used to find out the best configuration (work in progress): https://colab.research.google.com/drive/1R0FF5qlB900CrxuxV_XW89Ds-Pup_S8e#scrollTo=PDzMw_o4IKyo