Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unexpected default for nn.softmax #69

Closed
JackTemaki opened this issue Nov 9, 2021 · 1 comment
Closed

Unexpected default for nn.softmax #69

JackTemaki opened this issue Nov 9, 2021 · 1 comment
Milestone

Comments

@JackTemaki
Copy link
Contributor

JackTemaki commented Nov 9, 2021

I just lost a couple of hours on debugging because I forgot that softmax_over_spatial, which is what nn.softmaxmaps to, does something completely different than the old "softmax" layer. Because it does not do the softmax per default over "F", but something different (defaulting to the time axis). This is really dangerous when you expect that you can use nn.softmax as an activation function.

I am not sure how to solve this best, I would say"softmax_over_spatial" is okay in its behavior (so no RETURNN changes), but nn.softmax should definitely not default to that behavior.

Maybe this issue is already resolved if nn.softmax needs an explicit dimension tag in the future, but if not, it needs to be fixed.

@albertz
Copy link
Member

albertz commented Dec 16, 2021

This is fixed now.

@albertz albertz closed this as completed Dec 16, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants