question regards to loss calculation

[kwonmha]

For augmented model, you added alpha(0.5) * temperature(10) * augmented_loss to ordinary loss.
How did you choose alpha and temperature?
And why did you inserted multiplying temperature to augmented loss?
Because it's not shown in the paper.
And have you tested using only augmented loss without adding it to ordinary loss?
I think it's not explicitly mentioned in paper.
TY

[icoxfog417]

Augmented loss is calculated as follows (p2, formula 3.3).

So we have to decide the parameter α. The author describes the way to compute α in the Appendix(p12).

α=γτ. I implement this at here.

[kwonmha]

Thank you for answer.
Maybe I've overlooked the appendix part.

[kwonmha]

Sorry but I have another question to make certain.
How should I get output of the network for validating or testing?
Is it softmax( (Wh+b)/t )?

And I think you didn't divide Wh+b by temperature when calculating cross entropy like eq(3.1) on paper.

[kwonmha]

You can ignore last 2 sentences of my previous comment.
I got confused.
Regards to the output of the network for validating or testing, it can be softmax(Wh+b), right?

[icoxfog417]

I think softmax( (Wh+b)/t ) is used to calculate augmented loss only.
So network output will be softmax(Wh+b).

[kwonmha]

I think so too.
Thank you!

[kwonmha]

It's a kind of glitch, but in formulation.png, it looks like you used softmax(Wh/t) to calculate both CE and KL.
Which may be not the case.