Dimensions of Q, K, V for transformer

[guocheng98]

Hi there!

I was changing --transformer-heads from default 8 to 12, and marian returned back reshape error.
Error: Reshape must not change the number of elements (from shape=1x512x10x512 size=2621440 to shape=512x10x12x42 size=2580480)
I instantly realized that I also need to change --dim-emb as well so that dim-emb/transformer-heads == integer
And it did work after I set dim-emb to 768.
I also tried setting the dim-emb to 512 and transformer-heads to 16. There was no problem neither.

However, it also made me curious about how marianNMT handles the dimensions of Q, K, V in transformer.

Does marianNMT set them to a fix value (e.g. 64)? Or set all of them to dim-emb/transformer-heads?

Many thanks in advance!
Guocheng