How to manage the parameters units when using Mxnet for distributed deep learning?

[ChrisQiqiang]

I am a beginner in Mxnet. Parameters are transferred in the form of <key, value> pairs when using MXNET for distributed deep learning, according to the paper Parameter Server in 2014. (It implies parameters will be transferred in the unit of each layer.)
If I want to define the form by myself, for instance, I want the parameters in layer 1 and layer 2 or more layers packed and transferred together, what should I do?

[roywei]

Hi @ChrisQiqiang , it depends on what do you want to achieve when you pack them and transfer together.
I believe the parameters are still been updated one by one according to here: https://github.com/apache/incubator-mxnet/blob/master/python/mxnet/gluon/trainer.py#L361
However, KV store APIs allows you to update several keys together: https://mxnet.incubator.apache.org/api/python/docs/api/kvstore/index.html

You may want to ask questions on our discussion forums:
English: https://discuss.mxnet.io
Chinese: https://discuss.gluon.ai/