Insufficient documentation for GlobalAvgPool1D

[sravanbabuiitm]

I m trying to understand the functionality of https://mxnet.incubator.apache.org/api/python/gluon/nn.html#mxnet.gluon.nn.GlobalAvgPool1D

The documentation doesnt seem complete for this, but I have noted discrepancy compared to the behavior of same api in keras.

model = Sequential()

model.add(Embedding(vocab_size,
embedding_dims,
weights=[embedding_matrix],
input_length=max_len_doc, trainable=False))
model.add(GlobalAveragePooling1D())
model.add(Dense(n_labels, activation='sigmoid'))
model.compile(loss='categorical_crossentropy',
optimizer='adam',
metrics=['accuracy'])
model.summary()

Embedding layer followed by global average pooling layer summed along the column/features.
Build model...

---

Layer (type) Output Shape Param #
<><><><><>
embedding_2 (Embedding) (None, 65, 300) 4239000

---

global_average_pooling1d_2 ( (None, 300) 0

---

dense_2 (Dense) (None, 3) 903
<><><><><>
Total params: 4,239,903
Trainable params: 903
Non-trainable params: 4,239,000

---

In Gluon, I have tried the same operation :

embedding = nn.Embedding(20,5,sparse_grad=True, weight_initializer = mx.init.Uniform())
embedding.initialize()
pooling = gluon.nn.GlobalAvgPool1D()
print(embedding(mx.nd.array([[1,3,7]])))
print(pooling(embedding(mx.nd.array([[1,3,7]]))))

Output :

[[[ 0.05667423 -0.02676572 0.0301794 0.02835616 -0.0437789 ]
[ 0.01422429 0.01972841 -0.05923161 -0.01276907 -0.06303393]
[ 0.05422723 0.04757585 0.00387447 -0.01476508 0.06609883]]]
<NDArray 1x3x5 @cpu(0)>

[[[ 0.00893304]
[-0.02021638]
[ 0.03140226]]]
<NDArray 1x3x1 @cpu(0)>

I was expecting to see 1X1X5 as output.

It isn't even summing along the rows since when I did the following :

np.sum([ 0.05667423, -0.02676572, 0.0301794, 0.02835616, -0.0437789 ])
output : 0.04466516999999998

can you add more documentation to the API and also on how it is different from libraries offering same API's ?

[haojin2]

@sravanbabuiitm Thanks for reporting the insufficient doc, I agree that we should provide a more detailed doc at https://mxnet.incubator.apache.org/api/python/gluon/nn.html#mxnet.gluon.nn.GlobalAvgPool1D.
I also noticed several things to help you understand the behavior of the code:

1. You're using GlobalAvgPool1D(), so if you divide the result you get from np.sum by 5 you get 0.008933034, which should appear on the first row of the result you get on your example.
2. The default value for argument layout is saying that the default layout is NCW, so in your (1,3,5) sample input the axis 1 is recognized as the channel axis and axis 2 is the data axis, so the average operation is performed on the last axis. That's why you're getting a (1,3,1) tensor as the result.
   Please let me know if you have further questions or concerns. We'll work on addressing the problem with the doc at the same time.

[haojin2]

Doc fix in #11832

[haojin2]

@sravanbabuiitm The docs have been updated, do you have further questions regarding the usage of GlobalAvgPool*D() Gluon APIs? If you don't have further questions would you mind closing this issue? Thanks!

[sravanbabuiitm]

resolving since its clarified now