Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

Set range in NDArray from Python list fails #16647

Closed
fhieber opened this issue Oct 27, 2019 · 2 comments · Fixed by #16895
Closed

Set range in NDArray from Python list fails #16647

fhieber opened this issue Oct 27, 2019 · 2 comments · Fixed by #16895
Assignees

Comments

@fhieber
Copy link
Contributor

fhieber commented Oct 27, 2019

Description

Setting a range in an NDArray using a Python list fails with an obscure error about broadcasting when the range to set values in is equal to the size of the respective dimension in the target array.
Bug occurs in latest master (not in MXNet 1.5.1).
When another NDArray is used as data source, no error is thrown.

Error Message

ValueError                                Traceback (most recent call last)
<ipython-input-32-7d7d8c3fa8f4> in <module>
----> 1 dst[0, :len(src), 0] = src

~/anaconda3/lib/python3.7/site-packages/mxnet/ndarray/ndarray.py in __setitem__(self, key, value)
    523             indexing_dispatch_code = get_indexing_dispatch_code(slc_key)
    524             if indexing_dispatch_code == _NDARRAY_BASIC_INDEXING:
--> 525                 self._set_nd_basic_indexing(key, value)
    526             elif indexing_dispatch_code == _NDARRAY_ADVANCED_INDEXING:
    527                 self._set_nd_advanced_indexing(key, value)

~/anaconda3/lib/python3.7/site-packages/mxnet/ndarray/ndarray.py in _set_nd_basic_indexing(self, key, value)
    958                 # Other array-like
    959                 value_nd = self._prepare_value_nd(
--> 960                     value, bcast_shape=self.shape
    961                 )
    962                 value_nd.copyto(self)

~/anaconda3/lib/python3.7/site-packages/mxnet/ndarray/ndarray.py in _prepare_value_nd(self, value, bcast_shape, squeeze_axes)
    755                 value_nd = value_nd.reshape(bcast_shape)
    756             else:
--> 757                 value_nd = value_nd.broadcast_to(bcast_shape)
    758         return value_nd
    759

~/anaconda3/lib/python3.7/site-packages/mxnet/ndarray/ndarray.py in broadcast_to(self, shape)
   2275         broadcasting_axes = np.nonzero(cur_shape_arr != np.array(shape))
   2276         if (cur_shape_arr[broadcasting_axes] != 1).any():
-> 2277             raise ValueError(err_str)
   2278         if cur_shape != self.shape:
   2279             return op.broadcast_to(self.reshape(cur_shape), shape=shape)

ValueError: operands could not be broadcast together with remapped shapes[original->remapped]: (3,) and requested shape (1, 3, 1)

To Reproduce

dst = mx.nd.zeros((1, 3, 1))  # destination array
src = [1, 2, 3]
dst[0, :len(src), 0] = src

Using an NDArray as source data works

dst = mx.nd.zeros((1, 3, 1))  # destination array
src = [1, 2, 3]
dst[0, :len(src), 0] = mx.nd.array(src)

And also any range that is < dim2 of dst

dst = mx.nd.zeros((1, 3, 1))  # destination array
src = [1, 2]
dst[0, :len(src), 0] = src

Environment

----------Python Info----------
Version      : 3.7.1
Compiler     : Clang 4.0.1 (tags/RELEASE_401/final)
Build        : ('default', 'Dec 14 2018 13:28:58')
Arch         : ('64bit', '')
------------Pip Info-----------
Version      : 18.1
Directory    : /Users/fhieber/anaconda3/lib/python3.7/site-packages/pip
----------MXNet Info-----------
Version      : 1.6.0
Directory    : /Users/fhieber/anaconda3/lib/python3.7/site-packages/mxnet
Num GPUs     : 0
Commit Hash   : 78627387d41f284067bec164063ae53dbc32adf3
----------System Info----------
Platform     : Darwin-19.0.0-x86_64-i386-64bit
system       : Darwin
node         : -----
release      : 19.0.0
version      : Darwin Kernel Version 19.0.0: Wed Sep 25 20:18:50 PDT 2019; root:xnu-6153.11.26~2/RELEASE_X86_64
----------Hardware Info----------
machine      : x86_64
processor    : i386
b'machdep.cpu.brand_string: Intel(R) Core(TM) i5-7360U CPU @ 2.30GHz'
b'machdep.cpu.features: FPU VME DE PSE TSC MSR PAE MCE CX8 APIC SEP MTRR PGE MCA CMOV PAT PSE36 CLFSH DS ACPI MMX FXSR SSE SSE2 SS HTT TM PBE SSE3 PCLMULQDQ DTES64 MON DSCPL VMX SMX EST TM2 SSSE3 FMA CX16 TPR PDCM SSE4.1 SSE4.2 x2APIC MOVBE POPCNT AES PCID XSAVE OSXSAVE SEGLIM64 TSCTMR AVX1.0 RDRAND F16C'
b'machdep.cpu.leaf7_features: RDWRFSGS TSC_THREAD_OFFSET SGX BMI1 HLE AVX2 SMEP BMI2 ERMS INVPCID RTM FPU_CSDS MPX RDSEED ADX SMAP CLFSOPT IPT MDCLEAR TSXFA IBRS STIBP L1DF SSBD'
b'machdep.cpu.extfeatures: SYSCALL XD 1GBPAGE EM64T LAHF LZCNT PREFETCHW RDTSCP TSCI'
----------Network Test----------
Setting timeout: 10
Timing for MXNet: https://github.com/apache/incubator-mxnet, DNS: 0.1085 sec, LOAD: 0.8791 sec.
Timing for GluonNLP GitHub: https://github.com/dmlc/gluon-nlp, DNS: 0.0012 sec, LOAD: 0.6442 sec.
Timing for GluonNLP: http://gluon-nlp.mxnet.io, DNS: 0.0397 sec, LOAD: 0.2313 sec.
Timing for D2L: http://d2l.ai, DNS: 0.1149 sec, LOAD: 0.3263 sec.
Timing for D2L (zh-cn): http://zh.d2l.ai, DNS: 0.0340 sec, LOAD: 0.4763 sec.
Timing for FashionMNIST: https://repo.mxnet.io/gluon/dataset/fashion-mnist/train-labels-idx1-ubyte.gz, DNS: 0.0398 sec, LOAD: 0.8817 sec.
Timing for PYPI: https://pypi.python.org/pypi/pip, DNS: 0.1047 sec, LOAD: 0.6946 sec.
Timing for Conda: https://repo.continuum.io/pkgs/free/, DNS: 0.0200 sec, LOAD: 0.2118 sec.
@fhieber fhieber added the Bug label Oct 27, 2019
@szha
Copy link
Member

szha commented Oct 27, 2019

This should be a release blocker.

cc @ptrendx @reminisce

@samskalicky
Copy link
Contributor

@lanking520 assign @szha

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants