Model loading became very slow after #11001

[geoalgo]

Description

The PR (#11001) introduced a checked in SymbolBlock.imports that makes the loading of large graph takes very long time (it used to be instantaneous):

The issue is the code checking if any symbol is row_sparse, removing this check allows to load large model instantaneously again.

Environment info (Required)

----------Python Info----------
Version : 3.6.5
Compiler : GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)
Build : ('default', 'Jun 17 2018 12:26:58')
Arch : ('64bit', '')
------------Pip Info-----------
Version : 18.0
Directory : /usr/local/lib/python3.6/site-packages/pip
----------MXNet Info-----------
/usr/local/lib/python3.6/site-packages/requests/init.py:80: RequestsDependencyWarning: urllib3 (1.24) or chardet (3.0.4) doesn't match a supported version!
RequestsDependencyWarning)
Version : 1.3.0
Directory : /usr/local/lib/python3.6/site-packages/mxnet
Commit Hash : b3be92f
----------System Info----------
Platform : Darwin-16.7.0-x86_64-i386-64bit
system : Darwin
node : 186590d6796f.ant.amazon.com
release : 16.7.0
version : Darwin Kernel Version 16.7.0: Wed Oct 10 20:06:00 PDT 2018; root:xnu-3789.73.24~1/RELEASE_X86_64
----------Hardware Info----------
machine : x86_64
processor : i386
b'machdep.cpu.extfeatures: SYSCALL XD 1GBPAGE EM64T LAHF LZCNT PREFETCHW RDTSCP TSCI'
b'machdep.cpu.leaf7_features: SMEP ERMS RDWRFSGS TSC_THREAD_OFFSET BMI1 AVX2 BMI2 INVPCID SMAP RDSEED ADX IPT FPU_CSDS'
b'machdep.cpu.features: FPU VME DE PSE TSC MSR PAE MCE CX8 APIC SEP MTRR PGE MCA CMOV PAT PSE36 CLFSH DS ACPI MMX FXSR SSE SSE2 SS HTT TM PBE SSE3 PCLMULQDQ DTES64 MON DSCPL VMX EST TM2 SSSE3 FMA CX16 TPR PDCM SSE4.1 SSE4.2 x2APIC MOVBE POPCNT AES PCID XSAVE OSXSAVE SEGLIM64 TSCTMR AVX1.0 RDRAND F16C'
b'machdep.cpu.brand_string: Intel(R) Core(TM) i7-5557U CPU @ 3.10GHz'
----------Network Test----------
Setting timeout: 10
Timing for MXNet: https://github.com/apache/incubator-mxnet, DNS: 0.0543 sec, LOAD: 1.2793 sec.
Timing for Gluon Tutorial(en): http://gluon.mxnet.io, DNS: 0.0628 sec, LOAD: 0.9555 sec.
Timing for Gluon Tutorial(cn): https://zh.gluon.ai, DNS: 0.0802 sec, LOAD: 0.8386 sec.
Timing for FashionMNIST: https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/gluon/dataset/fashion-mnist/train-labels-idx1-ubyte.gz, DNS: 0.0073 sec, LOAD: 1.2169 sec.
Timing for PYPI: https://pypi.python.org/pypi/pip, DNS: 0.0521 sec, LOAD: 1.1819 sec.
Timing for Conda: https://repo.continuum.io/pkgs/free/, DNS: 0.0419 sec, LOAD: 0.3027 sec.

Minimum reproducible example

(If you are using your own code, please provide a short script that reproduces the error. Otherwise, please provide link to the existing example.)

Steps to reproduce

(Paste the commands you ran that produced the error.)

1. download model attached and script to load it
   model-loading-issue.zip
2. python slow_model_loading.py

What have you tried to solve it?

1. comment check that symbol is row_sparse

[geoalgo]

It will be great to test loading a large graph to avoid such regression (the model I attached could serve as a unit-test).

[vrakesh]

@geoalgo Thank you for reporting the performance regression in , model loading.

[vrakesh]

@mxnet-label-bot add [Performance]

[ThomasDelteil]

Looks like there might be some issue with the .get_internals() function which is the true bottle neck, some quadratic complexity maybe.

[karry-yu]

meet the same problem too.