[v1.x] Reduce after quantization memory usage

[bgawrych]

Description

Port of #20894

Script:

import mxnet as mx
from mxnet.gluon.model_zoo import vision
import psutil
import os

def get_process_memory():
    process = psutil.Process(os.getpid())
    mem_info = process.memory_info()
    return mem_info.rss * 1e-6


batch_shape = (1, 3, 224, 224)
data = mx.nd.random.normal(shape=batch_shape)

print("memory before loading model: ", get_process_memory())
net = vision.resnet50_v1(pretrained=True)
print("memory after loading model: ", get_process_memory()) 
out = net(data)
out.wait_to_read()
print("memory after fp32 forward pass", get_process_memory())

indata = {'data':data}
label = {'label':mx.nd.zeros(shape=(1,))}
dataiter = mx.io.NDArrayIter(indata, label, 3, True, last_batch_handle='discard')
net_quantized = mx.contrib.quant.quantize_net(net, quantized_dtype='auto',
                                                quantize_mode="smart",
                                                calib_mode='naive',
                                                calib_data=dataiter,
                                                num_calib_examples=1,
                                                ctx=mx.current_context())

print("memory after quantization: ", get_process_memory())

outputs = net_quantized(data)
outputs.wait_to_read()
print("memory after int8 forward pass: ", get_process_memory())

Output before:

memory before loading model:  201.936896
memory after loading model:  433.41004799999996
memory after fp32 forward pass 523.698176
memory after quantization:  1308.803072
memory after int8 forward pass:  1313.349632

Output after:

memory before loading model:  202.502144
memory after loading model:  434.184192
memory after fp32 forward pass 520.986624
memory after quantization:  1136.570368
memory after int8 forward pass:  1141.485568

[mxnet-bot]

Hey @bgawrych , Thanks for submitting the PR
All tests are already queued to run once. If tests fail, you can trigger one or more tests again with the following commands:

• To trigger all jobs: @mxnet-bot run ci [all]
• To trigger specific jobs: @mxnet-bot run ci [job1, job2]

---

CI supported jobs: [sanity, website, windows-gpu, clang, centos-cpu, windows-cpu, unix-gpu, unix-cpu, edge, miscellaneous, centos-gpu]

---

Note:
Only following 3 categories can trigger CI :PR Author, MXNet Committer, Jenkins Admin.
All CI tests must pass before the PR can be merged.

[bgawrych]

@mxnet-bot run ci [windows-gpu]

[mxnet-bot]

Jenkins CI successfully triggered : [windows-gpu]