How to read images into gluoncv from zip file?

[zhoulukuan]

I had to read images from the entire zip file because the number of images under a folder exceeds the limit of the Linux system. I used such code

import zipfile
import numpy as np
from PIL import Image
import io
import mxnet as mx

z = zipfile.ZipFile("train.zip", "r")
data = z.read(z.namelist()[0])
data = io.BytesIO(data)
im = Image.open(data)
im = np.asarray(img)
im = mx.nd.array(im)

It seems that I get the BGR image which is similar with what I get when using mx.image.imread(img_path, 1) in gluoncv. But I am curious about whether such a reading method will encounter some problems when dealing with multiple processes. Is the speed very slow? Is there a better way to deal with this problem in mxnet?

[zhoulukuan]

In python3, when using multi gpu, will cause this error:
zipfile.BadZipFile: Bad CRC-32 for file
I must create a new zipfile.ZipFile for anthor image to avoid this problem. Any better solution?

[leleamol]

@zhoulukuan
Here is the link to ImageIterator and related APIs in MXNet. https://mxnet.incubator.apache.org/versions/master/api/python/image/image.html#module-mxnet.image

I have tagged this question so that experts can look into this . I would also recommend posting this question on https://discuss.mxnet.io/

@mxnet-label-bot add [Question]

[zhoulukuan]

@zhoulukuan
Here is the link to ImageIterator and related APIs in MXNet. https://mxnet.incubator.apache.org/versions/master/api/python/image/image.html#module-mxnet.image

I have tagged this question so that experts can look into this . I would also recommend posting this question on https://discuss.mxnet.io/

@mxnet-label-bot add [Question]

OK. I will post this question.