Add PaddlePaddle Classification notebook

[helena-intel]

PaddlePaddle Classification notebook showing classification on MobileNet model from PaddleHub. Model is downloaded, converted to ONNX and IR and inference results are shown. Model source: https://www.paddlepaddle.org.cn/hubdetail?name=mobilenet_v3_large_imagenet_ssld&en_category=ImageClassification

The PaddlePaddle model provides a very useful .classification() method that takes a path to an input images and returns a class name. I liked that a lot. But they do quite a bit of preprocessing behind the scenes, and to get the same results, you need to do the same preprocessing. I opted to just import their process_image function in this notebook, and not include mean and std in the MO command to include in the model. I added markdown text to explain that you can do that and how to get the mean and std from the classification model (PaddlePaddle includes that information in the model - really cool!).

[ryanloney]

On my first try running in an environment where openvino_notebooks requirements were already installed. Here is what I see:

ERROR: Could not install packages due to an OSError: [WinError 5] Access is denied: 'c:\\users\\ryanlone\\code\\repos\\openvino_env\\lib\\site-packages\\~umpy\\.libs\\libopenblas.WCDJNK7YVMPZQ2ME2ZZHJJRJ3JIKNDB7.gfortran-win_amd64.dll'
Check the permissions.

[ryanloney]

It looks like tensorflow 2.5.0 was the problem. I was able to install but now running into this error. I will try clearing out the directory.

OSError: [WinError 145] The directory is not empty: 'C:\\Users\\ryanlone\\.paddlehub\\tmp\\tmp71tp1xcz\\mobilenet_v3_large_imagenet_ssld'

[helena-intel]

@ryanloney Sometimes these errors also go away by stopping all kernels (from the Kernel menu).

Maybe we should add PaddlePaddle to the requirements.txt.

[ryanloney]

I deleted the .paddlehub directory and the notebook works now. I think adding PaddlePaddle to the requirements is a good idea... if you don't think it adds too much footprint?

The classification is accurate! Coco is indeed a Labrador retriever (mix)
OpenVINO result: ['Labrador retriever', 0.60035]

[helena-intel]

I deleted the .paddlehub directory and the notebook works now. I think adding PaddlePaddle to the requirements is a good idea... if you don't think it adds too much footprint?

It does add up. It is another 500-600MB and we already are at 2,3GB. We may need to reconsider how we install MO requirements in the future and have some optional things. Like how with openvino-dev you can also install openvino-dev[tf2] to just install the MO reqs for tensorflow. But if we do that, it would be good if for the 200-series demos and the POT tutorials we can always include an IR model, so you only need the big frameworks if you actually want to learn how to convert a model from that framework.

For now, in my opinion it is better to include it in the requirements than install in the notebook. This also makes our Docker image easier to maintain. If you want to keep it in the install cell for now, I can also do pip install -r ..\requirements.txt instead of upgrading openvino-dev, with a flag that makes sure to downgrade tensorflow 2.5 to 2.4 if it is installed (and the added bonus that all requirements are up to date again).

The classification is accurate! Coco is indeed a Labrador retriever (mix)
OpenVINO result: ['Labrador retriever', 0.60035]

That's great! I want to do a battle of the frameworks now and compare TensorFlow, PyTorch and PaddlePaddle on MobileNet and Coco classification :-)

[ryanloney]

I tried running this notebook on RHODS environment and seeing a permissions issue: PermissionError: [Errno 13] Permission denied: '/opt/app_root'

At this step:

This is the full error:

---------------------------------------------------------------------------
PermissionError                           Traceback (most recent call last)
/tmp/ipykernel_375/1833839113.py in <module>
----> 1 classifier = hub.Module(name=MODEL_NAME)
      2 # Load image in BGR format, as specified in model documentation
      3 image = cv2.imread(IMAGE_FILENAME)
      4 plt.imshow(cv2.cvtColor(image, cv2.COLOR_BGR2RGB))
      5 result = classifier.classification(images=[image])

/opt/app-root/lib64/python3.8/site-packages/paddlehub/module/module.py in __new__(cls, name, directory, version, source, update, branch, ignore_env_mismatch, **kwargs)
    386             # This branch come from hub.Module(name='xxx') or hub.Module(directory='xxx')
    387             if name:
--> 388                 module = cls.init_with_name(
    389                     name=name,
    390                     version=version,

/opt/app-root/lib64/python3.8/site-packages/paddlehub/module/module.py in init_with_name(cls, name, version, source, update, branch, ignore_env_mismatch, **kwargs)
    485         user_module_cls = manager.search(name, source=source, branch=branch)
    486         if not user_module_cls or not user_module_cls.version.match(version):
--> 487             user_module_cls = manager.install(
    488                 name=name,
    489                 version=version,

/opt/app-root/lib64/python3.8/site-packages/paddlehub/module/manager.py in install(self, name, directory, archive, url, version, source, update, branch, ignore_env_mismatch)
    188                 if source:
    189                     return self._install_from_source(name, version, source, update, branch)
--> 190                 return self._install_from_name(name, version, ignore_env_mismatch)
    191         elif directory:
    192             return self._install_from_directory(directory)

/opt/app-root/lib64/python3.8/site-packages/paddlehub/module/manager.py in _install_from_name(self, name, version, ignore_env_mismatch)
    263         for item in result:
    264             if name.lower() == item['name'].lower() and utils.Version(item['version']).match(version):
--> 265                 return self._install_from_url(item['url'])
    266 
    267         module_infos = module_server.get_module_compat_info(name=name)

/opt/app-root/lib64/python3.8/site-packages/paddlehub/module/manager.py in _install_from_url(self, url)
    256                     bar.update(float(ds) / ts)
    257 
--> 258             return self._install_from_archive(file)
    259 
    260     def _install_from_name(self, name: str, version: str = None, ignore_env_mismatch: bool = False) -> HubModule:

/opt/app-root/lib64/python3.8/site-packages/paddlehub/module/manager.py in _install_from_archive(self, archive)
    378             path = os.path.normpath(path)
    379             directory = os.path.join(_tdir, path.split(os.sep)[0])
--> 380             return self._install_from_directory(directory)
    381 
    382     def _install_module_requirements(self, directory: str):

/opt/app-root/lib64/python3.8/site-packages/paddlehub/module/manager.py in _install_from_directory(self, directory)
    350             tempdir = os.path.join(_dir, module_info.name)
    351             tempdir = self._get_normalized_name(tempdir)
--> 352             shutil.copytree(directory, tempdir)
    353 
    354             # Uninstall local module

/usr/lib64/python3.8/shutil.py in copytree(src, dst, symlinks, ignore, copy_function, ignore_dangling_symlinks, dirs_exist_ok)
    552     with os.scandir(src) as itr:
    553         entries = list(itr)
--> 554     return _copytree(entries=entries, src=src, dst=dst, symlinks=symlinks,
    555                      ignore=ignore, copy_function=copy_function,
    556                      ignore_dangling_symlinks=ignore_dangling_symlinks,

/usr/lib64/python3.8/shutil.py in _copytree(entries, src, dst, symlinks, ignore, copy_function, ignore_dangling_symlinks, dirs_exist_ok)
    453         ignored_names = set()
    454 
--> 455     os.makedirs(dst, exist_ok=dirs_exist_ok)
    456     errors = []
    457     use_srcentry = copy_function is copy2 or copy_function is copy

/usr/lib64/python3.8/os.py in makedirs(name, mode, exist_ok)
    211     if head and tail and not path.exists(head):
    212         try:
--> 213             makedirs(head, exist_ok=exist_ok)
    214         except FileExistsError:
    215             # Defeats race condition when another thread created the path

/usr/lib64/python3.8/os.py in makedirs(name, mode, exist_ok)
    211     if head and tail and not path.exists(head):
    212         try:
--> 213             makedirs(head, exist_ok=exist_ok)
    214         except FileExistsError:
    215             # Defeats race condition when another thread created the path

/usr/lib64/python3.8/os.py in makedirs(name, mode, exist_ok)
    211     if head and tail and not path.exists(head):
    212         try:
--> 213             makedirs(head, exist_ok=exist_ok)
    214         except FileExistsError:
    215             # Defeats race condition when another thread created the path

/usr/lib64/python3.8/os.py in makedirs(name, mode, exist_ok)
    211     if head and tail and not path.exists(head):
    212         try:
--> 213             makedirs(head, exist_ok=exist_ok)
    214         except FileExistsError:
    215             # Defeats race condition when another thread created the path

/usr/lib64/python3.8/os.py in makedirs(name, mode, exist_ok)
    211     if head and tail and not path.exists(head):
    212         try:
--> 213             makedirs(head, exist_ok=exist_ok)
    214         except FileExistsError:
    215             # Defeats race condition when another thread created the path

/usr/lib64/python3.8/os.py in makedirs(name, mode, exist_ok)
    221             return
    222     try:
--> 223         mkdir(name, mode)
    224     except OSError:
    225         # Cannot rely on checking for EEXIST, since the operating system

PermissionError: [Errno 13] Permission denied: '/opt/app_root'

[helena-intel]

I tried running this notebook on RHODS environment and seeing a permissions issue: PermissionError: [Errno 13] Permission denied: '/opt/app_root'

Thanks! I'll fix it. Also added Docker tests to tests that are run when a PR is made so we will catch this with CI in the future #176

[helena-intel]

This is an interesting bug... PaddlePaddle replaces dashes with underscores (?), so instead of /opt/apt-root it wants to write to /opt/apt_root, which does not exist, so it tries to make the directory, but does not have permission to write to /opt.

[helena-intel]

I set HUB_HOME to /tmp to prevent this renaming issue - not ideal, but it works (on my machine). Sadly the docker test does not work yet because it pulls in the notebooks from main. I'll fix that later.

[ryanloney]

That should work fine. There is a /tmp in the OpenShift test cluster and users have read/write permission. Thanks for finding the problem!

[helena-intel]

Tests were failing because of #182, now fixed. I tested the Docker solution manually.