[sonic-yang-mgmt] Build PY3 & PY2 packages

[praveen-li]

Changes:
-- change dep and mk files in rules for PY2.
-- add dep and mk files in rules for PY3.

Signed-off-by: Praveen Chaudhary[email protected]

- Why I did it
Moving sonic-yang-mgmt to PY3 to support move of sonic-utilities to PY3.

- How I did it

- How to verify it
Build time Test and VS tests must pass with new PKG.

- Which release branch to backport (provide reason below if selected)

• 201811
• 201911
• 202006

- Description for the changelog

- A picture of a cute animal (not mandatory but encouraged)

[jleveque]

@praveen-li: There are build failures. Can you check? Also, it appears you are no longer building a Python 2 version of the package. Will the Python 2 version of sonic-utilities continue to work until I finish converting it to Python 3?

[praveen-li]

retest vs please

[praveen-li]

@jleveque

I think, to support test run in sonic-slave docker, we need these PKGs there. Because below errors are seen, even when tests_require is added.

06:55:25  Ran 0 tests in 0.000s
06:55:25  
06:55:25  OK
06:55:25  running bdist_wheel
06:55:25  running build
06:55:25  running build_py
06:55:25  Traceback (most recent call last):
06:55:25    File "./tests/yang_model_tests/yangModelTesting.py", line 7, in <module>
06:55:25      import ijson
06:55:25  ImportError: No module named ijson
06:55:25  YANG Tests failed

[jleveque]

From the build output, I don't see any attempt from setuptools to install the package. Also, it shows that zero tests were run. Is this expected? If not, it seems like there may be other issues with your setup.py file.

[praveen-li]

From the build output, I don't see any attempt from setuptools to install the package. Also, it shows that zero tests were run. Is this expected? If not, it seems like there may be other issues with your setup.py file.

[PC]: Zero tests are expected due to import error. But let me check because I noticed an attempt to install PKGs in earlier builds.

[praveen-li]

From my build, I see an attempt to install the PKGs, I feel it is done because of install_require part.
But then still the tests are failing while importing those packages

Searching for ijson==2.6.1
Best match: ijson 2.6.1
Processing ijson-2.6.1-py2.7-linux-x86_64.egg

Using /sonic/src/sonic-yang-mgmt/.eggs/ijson-2.6.1-py2.7-linux-x86_64.egg
Searching for xmltodict==0.12.0
Best match: xmltodict 0.12.0
Processing xmltodict-0.12.0-py2.7.egg

Using /sonic/src/sonic-yang-mgmt/.eggs/xmltodict-0.12.0-py2.7.egg
running egg_info
writing requirements to sonic_yang_mgmt.egg-info/requires.txt
writing sonic_yang_mgmt.egg-info/PKG-INFO
writing top-level names to sonic_yang_mgmt.egg-info/top_level.txt
writing dependency_links to sonic_yang_mgmt.egg-info/dependency_links.txt
reading manifest file 'sonic_yang_mgmt.egg-info/SOURCES.txt'
writing manifest file 'sonic_yang_mgmt.egg-info/SOURCES.txt'
running build_ext
----------------------------------------------------------------------
Ran 0 tests in 0.000s

OK
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'setup_requirements'
  warnings.warn(msg)
running bdist_wheel
running build
running build_py
  =========================== test session starts ==============================
platform linux2 -- Python 2.7.16, pytest-3.10.1, py-1.7.0, pluggy-0.8.0
rootdir: /sonic/src/sonic-yang-mgmt, inifile:
plugins: cov-2.6.0
collected 1 item / 1 errors

==================================== ERRORS ====================================
________ ERROR collecting tests/libyang-python-tests/test_sonic_yang.py ________
ImportError while importing test module '/sonic/src/sonic-yang-mgmt/tests/libyang-python-tests/test_sonic_yang.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python2.7/dist-packages/six.py:709: in exec_
    exec("""exec _code_ in _globs_, _locs_""")
tests/libyang-python-tests/test_sonic_yang.py:4: in <module>
    import sonic_yang as sy
sonic_yang.py:6: in <module>
    from sonic_yang_ext import SonicYangExtMixin, SonicYangException
sonic_yang_ext.py:10: in <module>
    from xmltodict import parse
E   ImportError: No module named xmltodict
!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!
=========================== 1 error in 0.25 seconds ============================

[jleveque]

@praveen-li: Your build output is different from the check build logs here. I'm not sure why. I don't understand what's going on here, but it appears the dependencies aren't getting installed properly. I haven't encountered this issue with other packages.

[jleveque]

Your setup.py file seems to be a bit more complex than others in the SONiC codebase (like sonic-utilities). Maybe the issue has something to do with the additional functions you define, like pkgBuild()?

[lgtm-com]

This pull request introduces 2 alerts and fixes 1 when merging 46f1e6b809ba9a672b805607f3b6b2ee2fa10865 into 67408c8 - view on LGTM.com

new alerts:

• 2 for Unused import

fixed alerts:

• 1 for Unused import

[lgtm-com]

This pull request introduces 2 alerts and fixes 1 when merging a4f359d0aeda52f5a7dc1b01833da2f7695741e4 into 9e34003 - view on LGTM.com

new alerts:

• 2 for Unused import

fixed alerts:

• 1 for Unused import

[lgtm-com]

This pull request fixes 1 alert when merging f43400274e07b3596fda178c7e073018d2ab3a49 into 9e34003 - view on LGTM.com

fixed alerts:

• 1 for Unused import

[lgtm-com]

This pull request fixes 1 alert when merging 30d76ec1550d8fc5c43abaef1fed1e6bcb6e3f6d into dfe0055 - view on LGTM.com

fixed alerts:

• 1 for Unused import

[praveen-li]

retest mellanox please

[lgtm-com]

This pull request fixes 1 alert when merging b54e9dc8a2beccb7b5064dd93dab58b7e64cd716 into 09d5a62 - view on LGTM.com

fixed alerts:

• 1 for Unused import

[jleveque]

Retest broadcom please

[jleveque]

Retest mellanox please

[jleveque]

Retest vsimage please

[lguohan]

retest vsimage please

[lguohan]

retest vsimage please

[lgtm-com]

This pull request fixes 1 alert when merging 9207b2f into 13ff7b3 - view on LGTM.com

fixed alerts:

• 1 for Unused import

[lguohan]

retest broadcom please

[lguohan]

@jleveque, can you check this pr? everything passed.

[praveen-li]

@jleveque let me know if anything pending on me, thanks.

[qiluo-msft]

                     './yang-models/sonic_yang_tree']),

This file is generated by test, not part of source code.
This PR breaks pip3 install . if the file is not existing.

---

In reply to: 947590563

---

Refers to: src/sonic-yang-models/setup.py:57 in 9207b2f. [](commit_id = 9207b2f, deletion_comment = False)

[qiluo-msft]

                     './yang-models/sonic_yang_tree']),

How critical is this file? I think it is just for document purpose. A quick way to unblock the command is to remvoe it.

---

In reply to: 947590563

---

Refers to: src/sonic-yang-models/setup.py:57 in 9207b2f. [](commit_id = 9207b2f, deletion_comment = False)

[praveen-li]

                     './yang-models/sonic_yang_tree']),

This file is generated by test, not part of source code. This PR breaks pip3 install . if the file is not existing.

Refers to: src/sonic-yang-models/setup.py:57 in 9207b2f. [](commit_id = 9207b2f, deletion_comment = False)

Hi Qi,

This file is auto-generated by test. We made it part of test, because it also verified that all syntax and hierarchy in yang models are correct.
We need this file, if It is not generated then YANG models are not correct and will fail at runtime. Thx.

[qiluo-msft]

                     './yang-models/sonic_yang_tree']),

Offline discussed. We could ondemand get the content by

pyang -Vf tree -p /usr/local/share/yang/modules/ietf /usr/local/yang-models/*.yang

It is not a critical content in the package, so we can safely remove it.

---

In reply to: 947591498

---

Refers to: src/sonic-yang-models/setup.py:57 in 9207b2f. [](commit_id = 9207b2f, deletion_comment = False)