[Nokia] Add protobuf and grpc C++ and python lib to support Nokia IXR7250E platform #8366

mlok-nokia · 2021-08-06T22:33:06Z

Why I did it

Nokia IXR7250E platform requires grpcio, grpcio-tools python library, and libprotobuf-dev, libgrpc++ library

How I did it

Modified the build_debian.sh install libprotobuf-dev and libgrpc++ to support nokia ndk
Modified the sonic_debian_extension.j2 to install the grpcio and grpcio-tools in the host
Modified the docker-platform-monitor/Dockerfile.js to install grpcio and grpcio-tools for the pmon container.

How to verify it

Image running success.

Which release branch to backport (provide reason below if selected)

Description for the changelog

A picture of a cute animal (not mandatory but encouraged)

mlok-nokia · 2021-08-09T14:22:03Z

@lguohan @judyjoseph @minionatwork Could you please review this PR

minionatwork · 2021-08-16T20:00:17Z

@judyjoseph @abdosi Can you please review this one as well?

mlok-nokia · 2021-08-18T14:24:30Z

@lguohan @judyjoseph @abdosi @minionatwork Please help to review this PR

mlok-nokia · 2021-08-24T17:17:32Z

@lguohan @judyjoseph @abdosi @minionatwork Please help to review this PR

mlok-nokia · 2021-08-30T22:54:36Z

@judyjoseph @lguohan Please help to review this PR. Thanks

mlok-nokia · 2021-09-21T18:06:24Z

@lguohan @Blueve @judyjoseph @minionatwork Please help to review this PR. Thanks

anshuv-mfst · 2021-09-22T16:55:03Z

Hi @ysmanman - please assign reviewers from Arista for this PR.
@abdosi - please review or add respective reviewers from MSFT team.

mlok-nokia · 2021-09-29T15:37:51Z

@abdosi Song has approved this PR. Could you please review it and help to merge it to master branch?

judyjoseph · 2021-10-14T20:45:56Z

LGTM, Adding @qiluo-msft to take a look into this PR as well.

qiluo-msft · 2021-10-15T06:35:33Z

build_debian.sh

@@ -319,6 +319,8 @@ sudo LANG=C DEBIAN_FRONTEND=noninteractive chroot $FILESYSTEM_ROOT apt-get -y in
    python3-distutils       \
    python3-pip             \
    cron                    \
+    libprotobuf-dev         \
+    libgrpc++               \


Do we truly need dev packages in sonic image? #Closed

I feel you install them because pip need them to build wheel from source code. Then is it possible to prebuilt and minimize the sonic image disk usage?

I have updated the commit to change the libprotobuf-dev to the buster libprotobuf17.

qiluo-msft · 2021-10-15T06:42:19Z

dockers/docker-platform-monitor/Dockerfile.j2

@@ -42,7 +42,12 @@ RUN pip2 install 'setuptools==40.8.0'
 # NOTE: Only install enum34 for Python 2, as our version of Python 3 is 3.7, which
 # contains 'enum' as part of the standard library. Installing enum34 there will
 # cause conflicts.
-RUN pip2 install enum34
+RUN pip2 install enum34 \
+        grpcio==1.26.0 \


grpcio==1.26.0

grpcio==1.26.0 \

why pip2 need install dependency explicitly, pip3 does not? #Closed

We cannot use the default version because upstream is always changing and current version 1.41.0 doesn't build for python2
I have updated to commit to change the grpcio version from 1.26.0 to the 1.39.0 for both pip2 and pip3.

qiluo-msft · 2021-10-15T06:42:49Z

dockers/docker-platform-monitor/Dockerfile.j2

+        grpcio-tools==1.26.0
+
+RUN pip3 install grpcio==1.26.0 \
+        grpcio-tools==1.26.0


1.26.0

grpcio-tools==1.26.0

Why specify old version? any blocking issue to use latest version? #Closed

See previous comment.

…platform build Signed-off-by: mlok <[email protected]>

-- Changed the libbrotbuf-dev to buster lightly version libprotobuf17 -- Change the grpcio 1.26.0 to 1.39.0

mlok-nokia · 2021-10-22T16:13:02Z

@ysmanman Please help to review this PR.

mlok-nokia · 2021-10-25T21:02:41Z

@qiluo-msft Could you please help to merge this PR? or need other people's review

xumia · 2022-03-08T14:15:08Z

@mlok-nokia , do you know how to support to install grpcio in armhf?

/# uname -m
armv8l

2022-03-08T11:27:46.4951787Z Collecting grpcio==1.39.0
2022-03-08T11:27:46.4952884Z   Downloading grpcio-1.39.0.tar.gz (21.3 MB)
2022-03-08T11:27:46.4953551Z      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 21.3/21.3 MB 33.8 MB/s eta 0:00:00
2022-03-08T11:27:46.4953995Z   Preparing metadata (setup.py): started
2022-03-08T11:27:46.4954617Z   Preparing metadata (setup.py): finished with status 'error'
2022-03-08T11:27:46.4955235Z   error: subprocess-exited-with-error
2022-03-08T11:27:46.4955564Z   
2022-03-08T11:27:46.4956066Z   × python setup.py egg_info did not run successfully.
2022-03-08T11:27:46.4956601Z   │ exit code: 1
2022-03-08T11:27:46.4963191Z   ╰─> [12 lines of output]
2022-03-08T11:27:46.4963614Z       Traceback (most recent call last):
2022-03-08T11:27:46.4963995Z         File "<string>", line 2, in <module>
2022-03-08T11:27:46.4964573Z         File "<pip-setuptools-caller>", line 34, in <module>
2022-03-08T11:27:46.4965359Z         File "/tmp/pip-install-cylmaawx/grpcio_633d44a4a5d345fbaaab0951bf1d9d3f/setup.py", line 257, in <module>
2022-03-08T11:27:46.4965900Z           if check_linker_need_libatomic():
2022-03-08T11:27:46.4966699Z         File "/tmp/pip-install-cylmaawx/grpcio_633d44a4a5d345fbaaab0951bf1d9d3f/setup.py", line 204, in check_linker_need_libatomic
2022-03-08T11:27:46.4967548Z           cpp_test = subprocess.Popen([cxx, '-x', 'c++', '-std=c++11', '-'],
2022-03-08T11:27:46.4968062Z         File "/usr/lib/python3.9/subprocess.py", line 951, in __init__
2022-03-08T11:27:46.4968555Z           self._execute_child(args, executable, preexec_fn, close_fds,
2022-03-08T11:27:46.4969072Z         File "/usr/lib/python3.9/subprocess.py", line 1823, in _execute_child
2022-03-08T11:27:46.4969578Z           raise child_exception_type(errno_num, err_msg, err_filename)
2022-03-08T11:27:46.4970264Z       FileNotFoundError: [Errno 2] No such file or directory: 'c++'
2022-03-08T11:27:46.4970671Z       [end of output]
2022-03-08T11:27:46.4970940Z   
2022-03-08T11:27:46.4971351Z   note: This error originates from a subprocess, and is likely not a problem with pip.
2022-03-08T11:27:46.4972008Z error: metadata-generation-failed
2022-03-08T11:27:46.4972189Z 
2022-03-08T11:27:46.4972951Z × Encountered error while generating package metadata.
2022-03-08T11:27:46.4973503Z ╰─> See above for output.

xumia · 2022-03-08T14:19:09Z

See https://dev.azure.com/mssonic/build/_build/results?buildId=79031&view=logs&j=268525a4-f0a8-5106-2898-2602f02ebdb0&t=59457689-49a2-5da2-bad6-5355ff2da0c7

mlok-nokia · 2022-03-09T16:58:33Z

@mlok-nokia , do you know how to support to install grpcio in armhf?

/# uname -m armv8l

2022-03-08T11:27:46.4951787Z Collecting grpcio==1.39.0
2022-03-08T11:27:46.4952884Z   Downloading grpcio-1.39.0.tar.gz (21.3 MB)
2022-03-08T11:27:46.4953551Z      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 21.3/21.3 MB 33.8 MB/s eta 0:00:00
2022-03-08T11:27:46.4953995Z   Preparing metadata (setup.py): started
2022-03-08T11:27:46.4954617Z   Preparing metadata (setup.py): finished with status 'error'
2022-03-08T11:27:46.4955235Z   error: subprocess-exited-with-error
2022-03-08T11:27:46.4955564Z   
2022-03-08T11:27:46.4956066Z   × python setup.py egg_info did not run successfully.
2022-03-08T11:27:46.4956601Z   │ exit code: 1
2022-03-08T11:27:46.4963191Z   ╰─> [12 lines of output]
2022-03-08T11:27:46.4963614Z       Traceback (most recent call last):
2022-03-08T11:27:46.4963995Z         File "<string>", line 2, in <module>
2022-03-08T11:27:46.4964573Z         File "<pip-setuptools-caller>", line 34, in <module>
2022-03-08T11:27:46.4965359Z         File "/tmp/pip-install-cylmaawx/grpcio_633d44a4a5d345fbaaab0951bf1d9d3f/setup.py", line 257, in <module>
2022-03-08T11:27:46.4965900Z           if check_linker_need_libatomic():
2022-03-08T11:27:46.4966699Z         File "/tmp/pip-install-cylmaawx/grpcio_633d44a4a5d345fbaaab0951bf1d9d3f/setup.py", line 204, in check_linker_need_libatomic
2022-03-08T11:27:46.4967548Z           cpp_test = subprocess.Popen([cxx, '-x', 'c++', '-std=c++11', '-'],
2022-03-08T11:27:46.4968062Z         File "/usr/lib/python3.9/subprocess.py", line 951, in __init__
2022-03-08T11:27:46.4968555Z           self._execute_child(args, executable, preexec_fn, close_fds,
2022-03-08T11:27:46.4969072Z         File "/usr/lib/python3.9/subprocess.py", line 1823, in _execute_child
2022-03-08T11:27:46.4969578Z           raise child_exception_type(errno_num, err_msg, err_filename)
2022-03-08T11:27:46.4970264Z       FileNotFoundError: [Errno 2] No such file or directory: 'c++'
2022-03-08T11:27:46.4970671Z       [end of output]
2022-03-08T11:27:46.4970940Z   
2022-03-08T11:27:46.4971351Z   note: This error originates from a subprocess, and is likely not a problem with pip.
2022-03-08T11:27:46.4972008Z error: metadata-generation-failed
2022-03-08T11:27:46.4972189Z 
2022-03-08T11:27:46.4972951Z × Encountered error while generating package metadata.
2022-03-08T11:27:46.4973503Z ╰─> See above for output.

Hi xumia, This grpc installation is introduced for the amd64 image build. Other platform_arch build may not need it. Please try the following. It should address this build issue.

Modify the the docker/docker-platform-monitor/Dockerfile.j2.

{% if CONFIGURED_ARCH == "amd64" %}
RUN pip3 install grpcio==1.39.0
grpcio-tools==1.39.0
{%- endif %}

…7250E platform (#8366) Nokia IXR7250E platform requires grpcio, grpcio-tools python library, and libprotobuf-dev, libgrpc++ library Modified the build_debian.sh install libprotobuf-dev and libgrpc++ to support nokia ndk Modified the sonic_debian_extension.j2 to install the grpcio and grpcio-tools in the host Modified the docker-platform-monitor/Dockerfile.js to install grpcio and grpcio-tools for the pmon container. Image running success.

mlok-nokia requested a review from lguohan as a code owner August 6, 2021 22:33

mlok-nokia changed the title ~~[Nokia] Add protobuf and grpc C++ and python lib to support nokia platform image build~~ [Nokia] Add protobuf and grpc C++ and python lib to support Nokia IXR7250E platform Aug 6, 2021

mlok-nokia force-pushed the build_grpclib branch from 2e5632a to b4bac75 Compare August 16, 2021 17:18

mlok-nokia force-pushed the build_grpclib branch from b4bac75 to 212a478 Compare August 17, 2021 14:03

Blueve added the Platform: Nokia label Aug 20, 2021

ysmanman previously approved these changes Sep 28, 2021

View reviewed changes

judyjoseph requested a review from qiluo-msft October 14, 2021 20:46

qiluo-msft reviewed Oct 15, 2021

View reviewed changes

mlok-nokia added 2 commits October 21, 2021 10:36

[platform] Add protobuf and grpc C++ and python lib to support nokia …

8291266

…platform build Signed-off-by: mlok <[email protected]>

Update the protobuf and grpcio lib version

c61d380

-- Changed the libbrotbuf-dev to buster lightly version libprotobuf17 -- Change the grpcio 1.26.0 to 1.39.0

mlok-nokia dismissed ysmanman’s stale review via c61d380 October 21, 2021 15:01

mlok-nokia force-pushed the build_grpclib branch from 212a478 to c61d380 Compare October 21, 2021 15:01

qiluo-msft approved these changes Oct 22, 2021

View reviewed changes

qiluo-msft merged commit b91190d into sonic-net:master Oct 27, 2021

judyjoseph added the Chassis 🤖 Modular chassis support label Oct 27, 2021

xumia mentioned this pull request Mar 11, 2022

[Build]: only install grpc in amd64 #10212

Merged

6 tasks

mlok-nokia deleted the build_grpclib branch April 16, 2022 17:06

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Nokia] Add protobuf and grpc C++ and python lib to support Nokia IXR7250E platform #8366

[Nokia] Add protobuf and grpc C++ and python lib to support Nokia IXR7250E platform #8366

mlok-nokia commented Aug 6, 2021 •

edited

Loading

mlok-nokia commented Aug 9, 2021

minionatwork commented Aug 16, 2021

mlok-nokia commented Aug 18, 2021

mlok-nokia commented Aug 24, 2021

mlok-nokia commented Aug 30, 2021

mlok-nokia commented Sep 21, 2021

anshuv-mfst commented Sep 22, 2021

mlok-nokia commented Sep 29, 2021

judyjoseph commented Oct 14, 2021

qiluo-msft Oct 15, 2021 •

edited

Loading

qiluo-msft Oct 15, 2021

mlok-nokia Oct 21, 2021 •

edited

Loading

qiluo-msft Oct 15, 2021 •

edited

Loading

mlok-nokia Oct 21, 2021

qiluo-msft Oct 15, 2021 •

edited

Loading

mlok-nokia Oct 21, 2021

mlok-nokia commented Oct 22, 2021

mlok-nokia commented Oct 25, 2021

xumia commented Mar 8, 2022

xumia commented Mar 8, 2022

mlok-nokia commented Mar 9, 2022 •

edited

Loading

[Nokia] Add protobuf and grpc C++ and python lib to support Nokia IXR7250E platform #8366

[Nokia] Add protobuf and grpc C++ and python lib to support Nokia IXR7250E platform #8366

Conversation

mlok-nokia commented Aug 6, 2021 • edited Loading

Why I did it

How I did it

How to verify it

Which release branch to backport (provide reason below if selected)

Description for the changelog

A picture of a cute animal (not mandatory but encouraged)

mlok-nokia commented Aug 9, 2021

minionatwork commented Aug 16, 2021

mlok-nokia commented Aug 18, 2021

mlok-nokia commented Aug 24, 2021

mlok-nokia commented Aug 30, 2021

mlok-nokia commented Sep 21, 2021

anshuv-mfst commented Sep 22, 2021

mlok-nokia commented Sep 29, 2021

judyjoseph commented Oct 14, 2021

qiluo-msft Oct 15, 2021 • edited Loading

Choose a reason for hiding this comment

qiluo-msft Oct 15, 2021

Choose a reason for hiding this comment

mlok-nokia Oct 21, 2021 • edited Loading

Choose a reason for hiding this comment

qiluo-msft Oct 15, 2021 • edited Loading

Choose a reason for hiding this comment

grpcio==1.26.0 \

mlok-nokia Oct 21, 2021

Choose a reason for hiding this comment

qiluo-msft Oct 15, 2021 • edited Loading

Choose a reason for hiding this comment

grpcio-tools==1.26.0

mlok-nokia Oct 21, 2021

Choose a reason for hiding this comment

mlok-nokia commented Oct 22, 2021

mlok-nokia commented Oct 25, 2021

xumia commented Mar 8, 2022

xumia commented Mar 8, 2022

mlok-nokia commented Mar 9, 2022 • edited Loading

mlok-nokia commented Aug 6, 2021 •

edited

Loading

qiluo-msft Oct 15, 2021 •

edited

Loading

mlok-nokia Oct 21, 2021 •

edited

Loading

qiluo-msft Oct 15, 2021 •

edited

Loading

qiluo-msft Oct 15, 2021 •

edited

Loading

mlok-nokia commented Mar 9, 2022 •

edited

Loading