[dockers] label SONiC Docker with manifest #5939

stepanblyschak · 2020-11-16T20:37:31Z

Signed-off-by: Stepan Blyschak [email protected]

This PR is part of SONiC Application Extension

Depends on #5938

- Why I did it
To provide an infrastructure change in order to support SONiC Application Extension feature.

- How I did it
Label every installable SONiC Docker with a minimal required manifest and auto-generate packages.json file based on
installed SONiC images.

- How to verify it
Build an image, execute the following command:

admin@sonic:~$ docker inspect docker-snmp:1.0.0 | jq '.[0].Config.Labels["com.azure.sonic.manifest"]' -r | jq

Cat /var/lib/sonic-package-manager/packages.json file to verify all dockers are listed there.

- Which release branch to backport (provide reason below if selected)

201811
201911
202006

- Description for the changelog

- A picture of a cute animal (not mandatory but encouraged)

Previously all docker images built into SONiC binary were tagged with SONiC version string. With the introduction of container upgrades and SONiC extension framework all docker images should have their own version numbers that do not relate with SONiC version. All dockers are tagged with version 1.0.0 in this PR. Whenever changes are introduced to a docker image that require a version change, the docker image maintainer should update the version number. Signed-off-by: Stepan Blyschak <[email protected]>

stepanblyschak · 2020-11-17T10:26:32Z

retest mellanox please

stepanblyschak · 2020-11-17T11:41:50Z

retest mellanox please

stepanblyschak · 2020-11-19T08:42:55Z

retest mellanox please

Previously all docker images built into SONiC binary were tagged with SONiC version string. With the introduction of container upgrades and SONiC extension framework all docker images should have their own version numbers that do not relate with SONiC version. All dockers are tagged with version 1.0.0 in this PR. Whenever changes are introduced to a docker image that require a version change, the docker image maintainer should update the version number. Signed-off-by: Stepan Blyschak <[email protected]>

Signed-off-by: Stepan Blyshchak <[email protected]>

stepanblyschak · 2020-12-07T06:51:24Z

retest broadcom please

Signed-off-by: Stepan Blyshchak <[email protected]>

…rs_manifest Signed-off-by: Stepan Blyshchak <[email protected]>

…rs_version_tags Signed-off-by: Stepan Blyshchak <[email protected]>

…rs_manifest Signed-off-by: Stepan Blyshchak <[email protected]>

Signed-off-by: Stepan Blyshchak <[email protected]>

…rs_manifest

…rs_version_tags

Signed-off-by: Stepan Blyshchak <[email protected]>

…c-buildimage into dockers_manifest Signed-off-by: Stepan Blyshchak <[email protected]>

Signed-off-by: Stepan Blyshchak <[email protected]>

rajendra-dendukuri

Can we create various "manifest.json.j2" files from a single template file. A lot of its contents is repetitive.

manifest_json_tmpl.j2 -->docker/docker-$var/mainfiest.json.j2

Signed-off-by: Stepan Blyshchak <[email protected]>

Signed-off-by: Stepan Blyschak <[email protected]>

…ildimage into dockers_manifest Signed-off-by: Stepan Blyschak <[email protected]>

files/build_templates/manifest.json.j2

…onents approacgh Signed-off-by: Stepan Blyschak <[email protected]>

…rs_manifest

dockers/docker-database/Dockerfile.j2

dockers/docker-dhcp-relay/Dockerfile.j2

files/build_templates/manifest.json.j2

slave.mk

dockers/docker-database/Dockerfile.j2

files/build_templates/sonic_debian_extension.j2

platform/mellanox/docker-syncd-mlnx.mk

platform/mellanox/docker-syncd-mlnx-rpc.mk

slave.mk

Signed-off-by: Stepan Blyschak <[email protected]>

liat-grozovik · 2021-04-21T11:12:07Z

/azp run

azure-pipelines · 2021-04-21T11:12:24Z

Azure Pipelines successfully started running 1 pipeline(s).

Signed-off-by: Stepan Blyschak <[email protected]>

Signed-off-by: Stepan Blyschak [email protected] This PR is part of SONiC Application Extension Depends on sonic-net#5938 - Why I did it To provide an infrastructure change in order to support SONiC Application Extension feature. - How I did it Label every installable SONiC Docker with a minimal required manifest and auto-generate packages.json file based on installed SONiC images. - How to verify it Build an image, execute the following command: admin@sonic:~$ docker inspect docker-snmp:1.0.0 | jq '.[0].Config.Labels["com.azure.sonic.manifest"]' -r | jq Cat /var/lib/sonic-package-manager/packages.json file to verify all dockers are listed there.

- Why I did it Make DHCP relay docker an extension. DHCP relay now carries dhcp relay commands CLI plugin and has a complete manifest. It is installed as extension if INCLUDE_DHCP_REALY is set to y. DEPENDS on #5939 - How I did it Modify DHCP relay docker makefile and dockerfile. Make changes to sonic_debian_extension.j2 to install sonic packages. I moved DHCP related CLI tests from sonic-utilities to DHCP relay docker. This PR introduces a way to write a plugin as part of docker image and run the tests from cli-plugin-tests directory under docker directory. The test result is available in target/docker-dhcp-relay.gz.log: [ REASON ] : target/docker-dhcp-relay.gz does not exist NON-EXISTENT PREREQUISITES: docker-start target/docker-config-engine-buster.gz-load target/python-wheels/sonic_utilities-1.2-py3-none-any.whl-in stall target/debs/buster/python3-swsscommon_1.0.0_amd64.deb-install [ FLAGS FILE ] : [] [ FLAGS DEPENDS ] : [] [ FLAGS DIFF ] : [] ============================= test session starts ============================== platform linux -- Python 3.7.3, pytest-3.10.1, py-1.7.0, pluggy-0.8.0 -- /usr/bin/python3 cachedir: .pytest_cache rootdir: /sonic/dockers/docker-dhcp-relay/cli-plugin-tests, inifile: plugins: cov-2.6.0 collecting ... collected 10 items test_config_dhcp_relay.py::TestConfigVlanDhcpRelay::test_plugin_registration PASSED [ 10%] test_config_dhcp_relay.py::TestConfigVlanDhcpRelay::test_config_vlan_add_dhcp_relay_with_nonexist_vlanid PASSED [ 20%] test_config_dhcp_relay.py::TestConfigVlanDhcpRelay::test_config_vlan_add_dhcp_relay_with_invalid_vlanid PASSED [ 30%] test_config_dhcp_relay.py::TestConfigVlanDhcpRelay::test_config_vlan_add_dhcp_relay_with_invalid_ip PASSED [ 40%] test_config_dhcp_relay.py::TestConfigVlanDhcpRelay::test_config_vlan_add_dhcp_relay_with_exist_ip PASSED [ 50%] test_config_dhcp_relay.py::TestConfigVlanDhcpRelay::test_config_vlan_add_del_dhcp_relay_dest PASSED [ 60%] test_config_dhcp_relay.py::TestConfigVlanDhcpRelay::test_config_vlan_remove_nonexist_dhcp_relay_dest PASSED [ 70%] test_config_dhcp_relay.py::TestConfigVlanDhcpRelay::test_config_vlan_remove_dhcp_relay_dest_with_nonexist_vlanid PASSED [ 80%] test_show_dhcp_relay.py::TestVlanDhcpRelay::test_plugin_registration PASSED [ 90%] test_show_dhcp_relay.py::TestVlanDhcpRelay::test_dhcp_relay_column_output PASSED [100%] =============================== warnings summary =============================== /usr/local/lib/python3.7/dist-packages/tabulate.py:7 /usr/local/lib/python3.7/dist-packages/tabulate.py:7: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working from collections import namedtuple, Iterable -- Docs: https://docs.pytest.org/en/latest/warnings.html ==================== 10 passed, 1 warnings in 0.35 seconds =====================

Signed-off-by: Stepan Blyschak [email protected] This PR is part of SONiC Application Extension Depends on sonic-net#5938 - Why I did it To provide an infrastructure change in order to support SONiC Application Extension feature. - How I did it Label every installable SONiC Docker with a minimal required manifest and auto-generate packages.json file based on installed SONiC images. - How to verify it Build an image, execute the following command: admin@sonic:~$ docker inspect docker-snmp:1.0.0 | jq '.[0].Config.Labels["com.azure.sonic.manifest"]' -r | jq Cat /var/lib/sonic-package-manager/packages.json file to verify all dockers are listed there.

- Why I did it Make DHCP relay docker an extension. DHCP relay now carries dhcp relay commands CLI plugin and has a complete manifest. It is installed as extension if INCLUDE_DHCP_REALY is set to y. DEPENDS on sonic-net#5939 - How I did it Modify DHCP relay docker makefile and dockerfile. Make changes to sonic_debian_extension.j2 to install sonic packages. I moved DHCP related CLI tests from sonic-utilities to DHCP relay docker. This PR introduces a way to write a plugin as part of docker image and run the tests from cli-plugin-tests directory under docker directory. The test result is available in target/docker-dhcp-relay.gz.log: [ REASON ] : target/docker-dhcp-relay.gz does not exist NON-EXISTENT PREREQUISITES: docker-start target/docker-config-engine-buster.gz-load target/python-wheels/sonic_utilities-1.2-py3-none-any.whl-in stall target/debs/buster/python3-swsscommon_1.0.0_amd64.deb-install [ FLAGS FILE ] : [] [ FLAGS DEPENDS ] : [] [ FLAGS DIFF ] : [] ============================= test session starts ============================== platform linux -- Python 3.7.3, pytest-3.10.1, py-1.7.0, pluggy-0.8.0 -- /usr/bin/python3 cachedir: .pytest_cache rootdir: /sonic/dockers/docker-dhcp-relay/cli-plugin-tests, inifile: plugins: cov-2.6.0 collecting ... collected 10 items test_config_dhcp_relay.py::TestConfigVlanDhcpRelay::test_plugin_registration PASSED [ 10%] test_config_dhcp_relay.py::TestConfigVlanDhcpRelay::test_config_vlan_add_dhcp_relay_with_nonexist_vlanid PASSED [ 20%] test_config_dhcp_relay.py::TestConfigVlanDhcpRelay::test_config_vlan_add_dhcp_relay_with_invalid_vlanid PASSED [ 30%] test_config_dhcp_relay.py::TestConfigVlanDhcpRelay::test_config_vlan_add_dhcp_relay_with_invalid_ip PASSED [ 40%] test_config_dhcp_relay.py::TestConfigVlanDhcpRelay::test_config_vlan_add_dhcp_relay_with_exist_ip PASSED [ 50%] test_config_dhcp_relay.py::TestConfigVlanDhcpRelay::test_config_vlan_add_del_dhcp_relay_dest PASSED [ 60%] test_config_dhcp_relay.py::TestConfigVlanDhcpRelay::test_config_vlan_remove_nonexist_dhcp_relay_dest PASSED [ 70%] test_config_dhcp_relay.py::TestConfigVlanDhcpRelay::test_config_vlan_remove_dhcp_relay_dest_with_nonexist_vlanid PASSED [ 80%] test_show_dhcp_relay.py::TestVlanDhcpRelay::test_plugin_registration PASSED [ 90%] test_show_dhcp_relay.py::TestVlanDhcpRelay::test_dhcp_relay_column_output PASSED [100%] =============================== warnings summary =============================== /usr/local/lib/python3.7/dist-packages/tabulate.py:7 /usr/local/lib/python3.7/dist-packages/tabulate.py:7: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working from collections import namedtuple, Iterable -- Docs: https://docs.pytest.org/en/latest/warnings.html ==================== 10 passed, 1 warnings in 0.35 seconds =====================

stepanblyschak added 3 commits November 30, 2020 14:27

[dockers] label SONiC Docker with manifest

954cad9

Signed-off-by: Stepan Blyshchak <[email protected]>

[dockers] add package name to manifest

855fa60

Signed-off-by: Stepan Blyshchak <[email protected]>

stepanblyschak force-pushed the dockers_manifest branch from ca81090 to b4f307f Compare November 30, 2020 12:32

stepanblyschak requested review from renukamanavalan and lguohan December 1, 2020 17:16

remove swss dependency from dhcp-relay & router-advertiser

44685ba

Signed-off-by: Stepan Blyshchak <[email protected]>

stepanblyschak force-pushed the dockers_manifest branch from b4f307f to 44685ba Compare December 10, 2020 15:29

lguohan force-pushed the master branch from 3690c1a to 512eb6b Compare December 25, 2020 18:34

stepanblyschak added 2 commits January 5, 2021 12:13

Merge branch 'master' of github.com:azure/sonic-buildimage into docke…

d337e08

…rs_manifest Signed-off-by: Stepan Blyshchak <[email protected]>

Merge branch 'master' of github.com:azure/sonic-buildimage into docke…

5eb48a9

…rs_version_tags Signed-off-by: Stepan Blyshchak <[email protected]>

stepanblyschak mentioned this pull request Jan 8, 2021

SONiC Application Extension PR Tracker #6398

Open

stepanblyschak and others added 8 commits January 13, 2021 15:22

Merge branch 'master' of github.com:azure/sonic-buildimage into docke…

99d9d12

…rs_manifest Signed-off-by: Stepan Blyshchak <[email protected]>

remove sonic-sdk added by mistake in this change

ed1bd3f

Signed-off-by: Stepan Blyshchak <[email protected]>

Merge branch 'master' into dockers_version_tags

50237b4

Merge branch 'master' of github.com:azure/sonic-buildimage into docke…

caf5c9e

…rs_manifest

Merge branch 'master' of github.com:azure/sonic-buildimage into docke…

31d4bbf

…rs_version_tags

[docker-macsec] add version number for docker-macsec

823eede

Signed-off-by: Stepan Blyshchak <[email protected]>

Merge branch 'dockers_version_tags' of github.com:stepanblyschak/soni…

bdf0113

…c-buildimage into dockers_manifest Signed-off-by: Stepan Blyshchak <[email protected]>

[docker-macsec] add manifest for macsec docker

05df4ed

Signed-off-by: Stepan Blyshchak <[email protected]>

stepanblyschak mentioned this pull request Jan 22, 2021

[dhcp-relay] make DHCP relay an extension #6531

Merged

4 tasks

rajendra-dendukuri reviewed Jan 25, 2021

View reviewed changes

[sonic_debian_extension.j2] fix merge conflict markers

e0735d2

Signed-off-by: Stepan Blyshchak <[email protected]>

stepanblyschak added the appext Application Extension label Jan 27, 2021

[dockers] use single manifest template

2066aa9

Signed-off-by: Stepan Blyshchak <[email protected]>

stepanblyschak closed this Mar 25, 2021

stepanblyschak reopened this Mar 25, 2021

stepanblyschak added 2 commits March 25, 2021 17:02

remove obsolete file

3de0cb3

Signed-off-by: Stepan Blyschak <[email protected]>

Merge branch 'dockers_manifest' of github.com:stepanblyschak/sonic-bu…

a665d73

…ildimage into dockers_manifest Signed-off-by: Stepan Blyschak <[email protected]>

stepanblyschak commented Mar 26, 2021

View reviewed changes

files/build_templates/manifest.json.j2 Outdated Show resolved Hide resolved

remove obsolete field from manifest. will be replaced by base-os comp…

b2f8fc9

…onents approacgh Signed-off-by: Stepan Blyschak <[email protected]>

stepanblyschak mentioned this pull request Mar 29, 2021

[dockers] Tag all docker images with a version number #5938

Closed

3 tasks

Merge branch 'master' of github.com:azure/sonic-buildimage into docke…

46d75eb

…rs_manifest

renukamanavalan reviewed Apr 8, 2021

View reviewed changes

stepanblyschak added 3 commits April 9, 2021 15:19

fix review comments

84bf4dd

Signed-off-by: Stepan Blyschak <[email protected]>

fix review comments

b427b71

Signed-off-by: Stepan Blyschak <[email protected]>

add comment

92ad8b1

Signed-off-by: Stepan Blyschak <[email protected]>

stepanblyschak force-pushed the dockers_manifest branch 2 times, most recently from b8d0128 to 57ec2f2 Compare April 15, 2021 14:58

stepanblyschak and others added 2 commits April 15, 2021 17:58

remove not needed ARG in dockerfiles

57ec2f2

Signed-off-by: Stepan Blyschak <[email protected]>

Merge branch 'master' into dockers_manifest

c74ea21

stepanblyschak and others added 3 commits April 23, 2021 12:39

Merge branch 'master' into dockers_manifest

90a74dd

fix indentation

aa31898

Signed-off-by: Stepan Blyschak <[email protected]>

add todo about changelog

9e171a5

Signed-off-by: Stepan Blyschak <[email protected]>

renukamanavalan approved these changes Apr 26, 2021

View reviewed changes

renukamanavalan merged commit cd2c86e into sonic-net:master Apr 26, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[dockers] label SONiC Docker with manifest #5939

[dockers] label SONiC Docker with manifest #5939

stepanblyschak commented Nov 16, 2020

stepanblyschak commented Nov 17, 2020

stepanblyschak commented Nov 17, 2020

stepanblyschak commented Nov 19, 2020

stepanblyschak commented Dec 7, 2020

rajendra-dendukuri left a comment

liat-grozovik commented Apr 21, 2021

azure-pipelines bot commented Apr 21, 2021

[dockers] label SONiC Docker with manifest #5939

[dockers] label SONiC Docker with manifest #5939

Conversation

stepanblyschak commented Nov 16, 2020

stepanblyschak commented Nov 17, 2020

stepanblyschak commented Nov 17, 2020

stepanblyschak commented Nov 19, 2020

stepanblyschak commented Dec 7, 2020

rajendra-dendukuri left a comment

Choose a reason for hiding this comment

liat-grozovik commented Apr 21, 2021

azure-pipelines bot commented Apr 21, 2021