Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[db_migrator] Add migration of FLEX_COUNTER_DELAY_STATUS during 1911->master upgrade + fast-reboot. Add UT. #2839

Merged
merged 1 commit into from
Jun 15, 2023
Merged

[db_migrator] Add migration of FLEX_COUNTER_DELAY_STATUS during 1911->master upgrade + fast-reboot. Add UT. #2839

merged 1 commit into from
Jun 15, 2023

Conversation

vadymhlushko-mlnx
Copy link
Contributor

@vadymhlushko-mlnx vadymhlushko-mlnx commented May 17, 2023 •
edited
Loading

What I did

Add migration of FLEX_COUNTER_DELAY_STATUS attribute of config_db FLEX_COUNTER_TABLE during the SONiC to SONiC upgrade + fast-reboot from older versions 201911 -> master.

This change is required for the fast-reboot procedure because without it the counters will be created during the init flow which will waste a lot of resources and cause data plane degradation of more than 30 seconds.

How I did it

Modify the db_migrator.py.

How to verify it

Add UT.

Previous command output (if the output of a command-line utility has changed)

New command output (if the output of a command-line utility has changed)

@keboliu keboliu requested a review from qiluo-msft May 29, 2023 03:10
@keboliu keboliu requested a review from vaibhavhd May 31, 2023 08:21
@keboliu
Copy link
Collaborator

keboliu commented May 31, 2023

@vaibhavhd would you please help to review?

vaibhavhd
vaibhavhd reviewed May 31, 2023
View reviewed changes
scripts/db_migrator.py Outdated
@@ -811,6 +825,10 @@ def version_2_0_2(self):
This is the latest version for 202012 branch
"""
log.log_info('Handling version_2_0_2')

if self.stateDB.keys(self.stateDB.STATE_DB, "FAST_REBOOT|system"):
self.migrate_config_db_flex_counter_delay_status()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why has this logic changed 202012 specific version? AFAIK, the change is needed for 202211 and above, so only branch specific versions should be changed. This will likely also require separate PRs for feature branches.

Also this is missing call to set a new version and also a placeholder function for new version.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right, we need these changes for every branch starting from 202111, 202205, 202211, master
I will create a separate PR's

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here are the PR's:

[202211][db_migrator] Add migration of FLEX_COUNTER_DELAY_STATUS during 1911->2211 upgrade + fast-reboot. Add UT.
[202205][db_migrator] Add migration of FLEX_COUNTER_DELAY_STATUS during 1911->2205 upgrade + fast-reboot. Add UT.
[202111][db_migrator] Add migration of FLEX_COUNTER_DELAY_STATUS during 1911->2111 upgrade + fast-reboot. Add UT.

vaibhavhd
vaibhavhd reviewed May 31, 2023
View reviewed changes
Copy link
Contributor

@vaibhavhd vaibhavhd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a blocking comment to change vesions only for feature branches where this migration is relevant.

@vadymhlushko-mlnx vadymhlushko-mlnx force-pushed the master-flex-counter-delay-status-migration branch from 37c93bb to 90863c1 Compare June 9, 2023 12:48
@vadymhlushko-mlnx
[db_migtrator] Add migration of FLEX_COUNTER_DELAY_STATUS during 1911…
04c14bd 
…->master upgrade + fast-reboot. Add UT.

Signed-off-by: vadymhlushko-mlnx <[email protected]>
@vadymhlushko-mlnx vadymhlushko-mlnx force-pushed the master-flex-counter-delay-status-migration branch from 90863c1 to 04c14bd Compare June 13, 2023 13:47
@liat-grozovik
Copy link
Collaborator

@vaibhavhd could you please approve so we can merge?

@vaibhavhd vaibhavhd merged commit 3ba8241 into sonic-net:master Jun 15, 2023
@vadymhlushko-mlnx vadymhlushko-mlnx mentioned this pull request Jun 19, 2023
Merged
dprital added a commit to dprital/sonic-buildimage that referenced this pull request Jun 20, 2023
@dprital
[submodule] Advance sonic-utilities pointer
a8cca00 
Update sonic-utilities submodule pointer to include the following:
* 0b629ba1 Revert [chassis][voq] Clear fabric counters queue/port (2789) ([sonic-net#2882](sonic-net/sonic-utilities#2882))
* 3ba8241a [db_migtrator] Add migration of FLEX_COUNTER_DELAY_STATUS during 1911->master upgrade + fast-reboot. Add UT. ([sonic-net#2839](sonic-net/sonic-utilities#2839))
* fceef2ed [chassis][voq] Clear fabric counters queue/port ([sonic-net#2789](sonic-net/sonic-utilities#2789))
* 659ba24b [syslog] Adjust runningconfiguration syslog command ([sonic-net#2843](sonic-net/sonic-utilities#2843))
* 46fba26f [db_migrator] add required protocol field in ROUTE_TABLE ([sonic-net#2766](sonic-net/sonic-utilities#2766))
* f186376e Fix issue: show interfaces transceiver eeprom -d should display same entry for CMIS cable ([sonic-net#2864](sonic-net/sonic-utilities#2864))
* de491798 fix precedence in portstat CLI ([sonic-net#2874](sonic-net/sonic-utilities#2874))

Signed-off-by: dprital <[email protected]>
dprital added a commit to dprital/sonic-buildimage that referenced this pull request Jun 21, 2023
@dprital
[submodule] Advance sonic-utilities pointer
784c275 
Update sonic-utilities submodule pointer to include the following:
* 0b629ba1 Revert [chassis][voq] Clear fabric counters queue/port (2789) ([sonic-net#2882](sonic-net/sonic-utilities#2882))
* 3ba8241a [db_migtrator] Add migration of FLEX_COUNTER_DELAY_STATUS during 1911->master upgrade + fast-reboot. Add UT. ([sonic-net#2839](sonic-net/sonic-utilities#2839))
* fceef2ed [chassis][voq] Clear fabric counters queue/port ([sonic-net#2789](sonic-net/sonic-utilities#2789))
* 659ba24b [syslog] Adjust runningconfiguration syslog command ([sonic-net#2843](sonic-net/sonic-utilities#2843))
* 46fba26f [db_migrator] add required protocol field in ROUTE_TABLE ([sonic-net#2766](sonic-net/sonic-utilities#2766))
* f186376e Fix issue: show interfaces transceiver eeprom -d should display same entry for CMIS cable ([sonic-net#2864](sonic-net/sonic-utilities#2864))
* de491798 fix precedence in portstat CLI ([sonic-net#2874](sonic-net/sonic-utilities#2874))

Signed-off-by: dprital <[email protected]>
dgsudharsan added a commit to dgsudharsan/sonic-buildimage that referenced this pull request Jul 11, 2023
@dgsudharsan
[submodule] Advance sonic-utilities pointer
03d3573 
Update sonic-utilities submodule pointer to include the following:
* ff380e04 [hash]: Implement GH frontend ([sonic-net#2580](sonic-net/sonic-utilities#2580))
* 61bad064 [db_migrator] Set correct CURRENT_VERSION, extend UT ([sonic-net#2895](sonic-net/sonic-utilities#2895))
* 6b8ee47c [CLI][Show][BGP] Show BGP Change for no neighbor scenario ([sonic-net#2885](sonic-net/sonic-utilities#2885))
* 73d8d633 [doc] Update Command-Reference.md, change show bgp peer command to show bfd peer ([sonic-net#2750](sonic-net/sonic-utilities#2750))
* 7bc08c28 [db_migrator] Remove hardcoded config and migrate config from minigraph ([sonic-net#2887](sonic-net/sonic-utilities#2887))
* b1aa9426 [generate_dump]: Enhance show techsupport for Marvell platform ([sonic-net#2676](sonic-net/sonic-utilities#2676))
* 316b14c0 Add support for secure upgrade ([sonic-net#2698](sonic-net/sonic-utilities#2698))
* dc2945bc [dns] Implement config and show commands for static DNS. ([sonic-net#2737](sonic-net/sonic-utilities#2737))
* 8414a709 [chassis][multi asic] change acl_loader to use tcp socket for db communication ([sonic-net#2525](sonic-net/sonic-utilities#2525))
* 0b629ba1 Revert [chassis][voq] Clear fabric counters queue/port (2789) ([sonic-net#2882](sonic-net/sonic-utilities#2882))
* 3ba8241a [db_migtrator] Add migration of FLEX_COUNTER_DELAY_STATUS during 1911->master upgrade + fast-reboot. Add UT. ([sonic-net#2839](sonic-net/sonic-utilities#2839))
* fceef2ed [chassis][voq] Clear fabric counters queue/port ([sonic-net#2789](sonic-net/sonic-utilities#2789))

Signed-off-by: dgsudharsan <[email protected]>
@dgsudharsan dgsudharsan mentioned this pull request Jul 11, 2023
Merged
11 tasks
liat-grozovik pushed a commit to sonic-net/sonic-buildimage that referenced this pull request Jul 11, 2023
@dgsudharsan
[submodule] Advance sonic-utilities pointer (#15779)
5cbae46 
Update sonic-utilities submodule pointer to include the following:
* ff380e04 [hash]: Implement GH frontend ([#2580](sonic-net/sonic-utilities#2580))
* 61bad064 [db_migrator] Set correct CURRENT_VERSION, extend UT ([#2895](sonic-net/sonic-utilities#2895))
* 6b8ee47c [CLI][Show][BGP] Show BGP Change for no neighbor scenario ([#2885](sonic-net/sonic-utilities#2885))
* 73d8d633 [doc] Update Command-Reference.md, change show bgp peer command to show bfd peer ([#2750](sonic-net/sonic-utilities#2750))
* 7bc08c28 [db_migrator] Remove hardcoded config and migrate config from minigraph ([#2887](sonic-net/sonic-utilities#2887))
* b1aa9426 [generate_dump]: Enhance show techsupport for Marvell platform ([#2676](sonic-net/sonic-utilities#2676))
* 316b14c0 Add support for secure upgrade ([#2698](sonic-net/sonic-utilities#2698))
* dc2945bc [dns] Implement config and show commands for static DNS. ([#2737](sonic-net/sonic-utilities#2737))
* 8414a709 [chassis][multi asic] change acl_loader to use tcp socket for db communication ([#2525](sonic-net/sonic-utilities#2525))
* 0b629ba1 Revert [chassis][voq] Clear fabric counters queue/port (2789) ([#2882](sonic-net/sonic-utilities#2882))
* 3ba8241a [db_migtrator] Add migration of FLEX_COUNTER_DELAY_STATUS during 1911->master upgrade + fast-reboot. Add UT. ([#2839](sonic-net/sonic-utilities#2839))
* fceef2ed [chassis][voq] Clear fabric counters queue/port ([#2789](sonic-net/sonic-utilities#2789))

Signed-off-by: dgsudharsan <[email protected]>
@vaibhavhd vaibhavhd mentioned this pull request Aug 21, 2023
Merged
pdhruv-marvell pushed a commit to pdhruv-marvell/sonic-utilities that referenced this pull request Aug 23, 2023
@vadymhlushko-mlnx @pdhruv-marvell
[db_migtrator] Add migration of FLEX_COUNTER_DELAY_STATUS during 1911…
b871890 
…->master upgrade + fast-reboot. Add UT. (sonic-net#2839)

Add migration of FLEX_COUNTER_DELAY_STATUS attribute of config_db FLEX_COUNTER_TABLE during the SONiC to SONiC upgrade + fast-reboot from older versions 201911 -> master.

This change is required for the fast-reboot procedure because without it the counters will be created during the init flow which will waste a lot of resources and cause data plane degradation of more than 30 seconds.

How I did it
Modify the db_migrator.py.

How to verify it
Add UT.


Signed-off-by: vadymhlushko-mlnx <[email protected]>
sonic-otn pushed a commit to sonic-otn/sonic-buildimage that referenced this pull request Sep 20, 2023
@dgsudharsan @sonic-otn
[submodule] Advance sonic-utilities pointer (sonic-net#15779)
b777108 
Update sonic-utilities submodule pointer to include the following:
* ff380e04 [hash]: Implement GH frontend ([sonic-net#2580](sonic-net/sonic-utilities#2580))
* 61bad064 [db_migrator] Set correct CURRENT_VERSION, extend UT ([sonic-net#2895](sonic-net/sonic-utilities#2895))
* 6b8ee47c [CLI][Show][BGP] Show BGP Change for no neighbor scenario ([sonic-net#2885](sonic-net/sonic-utilities#2885))
* 73d8d633 [doc] Update Command-Reference.md, change show bgp peer command to show bfd peer ([sonic-net#2750](sonic-net/sonic-utilities#2750))
* 7bc08c28 [db_migrator] Remove hardcoded config and migrate config from minigraph ([sonic-net#2887](sonic-net/sonic-utilities#2887))
* b1aa9426 [generate_dump]: Enhance show techsupport for Marvell platform ([sonic-net#2676](sonic-net/sonic-utilities#2676))
* 316b14c0 Add support for secure upgrade ([sonic-net#2698](sonic-net/sonic-utilities#2698))
* dc2945bc [dns] Implement config and show commands for static DNS. ([sonic-net#2737](sonic-net/sonic-utilities#2737))
* 8414a709 [chassis][multi asic] change acl_loader to use tcp socket for db communication ([sonic-net#2525](sonic-net/sonic-utilities#2525))
* 0b629ba1 Revert [chassis][voq] Clear fabric counters queue/port (2789) ([sonic-net#2882](sonic-net/sonic-utilities#2882))
* 3ba8241a [db_migtrator] Add migration of FLEX_COUNTER_DELAY_STATUS during 1911->master upgrade + fast-reboot. Add UT. ([sonic-net#2839](sonic-net/sonic-utilities#2839))
* fceef2ed [chassis][voq] Clear fabric counters queue/port ([sonic-net#2789](sonic-net/sonic-utilities#2789))

Signed-off-by: dgsudharsan <[email protected]>
@dgsudharsan dgsudharsan mentioned this pull request Oct 17, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@stepanblyschak stepanblyschak stepanblyschak approved these changes

@vaibhavhd vaibhavhd vaibhavhd approved these changes

@qiluo-msft qiluo-msft Awaiting requested review from qiluo-msft

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@vadymhlushko-mlnx @keboliu @liat-grozovik @vaibhavhd @stepanblyschak