-
Notifications
You must be signed in to change notification settings - Fork 61
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Optimize cv_facts execution. #147
Labels
module: cv_facts
Issue related to cv_facts module
state: accepted
Issue is part of the development roadmap
status: testing
PR under testing
type: enhancement
New feature or request
Milestone
Comments
titom73
added
type: enhancement
New feature or request
module: cv_facts
Issue related to cv_facts module
status: Triage
New incoming issue
labels
Mar 5, 2020
titom73
added
state: accepted
Issue is part of the development roadmap
and removed
status: Triage
New incoming issue
labels
Mar 10, 2020
titom73
added a commit
that referenced
this issue
Mar 18, 2020
Rewrite def facts_configlets to use get_configlets_and_mappers function and reduce number of API call. No change in inputs and outputs. Test cases: ----------- - Manual cv_facts tasks - AVD integration Execution time enhacement: -------------------------- - Original implementation: 6 sec - New version: 0.3 sec test setup: ------------ 8 devices / 16 configlets / 10 containers Execution logs: --------------- 2020-03-18 18:26:09,700 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:269) - Collecting facts v2 2020-03-18 18:26:10,017 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:276) - Devices part of facts, using this version 2020-03-18 18:26:10,017 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:283) - Containers part of facts, using this version 2020-03-18 18:26:10,017 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet leaf02-baseline 2020-03-18 18:26:10,017 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet RADIUS_LAB 2020-03-18 18:26:10,018 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet DC1-AVD_DC1-LEAF2B 2020-03-18 18:26:10,018 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet Builder_LEAFs 2020-03-18 18:26:10,018 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet spine01-baseline 2020-03-18 18:26:10,018 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet DC1-AVD_DC1-SPINE2 2020-03-18 18:26:10,018 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet DC1-AVD_DC1-LEAF1B 2020-03-18 18:26:10,018 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet DC1-AVD_DC1-SPINE1 2020-03-18 18:26:10,018 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet GLOBAL-ALIASES 2020-03-18 18:26:10,019 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet DC1-AVD_DC1-L2LEAF1A 2020-03-18 18:26:10,019 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet leaf01-baseline 2020-03-18 18:26:10,019 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet veos-01 2020-03-18 18:26:10,019 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet DC1-AVD_DC1-L2LEAF2A 2020-03-18 18:26:10,019 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet DC1-AVD_DC1-LEAF2A 2020-03-18 18:26:10,019 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet SYS_TelemetryBuilderV3 2020-03-18 18:26:10,019 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet DC1-AVD_DC1-LEAF1A 2020-03-18 18:26:10,019 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet data.device.leaf01 2020-03-18 18:26:10,019 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet template.interfaces 2020-03-18 18:26:10,020 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:317) - All configlets facts collected
titom73
added
the
status: in-progress
Currently under investigation or implementation
label
Mar 18, 2020
titom73
added a commit
that referenced
this issue
Mar 18, 2020
Rewrite def facts_configlets to use get_configlets_and_mappers function and reduce number of API call. No change in inputs and outputs. Test cases: ----------- - Manual cv_facts tasks - AVD integration Execution time enhacement: -------------------------- - Original implementation: 6 sec - New version: 0.3 sec test setup: ------------ 8 devices / 16 configlets / 10 containers Execution logs: --------------- 2020-03-18 18:26:09,700 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:269) - Collecting facts v2 2020-03-18 18:26:10,017 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:276) - Devices part of facts, using this version 2020-03-18 18:26:10,017 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:283) - Containers part of facts, using this version 2020-03-18 18:26:10,017 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet leaf02-baseline 2020-03-18 18:26:10,017 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet RADIUS_LAB 2020-03-18 18:26:10,018 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet DC1-AVD_DC1-LEAF2B 2020-03-18 18:26:10,018 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet Builder_LEAFs 2020-03-18 18:26:10,018 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet spine01-baseline 2020-03-18 18:26:10,018 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet DC1-AVD_DC1-SPINE2 2020-03-18 18:26:10,018 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet DC1-AVD_DC1-LEAF1B 2020-03-18 18:26:10,018 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet DC1-AVD_DC1-SPINE1 2020-03-18 18:26:10,018 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet GLOBAL-ALIASES 2020-03-18 18:26:10,019 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet DC1-AVD_DC1-L2LEAF1A 2020-03-18 18:26:10,019 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet leaf01-baseline 2020-03-18 18:26:10,019 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet veos-01 2020-03-18 18:26:10,019 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet DC1-AVD_DC1-L2LEAF2A 2020-03-18 18:26:10,019 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet DC1-AVD_DC1-LEAF2A 2020-03-18 18:26:10,019 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet SYS_TelemetryBuilderV3 2020-03-18 18:26:10,019 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet DC1-AVD_DC1-LEAF1A 2020-03-18 18:26:10,019 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet data.device.leaf01 2020-03-18 18:26:10,019 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet template.interfaces 2020-03-18 18:26:10,020 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:317) - All configlets facts collected
titom73
added a commit
that referenced
this issue
Mar 18, 2020
Rewrite def facts_configlets to use get_configlets_and_mappers function and reduce number of API call. No change in inputs and outputs. Test cases: ----------- - Manual cv_facts tasks - AVD integration Execution time enhacement: -------------------------- - Original implementation: 6 sec - New version: 0.3 sec test setup: ------------ 8 devices / 16 configlets / 10 containers Execution logs: --------------- 2020-03-18 18:26:09,700 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:269) - Collecting facts v2 2020-03-18 18:26:10,017 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:276) - Devices part of facts, using this version 2020-03-18 18:26:10,017 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:283) - Containers part of facts, using this version 2020-03-18 18:26:10,017 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet leaf02-baseline 2020-03-18 18:26:10,017 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet RADIUS_LAB 2020-03-18 18:26:10,018 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet DC1-AVD_DC1-LEAF2B 2020-03-18 18:26:10,018 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet Builder_LEAFs 2020-03-18 18:26:10,018 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet spine01-baseline 2020-03-18 18:26:10,018 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet DC1-AVD_DC1-SPINE2 2020-03-18 18:26:10,018 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet DC1-AVD_DC1-LEAF1B 2020-03-18 18:26:10,018 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet DC1-AVD_DC1-SPINE1 2020-03-18 18:26:10,018 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet GLOBAL-ALIASES 2020-03-18 18:26:10,019 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet DC1-AVD_DC1-L2LEAF1A 2020-03-18 18:26:10,019 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet leaf01-baseline 2020-03-18 18:26:10,019 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet veos-01 2020-03-18 18:26:10,019 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet DC1-AVD_DC1-L2LEAF2A 2020-03-18 18:26:10,019 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet DC1-AVD_DC1-LEAF2A 2020-03-18 18:26:10,019 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet SYS_TelemetryBuilderV3 2020-03-18 18:26:10,019 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet DC1-AVD_DC1-LEAF1A 2020-03-18 18:26:10,019 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet data.device.leaf01 2020-03-18 18:26:10,019 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:296) - building list of mapping with devices and containers for configlet template.interfaces 2020-03-18 18:26:10,020 - arista.cvp.cv_facts: INFO - func: facts_configlets (L:317) - All configlets facts collected
titom73
added
status: To be merged
Issue solved in a volatile branch
and removed
status: in-progress
Currently under investigation or implementation
labels
Mar 30, 2020
titom73
added
status: testing
PR under testing
and removed
status: To be merged
Issue solved in a volatile branch
labels
Apr 7, 2020
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
module: cv_facts
Issue related to cv_facts module
state: accepted
Issue is part of the development roadmap
status: testing
PR under testing
type: enhancement
New feature or request
Issue Type
Module Name
cv_facts
Summary
In a large environment, collecting facts from Cloudvision can take a long time increasing execution time of the overall playbook.
It has been seen in deployment with few hundreds of devices and thousands of configlets.
To reduce execution time, it might be useful to reduce number of API calls sent to Cloudvision.
2 options could be a good source of optimization:
One option suggesting by @ankudinov is to use
get /configlet/getConfigletsAndAssociatedMappers.do
call to collect configlet and association in just one API call instead of current implementation described below:Second approach is to use
get_configlets_by_device_id()
in currentcv_api201x
wrappingprovisioning/getConfigletsByNetElementId.do?netElementId
API call.The text was updated successfully, but these errors were encountered: