[BUG]: dellemc.enterprise_sonic.sonic_facts gather_network_resources isn't working for some resources

[PatBoEPFL]

Bug Description

Hello,

I've tried the different resources that can be gathered with sonic_facts module:

- name: SONiC facts
  tags:
    - sonic.show.sonic_facts
  block:
    - name: Gather all facts
      register: sonic_facts_data
      connection: httpapi
      dellemc.enterprise_sonic.sonic_facts:
        # gather_subset: all
        gather_network_resources:
          # OK - "vlans"
          # OK - "interfaces"
          # OK - "l2_interfaces"
          # OK - "l3_interfaces"
          # OK - "lag_interfaces"
          # FAIL - "bgp"
          - "bgp"
          # FAIL - "bgp_af"
          # must be tested - "bgp_neighbors"
          # must be tested - "bgp_neighbors_af"
          # must be tested - "bgp_as_paths"
          # must be tested - "bgp_communities"
          # must be tested - "bgp_ext_communities"
          # OK - "mclag"
          # must be tested - "prefix_lists"
          # must be tested - "vlan_mapping"
          # OK - "vrfs"
          # OK- "vxlans"
          # OK - "users"
          # OK - "system"
          # OK - "port_breakout"
          # must be tested - "aaa"
          # must be tested - "tacacs_server"
          # must be tested - "radius_server"
          # must be tested - "static_routes"
          # OK - "ntp"
          # OK - "logging"
          # OK - "pki"
          # must be tested - "ip_neighbor"
          # FAIL - "port_group"
          # OK - "dhcp_relay"
          # must be tested - "acl_interfaces"
          # must be tested  "l2_acls"
          # must be tested - "l3_acls"
          # OK - "lldp_global"
          # OK- "mac"
          # OK - "bfd"
          # OK - "copp"
          # must be tested - "route_maps"
          # OK - "stp"
          # OK - "dhcp_snooping"
    - name: Show all facts
      ansible.builtin.debug:
        var: sonic_facts_data

With the above task trying 'bgp', I've got the following output:

ASK [sonic_show : Gather all facts] ***************************************************************************************************************************************************
task path: /Users/pbeaud/Code/dell-sonic-config/roles/sonic_show/tasks/main.yml:23
redirecting (type: connection) ansible.builtin.httpapi to ansible.netcommon.httpapi
redirecting (type: action) dellemc.enterprise_sonic.sonic_facts to dellemc.enterprise_sonic.sonic
redirecting (type: action) dellemc.enterprise_sonic.sonic_facts to dellemc.enterprise_sonic.sonic
fatal: [sw_t_in_s19_pod1_lf1]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "config": [
                {
                    "bestpath": {
                        "as_path": {
                            "confed": false,
                            "ignore": false,
                            "multipath_relax": false,
                            "multipath_relax_as_set": false
                        },
                        "compare_routerid": false,
                        "med": {
                            "always_compare_med": false,
                            "confed": false,
                            "missing_as_worst": false
                        }
                    },
                    "bgp_as": null,
                    "log_neighbor_changes": false,
                    "max_med": {
                        "on_startup": {
                            "med_val": null,
                            "timer": null
                        }
                    },
                    "router_id": null,
                    "rt_delay": null,
                    "timers": {
                        "holdtime": null,
                        "keepalive_interval": null
                    },
                    "vrf_name": "default"
                }
            ],
            "state": "merged"
        }
    },
    "msg": "missing required arguments: bgp_as found in config"
}

When a resource is marked as # must be tested - "<resource name>", it means there is no relevant configuration on the tested switch.

Product Name

DellEMC-S5232f-C32

Component or Module Name

httpapi, sonic_facts

DellEMC Enterprise SONiC Ansible Collection Version

dellemc.enterprise_sonic-2.4.0

SONiC Software Version

4.1.2-Enterprise_Standard

Configuration

COLLECTIONS_PATHS(env: ANSIBLE_COLLECTIONS_PATH) = ['/Users/pbeaud/Code/dell-sonic-config/ansible-deps-cache']
CONFIG_FILE() = None
DEFAULT_ROLES_PATH(env: ANSIBLE_ROLES_PATH) = ['/Users/pbeaud/Code/dell-sonic-config/ansible-deps-cache/roles']

Steps to Reproduce

For resources bgp and bgp_af see description.

For port_group, do the same but with only port_groupas resource:

TASK [sonic_show : Gather all facts] ***************************************************************************************************************************************************
task path: /Users/pbeaud/Code/dell-sonic-config/roles/sonic_show/tasks/main.yml:23
redirecting (type: connection) ansible.builtin.httpapi to ansible.netcommon.httpapi
redirecting (type: action) dellemc.enterprise_sonic.sonic_facts to dellemc.enterprise_sonic.sonic
redirecting (type: action) dellemc.enterprise_sonic.sonic_facts to dellemc.enterprise_sonic.sonic
The full traceback is:
  File "/Users/pbeaud/Code/dell-sonic-config/ansible-deps-cache/ansible_collections/dellemc/enterprise_sonic/plugins/module_utils/network/sonic/facts/port_group/port_group.py", line 98, in get_port_groups
    pgs_response = edit_config(self._module, to_request(self._module, pgs_request))
  File "/Users/pbeaud/Code/dell-sonic-config/ansible-deps-cache/ansible_collections/dellemc/enterprise_sonic/plugins/module_utils/network/sonic/sonic.py", line 139, in edit_config
    return connection.edit_config(commands)
  File "/Users/pbeaud/Code/dell-sonic-config/ansible-deps-cache/python-libs/lib/python/site-packages/ansible/module_utils/connection.py", line 200, in __rpc__
    raise ConnectionError(to_text(msg, errors='surrogate_then_replace'), code=code)
fatal: [sw_t_in_s19_pod1_lf1]: FAILED! => {
    "changed": false,
    "code": -32603,
    "invocation": {
        "module_args": {
            "gather_network_resources": [
                "port_group"
            ],
            "gather_subset": [
                "!config"
            ]
        }
    },
    "msg": "{'ietf-restconf:errors': {'error': [{'error-type': 'application', 'error-tag': 'invalid-value', 'error-message': 'Resource not found'}]}, 'code': 404, 'request_data': {'path': 'data/openconfig-port-group:port-groups/port-group', 'method': 'get'}}"
}

Expected Behavior

We expect to have the facts gathered without error.

Actual Behavior

Impossible to run without error

Logs

see output above

Screenshots

No response

Additional Information

No response

[kerry-meyer]

Thank you for bringing these issues to our attention.

For the BGP failure shown (and, I'm guessing, for the bgp_af failure as well), the problem is the result of omission of a required argument 'bgp_as' from the playbook. The bgp_as is the 'key' specifying which BGP instance is to be operated on for the playbook and it must be specified for successful execution of any playbook related to these resource modules.

For the 'port_groups' failure, please let us know which platform type was the target for the failing playbook. If you can provide the first ten lines of "show version" output for the target device, this will provide the platform information I am requesting.

For example: (needed line shown by "<<<" below)

admin@sonic:~$ show version

SONiC Software Version: SONiC-OS-rel_dell_sonic_4.x_share.1833-c9f874533
Product: Generic
Distribution: Debian 11.9
Kernel: 5.10.0-21-amd64
Config DB Version: version_4_3_1
Build commit: c9f874533
Build date: Thu Jun  6 09:17:49 UTC 2024
Built by: dngnetbuild.svc@jenkinsworker-eqx-03

Platform: x86_64-dellemc_s5296f_c3538-r0                 <<<

Some platforms do not support port groups. If that was the cause for this failure (to be determined from your "show version" output), we will modify our resource module collection to provide a more specific description of the reason for rejecting a request to gather or modify port group configuration on a device that doesn't support this feature.

[PatBoEPFL]

Hello Kerry,

Many thanks for your answer!

Sorry, but it remains unclear for me where/how should I enter the bgp_as parameter, when reading https://docs.ansible.com/ansible/latest/collections/dellemc/enterprise_sonic/sonic_facts_module.html

The platform can be:

• x86_64-dellemc_s5232f_c3538-r0
• x86_64-dellemc_n3248te_c3338-r0

Best regards,
Patrice

[kerry-meyer]

Hi Patrice,

The bgp_as value, when specified in a playbook, is an autonomous system number, either in "dotted" format in the range 0.1 to 65535.65535, or an integer value in the range 1 - 4294967295. It isn't needed for facts gathering, but the failing playbook, despite the comments printed in the output displayed above does not appear to be doing just facts gathering.

The playbook output shown in your description shows a playbook executing a 'merged' state task (a requested configuration change) on the BGP resource module. (This does require the bgp_as. To simply gather all facts for all BGP instances, you don't need to specify any AS. (All of them will be shown in the output.)

But the output shown above in this "Issue" doesn't look right for if it was just for facts gathering.

If you will provide the text of the playbook you executed to obtain the failure above, I can analyze it and explain more details on what is going wrong with execution of that playbook.

Here is an example playbook to gather BGP facts (verified on our server using the 2.4.0 version of our resource module collection):

---
- name: Ansible resource module example
  hosts: datacenter
  gather_facts: True
  connection: httpapi
  collections:
    - dellemc.enterprise_sonic
  tasks:
    - name: sonic_bgp_af facts gather
      sonic_facts:
        gather_subset: min
        gather_network_resources:
          - bgp
      register: result

    - name: "debug facts"
      debug:
        msg: "{{result.ansible_facts.ansible_network_resources}}"

---

Regarding the port group failure described above: Thank you for specifying the platforms on which you tried this.

Port groups are not supported on these platforms.

The platforms on which they are supported are: s5248 and s5296.

We will add better checking for this incompatibility, along with a more clear statement of the detected incompatibility in an upcoming release of our resource module collection.

[kerry-meyer]

I will close this bug unless there are remaining items that require further investigation or resolution.

Here is a summary of the points raised in this issue:

• Error output for port groups:
  This is expected for the platform on which the task was executed because port groups are not supported and there is no 'port_group' configuration (not even empty configuration). This error could be avoided on this platform by specifying a yaml list of the resources for which facts gathering is requested, excluding "port_group". e.g.

    - name: Gather all facts
      register: sonic_facts_data
      connection: httpapi
      dellemc.enterprise_sonic.sonic_facts:
        gather_network_resources:
          - tacacs_server
          - qos_scheduler
          - prefix_lists
                *
                *
                *

• Error output for an invalid 'merged' state configuration (not facts gathering) playbook specifying empty BGP configuration:
  The output is in a form that would occur for a configuration playbook specifying 'merged' state and all attributes are empty. Something is incorrect in the task/playbook structuring. We can not provide advice on how to fix it unless the task and playbook details are provided.

                    "router_id": null,
                    "rt_delay": null,
                    "timers": {
                        "holdtime": null,                    <<< All attributes are empty. There is no configuration here.
                        "keepalive_interval": null
                    },
                    "vrf_name": "default"
                }
            ],
            "state": "merged"          <<< This option for a configuration playbook is never included for facts gathering.
        }
    },
    "msg": "missing required arguments: bgp_as found in config"  <<<  Never required for facts gathering.
}

[PatBoEPFL]

Hello Kerry,
Thanks for your answer and explanation!
It wasn't clear for me that I was in merged state.
Best regards,
Patrice

[kerry-meyer]

Hi Patrice,

With the merging of the PR cited above (#396), it will now be possible to gather facts specifying 'gather_network_resources: all' on a platform without port group support. For that case, a warning message will be printed explaining why there is no output for port group configuration, but execution will continue with gathering the 'facts' for other resource modules.