How to handle different ifindex_offset in containerlab?

[ssasso]

I'm trying to understand if and how it's possible to handle the ifindex_offset in the clab links/endpoints data generation differently than the main topology data.

With some containers that are build with vrnetlab, eth0 is the container management interface, and eth1,2,3 are the "data ports".
However, this may clash with some device's ifindex_offset when is set to 0.

In example, trying to use the Juniper vMX (borrowing all the data from the vSRX model), I have the following topology.yml:

---
provider: clab

defaults.devices.vsrx.clab.image: vrnetlab/vr-vmx:18.2R1.9
defaults.devices.vsrx.clab.node.kind: vr-vmx
defaults.devices.vsrx.clab.interface.name: eth%d

module: [ bgp ]

nodes:
  r1:
    device: vsrx
    bgp.as: 101
  r2:
    device: vsrx
    bgp.as: 102

links:
- r1:
  r2:

Which will result in this clab file:

name: nsim

mgmt:
  network: netlab_mgmt
  ipv4_subnet: 192.168.121.0/24

topology:
  nodes:
    r1:
      mgmt_ipv4: 192.168.121.101
      kind: vr-vmx
      image: vrnetlab/vr-vmx:18.2R1.9
      runtime: docker
      group: as101
    r2:
      mgmt_ipv4: 192.168.121.102
      kind: vr-vmx
      image: vrnetlab/vr-vmx:18.2R1.9
      runtime: docker
      group: as102

  links:
  - endpoints:
    - "r1:eth0"
    - "r2:eth0"

Which is wrong on the "eth0" interface name for the container.

At the same time, if I increment the ifindex_offset to 1, then the endpoints are correct:

  links:
  - endpoints:
    - "r1:eth1"
    - "r2:eth1"

but the result is a wrong interface configuration (see the following extract from r1 host_vars):

interfaces:
- clab:
    name: eth1
  ifindex: 1
  ifname: ge-0/0/1
  ipv4: 10.1.0.1/30
  linkindex: 1
  name: r1 -> r2
  neighbors:
  - ifname: ge-0/0/1
    ipv4: 10.1.0.2/30
    node: r2
  role: external
  type: p2p

(the interface shall be ge-0/0/0)

@ipspace @jbemmel did you aready encountered such situation with clab? any idea on how this can be quickly solved?

[ssasso]

(current workaround:

1. set ifindex_offset to 1
2. run netlab create
3. manually start containerlab
4. set ifindex_offset to 0
5. run netlab create
6. run netlab initial

)

[ssasso]

At the code level, everything should happen here:

netlab/netsim/augment/links.py

Line 175 in ed204b8

pdata.name = pdata.name % ifindex

But I don't know if it makes sense to add an additiona pdata attribute to be used as additional offset?

[jbemmel]

I had a similar situation on SR OS, where port names depend on the ifindex range (e.g. "1/1/1" for regular ports, "1/1/c32/1" for some higher speed special connectors)

My solution was to apply eval to the interface name, to allow for Python logic expressions

For example:

defaults.devices.sros.interface_name_eval: "'1/1/%d' if ifindex<33 else '1/1/c%d/1'"

Perhaps this could work here too, e.g. f"ge-0/0/{ ifindex-1 }"

[ipspace]

I had a similar situation on SR OS, where port names depend on the ifindex range (e.g. "1/1/1" for regular ports, "1/1/c32/1" for some higher speed special connectors)

My solution was to apply eval to the interface name, to allow for Python logic expressions

I can't find that in the code base. Please help me out.

[jbemmel]

I did not submit that PR yet, but the code location is https://github.com/ipspace/netlab/blob/dev/netsim/augment/links.py#L166

My patch currently uses a different attribute name, but I'm thinking we could simply apply 'eval' to interface_name and if it's a regular string then that should be fine

[ipspace]

I knew that it was a bad idea packing VMs into containers, but why would anyone ever listen to a GONER...

The simplest hack would be to use string.format() when dealing with clab.interface.name using ifdata as kwargs. That would require changes in default settings for all platforms using clab, but it would just be replacing %d with {ifindex} in most cases. You could use expressions in curly braces for example {ifindex +1}.

[ipspace]

Well, we could always pick up the discussion we had in #131. I could modify the generic Vagrantfile template to use "raw" template for special VMs (and you could insert the multi-VM stuff in that template). Just let me know when you're ready to go.

As for string.format, I'll fix that (it's in the "yeah, that's cool, let's do it" category).

[ssasso]

I knew that it was a bad idea packing VMs into containers

And here I come with another "showstopper" about this :-(
vagrant waits for the VM to boot after passing the control back to netlab...
but the vrnetlab containers started from clab does not.

So after the container start, there are some minutes to wait until the VM-inside-container is ready.
The "nice" part is the following one:

• the container is replying to pings (so I cannot use ansible's wait_for_connection)
• the container does proxy the port 22 (with netcat), to redirect the content to the SSH port of the VM, even if the VM is not started. that means the 3-WAY handshake completes successfully, making it impossible to use ansible's wait_for port=22.

I'm trying to use a loop of gather_facts, but right now I'm stuck.

[ipspace]

Will you keep hammering this square peg into this round hole or is it time for Plan B? You could try junos_command in a loop until it succeeds. Here'a s snippet found after 3 seconds of googling around (replacing "file" with "junos_command" could do the trick)

---
- name: Retry until a file is available
  tasks:
    - name: Validate if the file is present
      file:
        path: /tmp/myprocess.pid
        state: file
      register: lsresult
      until: "lsresult is not failed"
      retries: 10
      delay: 10

I use something similar in netsim/ansible/tasks/nxos/initial.yml because Nexus OS needs a minute after it boots to recognize its Ethernet "linecard".

[ssasso]

Will you keep hammering this square peg into this round hole or is it time for Plan B? You could try junos_command in a loop until it succeeds. Here'a s snippet found after 3 seconds of googling around (replacing "file" with "junos_command" could do the trick)

---
- name: Retry until a file is available
  tasks:
    - name: Validate if the file is present
      file:
        path: /tmp/myprocess.pid
        state: file
      register: lsresult
      until: "lsresult is not failed"
      retries: 10
      delay: 10

I use something similar in netsim/ansible/tasks/nxos/initial.yml because Nexus OS needs a minute after it boots to recognize its Ethernet "linecard".

I was trying to make it generic for any vrnetlab container... but at this point it could be almost impossible to be generic.

[ipspace]

I was trying to make it generic for any vrnetlab container... but at this point it could be almost impossible to be generic.

Considering how badly broken that whole thing is, it would be hard to do something "generic", unless you want to use the raw module and have the command-to-execute specified in device defaults.

[ipspace]

The interface name fix is in: 5baae51. Now you can use clab.interface.name: eth{ifindex + 1}

@ssasso: Will you add vMX as a supported platform?

[ssasso]

The interface name fix is in: 5baae51. Now you can use clab.interface.name: eth{ifindex + 1}

My ignorance is total, forgive me. Having this on the topology-defaults:

  vmx:
    interface_name: ge-0/0/{ifindex}
    ifindex_offset: 0
    clab:
      image: vrnetlab/vr-vmx:18.2R1.9
      node:
        kind: vr-vmx
      interface:
        name: eth{ifindex + 1}

What I obtain is:

root@hippo:~/clab/test-vmx# netlab create
Traceback (most recent call last):
  File "/usr/local/bin/netlab", line 11, in <module>
    load_entry_point('networklab', 'console_scripts', 'netlab')()
  File "/root/GIT_H/netlab/netsim/cli/__init__.py", line 157, in lab_commands
    mod.run(sys.argv[arg_start:])   # type: ignore
  File "/root/GIT_H/netlab/netsim/cli/create.py", line 76, in run
    augment.main.transform(topology)
  File "/root/GIT_H/netlab/netsim/augment/main.py", line 85, in transform
    transform_data(topology)
  File "/root/GIT_H/netlab/netsim/augment/main.py", line 64, in transform_data
    augment.links.transform(topology.links,topology.defaults,topology.nodes,topology.pools)
  File "/root/GIT_H/netlab/netsim/augment/links.py", line 770, in transform
    create_node_interfaces(link,pools,nodes,defaults=defaults)
  File "/root/GIT_H/netlab/netsim/augment/links.py", line 579, in create_node_interfaces
    node_intf = add_node_interface(ndict[node],ifdata,defaults)   # Attach new interface to its node
  File "/root/GIT_H/netlab/netsim/augment/links.py", line 185, in add_node_interface
    pdata.name = get_interface_name(pdata.name,ifdata)
  File "/root/GIT_H/netlab/netsim/augment/links.py", line 155, in get_interface_name
    return fmt.format(**ifdata)
KeyError: 'ifindex + 1'

[ipspace]

This seems to be my misunderstanding of how str.format works :(( Will try to figure it out...

[ssasso]

This seems to be my misunderstanding of how str.format works :(( Will try to figure it out...

If this can help:
In "raw" python it works with direct interpolation:

>>> a=7
>>> b=1
>>> f'Res {a+b}'
'Res 8'
>>> f'Res {a+1}'
'Res 8'

But not with format:

>>> 'Res {a+1}'.format(a=a,b=b)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyError: 'a+1'
>>> 'Res {a+b}'.format(a=a,b=b)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyError: 'a+b'

[ipspace]

Found a really convoluted way of getting it done on (where else) StackOverflow ;))

Amended the commit (makes no sense to have another commit saying "I don't understand how Python works") and force-pushed it to dev branch. {ifindex + 1} works now (and this time I tested it ;)

[ssasso]

I can add the "official" support for vMX for sure, starting from the interface name fix.
I think it's also good if we can resume together the discussion of #131.

But, first of all, I'll open another discussion for defining how to handle different Juniper products with less work but keeping everything consistent.

[ipspace]

Implemented in 322e3f6