Skip to content

Layer 3

Jump to bottom
Amy Buck edited this page Jun 6, 2017 · 27 revisions

OpenSwitch OPX supports unicast routing over Linux interfaces using routes in the Linux kernel routing table. Applications can also use the CPS API to configure routes. This information describes how to configure Layer 3 unicast routing to provision the NPU.

The routing subsystem manages the forwarding information base (FIB), and programs routes with resolved next-hops using ARP/Neighbor table entries received from the Linux kernel.

IPv4 routing

A routing table entry consists of a destination IP address prefix and at least one next-hop address or a Linux interface.

$ ip route show
default dev eth0  scope link
3.3.3.0/24 dev e101-003-0  proto kernel  scope link  src 3.3.3.1

$ ip route add 11.10.10.0/24  dev e101-003-0

$ ip route show
default dev eth0  scope link
3.3.3.0/24 dev e101-003-0  proto kernel  scope link  src 3.3.3.1
11.10.10.0/24 dev e101-003-0  scope link

Configure static routing with next-hop

$ ip route add 30.30.30.0/24 via 3.3.3.3

$ ip route show
default dev eth0  scope link
3.3.3.0/24 dev e101-003-0   proto kernel  scope link  src 3.3.3.1
30.30.30.0/24 via 3.3.3.3 dev e101-003-0

Delete static route

$ ip route delete 11.10.10.0/24

$ ip route show
default dev eth0  scope link
3.3.3.0/24 dev e101-003-0  proto kernel  scope link  src 3.3.3.1

To add a persistent static route that is saved after a reboot, configure the route in the /etc/network/interfaces file.

IPv6 routing

You can add, delete, or modify the IPv6 routes and next-hops in the IPv6 routing table.

$ ip -6 route  add 5::5/64 via 3::3

$ ip -6 route show
3::/64 dev e101-003-0  proto kernel  metric 256
5::/64 via 3::3 dev e101-003-0  metric 1024

Debug and troubleshoot IPv6 routing

$ ip monitor
30.30.30.0/24 via 3.3.3.3 dev e00-3
3::/64 via 3::3 dev e101-003-0  metric 1024
5::/64 via 3::3 dev e101-003-0  metric 1024

ARP and neighbor tables

OPX uses ARP and neighbor table entries to resolve adjacencies by using the host IP address-to-MAC address binding. In Linux, the ARP table is used for IPv4 routing, and the neighbor table is used for IPv6 routing.

Configure and view kernel ARP table entries

$ arp -n
Address HWtype HWaddress Flags Mask Iface 3.3.3.4 ether 90:b1:1c:f4:9d:44 C

Configure IPv6 address

$ ifconfig e101-003-0 inet6 add 3::1/64

$ ifconfig e101-003-0
e101-003-0 Link encap:Ethernet  HWaddr 90:b1:1c:f4:a8:ea
inet addr:3.3.3.1  Bcast:3.3.3.255  Mask:255.255.255.0
inet6 addr: 3::1/64 Scope:Global

inet6 addr: fe80::92b1:1cff:fef4:a8ea/64 Scope:Link
UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
RX packets:532 errors:0 dropped:0 overruns:0 frame:0
TX packets:173 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:46451 (45.3 KiB)  TX bytes:25650 (25.0 KiB)

View IPv6 neighbor table

$ ip -6 neighbor show
3::3 dev e101-003-0  lladdr 90:b1:1c:f4:9d:44 router REACHABLE

Ping IPv6 neighbor

$ ping6 3::3 
PING 3::3(3::3) 56 data bytes 
64 bytes from 3::3: icmp_seq=1 ttl=64 time=1.74 ms

$ tcpdump -i e101-003-0 
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode 
listening on e101-003-0, link-type EN10MB (Ethernet), capture size 262144 bytes 
04:30:17.053115 IP6 3::1 > 3::3: ICMP6, echo request, seq 8, length 64

Equal cost multi-path (ECMP)

The Linux networking stack supports ECMP by adding multiple next-hops to the route.

Configure ECMP route

$ ip route add 40.40.40.0/24 nexthop via 3.3.3.6 nexthop via 4.4.4.7

$ ip route show
default dev eth0  scope link
3.3.3.0/24 dev e101-003-0  proto kernel  scope link  src 3.3.3.1
40.40.40.0/24
       nexthop via 3.3.3.6  dev e101-003-0 weight 1
       nexthop via 4.4.4.7  dev e101-004-0 weight 1

NOTE: The Linux kernel provides limited support for IPv6 multi-path routing.

Layer 3 routing topology example

When you configure an IP address, use any Linux utility command such as ip addr add or ifconfig to configure an interface.

Configure IP address on R1

$ ip addr add 10.1.1.1/24 dev e101-007-0
$ ip addr add 11.1.1.1/24 dev e101-001-0

Configure IP address on R2

$ ip addr add 10.1.1.2/24 dev e101-007-0
$ ip addr add 12.1.1.1/24 dev e101-001-0

Verify IP address configuration on R1

$ ip addr show e101-007-0
16: e101-007-0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group
default qlen 500
   link/ether 74:e6:e2:f6:af:87 brd ff:ff:ff:ff:ff:ff
   inet 10.1.1.1/24 scope global e101-007-0
      valid_lft forever preferred_lft forever
   inet6 fe80::76e6:e2ff:fef6:af87/64 scope link
      valid_lft forever preferred_lft forever

$ ip addr show e101-001-0
10: e101-001-0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group
default qlen 500
   link/ether 74:e6:e2:f6:af:81 brd ff:ff:ff:ff:ff:ff
   inet 11.1.1.1/24 scope global e101-001-0
      valid_lft forever preferred_lft forever
   inet6 fe80::76e6:e2ff:fef6:af81/64 scope link
      valid_lft forever preferred_lft forever

Verify IP address configuration on R2

$ ip addr show e101-007-0
16: e101-007-0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group
default qlen 500
   link/ether 74:e6:e2:f6:ba:87 brd ff:ff:ff:ff:ff:ff
   inet 10.1.1.2/24 scope global e101-007-0
      valid_lft forever preferred_lft forever
   inet6 fe80::76e6:e2ff:fef6:ba87/64 scope link
      valid_lft forever preferred_lft forever

$ ip addr show e101-001-0
10: e101-001-0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group
default qlen 500
   link/ether 74:e6:e2:f6:ba:81 brd ff:ff:ff:ff:ff:ff
   inet 12.1.1.1/24 scope global e101-001-0
      valid_lft forever preferred_lft forever
   inet6 fe80::76e6:e2ff:fef6:ba81/64 scope link
      valid_lft forever preferred_lft forever

Enable interfaces on R1 and R2

$ ip link set dev e101-007-0 up
$ ip link set dev e101-001-0 up

Configure static route on server on R1

$ ip route add 12.1.1.0/24 via 10.1.1.2

Configure static route on server on R2

$ ip route add 11.1.1.0/24 via 10.1.1.1

Ping neighbor route and server (Server 2) from R1

$ ping 11.1.1.2

View ARP table on R1

$ arp -n
Address      HWtype  HWaddress           Flags Mask      Iface
11.1.1.2     ether   00:00:00:1d:9a:bd   C               e101-001-0
10.1.1.2     ether   74:e6:e2:f6:ba:87   C               e101-007-0

View ARP table on R2

$ arp -n
Address      HWtype  HWaddress           Flags Mask      Iface
10.1.1.1     ether   74:e6:e2:f6:af:87   C               e101-007-0
12.1.1.2     ether   00:00:00:1d:9a:be   C               e101-001-0

See Programming examples for information on how to program routines using the CPS API.

Dynamic routing

To enable dynamic routing, configure BGP and OSPF using open-source routing stacks, such as Quagga, Bird, and other third-party applications.

© 2019 OpenSwitch project. All information is contributed to and made available by OPX under the Creative Commons Attribution 4.0 International License (available at http://creativecommons.org/licenses/by/4.0/).

Clone this wiki locally