Add support for Junos devices

[arulrajs]

Added Support for Juniper Junos devices.

The Junos configurations can be generated using "junos" option in arouteserver.
Example:

arouteserver junos --general rs1.yml --cfg arouteserver.yml -o junos.conf

Sample arouteserver generated config for Junos device is attached.
junos.conf.txt

[coveralls]

Coverage decreased (-1.08%) to 71.361% when pulling 14ba9a8 on arulrajs:master into 763ebcf on pierky:master.

[pierky]

Hello @arulrajs (and team),

thanks for the PR!

Having an additional BGP speaker added to the list of those available in ARouteServer will make the tool even more interesting for IX operators and hopefully will help to additionally increase the security of that ecosystem and to enrich diversity in the route server landscape.

To move the PR forward it would be great if you could share your plans regarding the features and the integration tests that you think to implement: a list of current "user expectations" is published here https://arouteserver.readthedocs.io/en/latest/SUPPORTED_SPEAKERS.html, I hope you could highlight which items you plan to cover and which to leave out for the time being.

Today all the features are tested both on BIRD and OpenBGPD: https://arouteserver.readthedocs.io/en/latest/LIVETESTS.html

Once I'll know the scope of your work I'll be able to have a better idea on how to review the PR and define further next steps.

Many thanks

[maelmans]

Feature | compliant
Path hiding mitigation (RFC7947, 2.3.1) | Yes

Basic filters:
NEXT_HOP enforcement - strict (RFC7948, 4.8) | Yes
NEXT_HOP enforcement - same AS (RFC7948, 4.8) | Yes
Min and max IPv4/IPv6 prefix length | Yes
Max AS_PATH length | Yes
Reject invalid AS_PATHs (private/invalid ASNs) | Yes
Reject AS_PATHs containing transit-free ASNs | Yes
Reject ‘never via route-servers’ ASNs | Yes
Reject bogons | Yes
Max-prefix limit | Yes

Prefixes and origin ASNs validation:
IRR-based filters (RFC7948, 4.6.2) | Yes
RPKI ROAs used as route objects | Yes
Origin AS from ARIN Whois database dump | Yes
NIC.BR Whois data (slide n. 26) from Registro.br | Yes
RPKI-based filtering (BGP Prefix Origin Validation) | Yes

Blackhole filtering support:
Optional NEXT_HOP rewriting | Yes
Signalling via BLACKHOLE and custom communities) | Yes
Client-by-client control over propagation | Yes

Graceful shutdown support:
GRACEFUL_SHUTDOWN BGP Community | Yes
Graceful shutdown of the route server itself | Yes

Control and informative communities:
Prefix/origin ASN in IRRDBs data | Yes
Do (not) announce to any / peer / on RTT basis | Yes
Prepend to any / peer / on RTT basis | Yes
Add NO_EXPORT / NO_ADVERTISE to any / peer | Yes
Custom informational BGP communities | Yes

Optional session features on a client-by-client basis:
Prepend route server ASN (RFC7947, 2.2.2.1) | Yes
Active sessions | Yes
GTSM (Generalized TTL Security Mechanism) | Yes
ADD_PATH capability (RFC7911) | Yes

[pierky]

Hello @maelmans,

from your comment I understand that the goal of this PR is to generate configurations that offer feature parity with BIRD and OpenBGPD, that's great.

The next step now is to validate those configurations using the integration testing framework that is already implemeted. Since all the features are expected to be available, the framework doesn't need any change and can be used as it is.

The framework works in this way:

1. there are several scenarios which are used to test different features of the configurations built by ARouteServer; they are already existing, and they are designed to cover all the features which are today exposed by the tool;
2. on each scenario, several Docker containers are used to represent some route server clients;
3. each client is configured to announce several routes, with different attributes and characteristics which are defined by the scenario;
4. those clients are configured to establish a BGP session with a couple of predefined IPv4 and IPv6 address, where the route server must be listening on;
5. a vanilla instance of the route server is spun up and configured from scratch through ARouteServer, using the general.yml and clients.yml files from the scenario;
6. the unittest cases are run by the framework and verify that the expectations defined by the scenario are actually met: for example, if a client announces a route that is RPKI INVALID, and the ARouteServer configuration for that scenario has RPKI BOV turned on, then the code verifies that the route is actually dropped by the route server and not propagated to other clients.

The whole framework is already in place, and it's used to validate the configurations for BIRD 1.6, BIRD 2 and OpenBGPD. What is needed in order to extend it to Junos is a new software interface that can be used in the step n. 6 to interact with the route server; this interface will be used by the unittest cases to retrieve the routes from the route server in an abstract and vendor-agnostic way, so that they can be consumed by the existing unittest cases to verify that the expectations are met.

So, the immediate next step is to write a class that implements the BGPSpeakerInstance interface.

As you can see, there are already 2 macro implementations of it, one for BIRD and one for OpenBGPD:

• the one for BIRD is based on a middle-class that is DockerInstance, because in step n. 5 BIRD is spun up as a Docker container;
• the one for OpenBGPD is based on KVMInstance, because OpenBSD can't run inside a Docker container. (There's actually a third one that is for OpenBGPD Portable that is based on Docker as well, beucase the Portable edition works inside a container.)

The goal of the BGPSpeakerInstance interface is to run local commands that can be used to retrieve the information from the route server instance: in case of Docker-based instances, those commands are simple docker exec ... commands; for KVM-based instances, ssh is used to run the actual commands on the "remote" host, that is on the VM.
Probably it's easier to go over the code to actually see how they work:

• https://github.com/pierky/arouteserver/blob/master/pierky/arouteserver/tests/live_tests/openbgpd.py#L116-L125
• https://github.com/pierky/arouteserver/blob/master/pierky/arouteserver/tests/live_tests/openbgpd.py#L228-L238
• https://github.com/pierky/arouteserver/blob/master/pierky/arouteserver/tests/live_tests/bird.py#L86-L107
• https://github.com/pierky/arouteserver/blob/master/pierky/arouteserver/tests/live_tests/bird.py#L201

From a networking perspective, all the clients and the route server are connected to the same linux bridge managed by Docker. https://arouteserver.readthedocs.io/en/latest/LIVETESTS.html#setting-up-the-environment-to-run-live-tests

The existing scenarios can be found here: https://github.com/pierky/arouteserver/tree/master/tests/live_tests/scenarios

Once the route server software interface is defined, it must be simply added to the base.py file of each scenario (as in here) and "consumed" by a new unittest file (as this).

When the tests will run, they'll spin up a brand new instance of the route server, they'll configure it from scratch, using only ARouteServer, and they'll verify that it behaves as expected by the scenario. On TravisCI you can see how the whole framework works: https://travis-ci.org/github/pierky/arouteserver/jobs/662473101 Unfortunately TravisCI doesn't allow spinning up KVM-based VMs, so you'll see only the BIRD and OpenBGPD Portable tests there: the OpenBGP/OpenBSD tests can be reproduced locally, after settig up the environment as described on the links that you'll find below.

Additional info can be found here:

• https://arouteserver.readthedocs.io/en/latest/LIVETESTS.html
• https://arouteserver.readthedocs.io/en/latest/LIVETESTS_CODEDOC.html#module-pierky.arouteserver.tests.live_tests.base

and of course in the code ;-)

At a first glance it seems that some features are not implemeted yet, like "Reject AS_PATHs containing transit-free ASNs", "Reject ‘never via route-servers’ ASNs", most of those inside the "Prefixes and origin ASNs validation", "NEXT_HOP rewriting", most of the informational BGP communities; I expect to see some tests failing, which is not bad in a TDD approach.

Looking forward to seeing the outcome of a full test suite execution, to better understand the current status of the implementation, and also to see how it will be possible to reproduce it.