Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refactor ASNLookup code and documentation #1257

Merged
merged 13 commits into from
Sep 17, 2024
Merged

Refactor ASNLookup code and documentation #1257

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
c6f1875
Make ASNLookup.pm use standard recursive DNS lookup
tgreenx Jul 18, 2023
233b757
Removes "asnroots" profile parameter - #1052
tgreenx Jul 19, 2023
620fa79
Update Profile.pm documentation
tgreenx Jul 19, 2023
7eae991
Update unitary test data
tgreenx Nov 7, 2023
37ede66
Enables backup source functionality in Zonemaster::Engine::ASNLookup
tgreenx May 16, 2024
677bff7
Discard responses with non NOERROR/NXDOMAIN rcodes in Cymru ASN lookup
tgreenx May 21, 2024
d807920
Fix bug by reworking code logic, refactoring
tgreenx May 23, 2024
9b3004e
Merge branch 'develop' into update-asnlookup
tgreenx Jun 17, 2024
752e0fd
Merge branch 'develop' into update-asnlookup
tgreenx Jul 17, 2024
49ed934
Restore deleted unit test for asn_db profile properties
tgreenx Jul 17, 2024
582a500
Update after review
tgreenx Jul 25, 2024
e57de32
Refactoring
tgreenx Jul 25, 2024
ec2ca1a
Add new message tag ASN_LOOKUP_SOURCE to the profile
tgreenx Jul 25, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
106 changes: 47 additions & 59 deletions lib/Zonemaster/Engine/ASNLookup.pm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ use warnings;
use version; our $VERSION = version->declare( "v1.0.11" );

use Zonemaster::Engine;
use Zonemaster::Engine::Util qw( name );
use Zonemaster::Engine::Nameserver;
use Zonemaster::Engine::Profile;

Expand All @@ -21,23 +22,9 @@ sub get_with_prefix {
my ( $class, $ip ) = @_;

if ( not @db_sources ) {
#
# Backward compatibility in case asnroots is still configured in profile
# but we prefer new model if present
#
my @roots;
if ( Zonemaster::Engine::Profile->effective->get( q{asnroots} ) ) {
@roots = map { Zonemaster::Engine->zone( $_ ) } @{ Zonemaster::Engine::Profile->effective->get( q{asnroots} ) };
}
if ( scalar @roots ) {
@db_sources = @roots;
$db_style = q{cymru};
}
else {
$db_style = Zonemaster::Engine::Profile->effective->get( q{asn_db.style} );
my %db_sources = %{ Zonemaster::Engine::Profile->effective->get( q{asn_db.sources} ) };
@db_sources = map { Zonemaster::Engine->zone( $_ ) } @{ $db_sources{ $db_style } };
}
$db_style = Zonemaster::Engine::Profile->effective->get( q{asn_db.style} );
my %db_sources = %{ Zonemaster::Engine::Profile->effective->get( q{asn_db.sources} ) };
@db_sources = map { name( $_ ) } @{ $db_sources{ $db_style } };
}

if ( not ref( $ip ) or not $ip->isa( 'Net::IP::XS' ) ) {
Expand All @@ -58,10 +45,10 @@ sub get_with_prefix {
}
else {
if ( not $db_style ) {
die "ASN database style is [UNDEFINED]";
die "ASN database style undefined";
}
else {
die "ASN database style value [$db_style] is illegal";
die "ASN database style value '$db_style' is illegal";
}
}
mattias-p marked this conversation as resolved.
Show resolved Hide resolved

Expand All @@ -75,60 +62,61 @@ sub _cymru_asn_lookup {
my $ip = shift;
my @asns = ();

my $reverse = $ip->reverse_ip;
my $db_source_nb = 0;
foreach my $db_source ( @db_sources ) {
my $domain = $db_source->name->string;
Zonemaster::Engine->logger->add( ASN_LOOKUP_SOURCE => { name => $db_source } );
my $reverse = $ip->reverse_ip;
my $domain = $db_source->string;
my $pair = {
'in-addr.arpa.' => "origin.$domain",
'ip6.arpa.' => "origin6.$domain",
};
$db_source_nb++;

foreach my $root ( keys %{$pair} ) {
if ( $reverse =~ s/$root/$pair->{$root}/ix ) {
my $p = $db_source->query_persistent( $reverse, 'TXT' );
my @rr;
my $p = Zonemaster::Engine->recurse( $reverse, 'TXT' );

if ( $p ) {
@rr = $p->get_records( 'TXT' );
}
if ( $p and ( $p->rcode eq q{NXDOMAIN} or ( $p->rcode eq q{NOERROR} and not scalar @rr ) ) ) {
return \@asns, undef, q{}, q{EMPTY_ASN_SET};
}
if ( not $p or $p->rcode ne q{NOERROR} ) {
if ( $db_source_nb == scalar @db_sources ) {
return \@asns, undef, q{}, q{ERROR_ASN_DATABASE};
if ( $p->rcode eq q{NOERROR} ) {
my @rr = $p->get_records( 'TXT', 'answer' );

if ( @rr ) {
my $prefix_length = 0;
my @fields;
my $str;

foreach my $rr ( @rr ) {
my $_str = $rr->txtdata;
my @_fields = split( /[ ][|][ ]?/x, $_str );
my @_asns = split( /\s+/x, $_fields[0] );
my $_prefix_length = ($_fields[1] =~ m!^.*[/](.*)!x)[0];
if ( $_prefix_length > $prefix_length ) {
$str = $_str;
@asns = @_asns;
@fields = @_fields;
$prefix_length = $_prefix_length;
}
}

return \@asns, Net::IP::XS->new( $fields[1] ), $str, q{AS_FOUND};
}
else {
return \@asns, undef, q{}, q{EMPTY_ASN_SET};
}
}
else {
last;
elsif ( $p->rcode eq q{NXDOMAIN} ) {
if ( $p->get_records( 'SOA', 'authority' ) and scalar $p->get_records( 'SOA', 'authority' ) == 1 and ($p->get_records( 'SOA', 'authority' ))[0]->owner eq name( $db_source ) ) {
return \@asns, undef, q{}, q{EMPTY_ASN_SET};
}
}
}

my $prefix_length = 0;
my @fields;
my $str;
foreach my $rr ( @rr ) {
my $_str = $rr->txtdata;
my @_fields = split( /[ ][|][ ]?/x, $_str );
my @_asns = split( /\s+/x, $_fields[0] );
my $_prefix_length = ($_fields[1] =~ m!^.*[/](.*)!x)[0];
if ( $_prefix_length > $prefix_length ) {
$str = $_str;
@asns = @_asns;
@fields = @_fields;
$prefix_length = $_prefix_length;
}
}
if ( scalar @rr ) {
return \@asns, Net::IP::XS->new( $fields[1] ), $str, q{AS_FOUND};
}
else {
if ( $db_source_nb == scalar @db_sources ) {
return \@asns, undef, $str, q{ERROR_ASN_DATABASE};
}
else {
last;
}
if ( $db_source_nb == scalar @db_sources ) {
return \@asns, undef, q{}, q{ERROR_ASN_DATABASE};
}

last;
}
}
} ## end foreach my $db_source ( @db_sources )
Expand All @@ -142,7 +130,7 @@ sub _ripe_asn_lookup {
my $db_source_nb = 0;
foreach my $db_source ( @db_sources ) {
$db_source_nb++;
my $socket = IO::Socket::INET->new( PeerAddr => $db_source->name->string,
my $socket = IO::Socket::INET->new( PeerAddr => $db_source->string,
PeerPort => q{43},
Proto => q{tcp} );
unless ( $socket ) {
Expand Down
35 changes: 9 additions & 26 deletions lib/Zonemaster/Engine/Profile.pm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -107,20 +107,6 @@ my %profile_properties_details = (
q{no_network} => {
type => q{Bool}
},
q{asnroots} => {
type => q{ArrayRef},
test => sub {
foreach my $ndd ( @{$_[0]} ) {
die "Property asnroots has a NULL item\n" if not defined $ndd;
die "Property asnroots has a non scalar item\n" if not defined ref($ndd);
die "Property asnroots has an item too long\n" if length($ndd) > 255;
foreach my $label ( split /[.]/, $ndd ) {
die "Property asnroots has a non domain name item\n" if $label !~ /^[a-z0-9](?:[-a-z0-9]{0,61}[a-z0-9])?$/;
}
}
},
default => ["asnlookup.zonemaster.net"],
},
q{asn_db.style} => {
type => q{Str},
test => sub {
Expand Down Expand Up @@ -734,25 +720,22 @@ A boolean. If true, network traffic is forbidden. Default false.
Use when you want to be sure that any data is only taken from a preloaded
cache.

=head2 asnroots (DEPRECATED)

An arrayref of domain names. Default C<["asnlookup.zonemaster.net"]>.
=head2 asn_db.style

The domains will be assumed to be Cymru-style AS lookup zones.
Only the first name in the list will be used.
A string that is either C<"Cymru"> or C<"RIPE"> (case-insensitive).

=head2 asn_db.style
Defines which service will be used for AS lookup zones.

A string that is either C<"Cymru"> or C<"RIPE">. Defines which method will
be used for AS lookup zones.
Default C<"Cymru">.

=head2 asn_db.sources

An arrayref of domain names when asn_db.style is set to C<"Cymru"> or whois
servers when asn_db.style is set to C<"RIPE">. Only the first item
in the list will be used.
Default C<"asnlookup.zonemaster.net">.
A hash of arrayrefs of strings. The currently supported keys are C<"Cymru"> or C<"RIPE"> (case-insensitive).

For C<"Cymru">, the strings are domain names. For C<"RIPE">, they are WHOIS servers. Normally only the first
item in the list will be used, the rest are backups in case the previous ones didn't work.

Default C<{Cymru: [ "asnlookup.zonemaster.net", "asn.cymru.com" ], RIPE: [ "riswhois.ripe.net" ]}>.

=head2 cache (EXPERIMENTAL)

Expand Down
3 changes: 2 additions & 1 deletion share/profile.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"asn_db" : {
"style" : "Cymru",
"sources" : {
"Cymru" : [ "asnlookup.zonemaster.net" ],
"Cymru" : [ "asnlookup.zonemaster.net", "asn.cymru.com" ],
"RIPE" : [ "riswhois.ripe.net" ]
}
},
Expand Down Expand Up @@ -420,6 +420,7 @@
"TEST_CASE_START" : "DEBUG"
},
"SYSTEM" : {
"ASN_LOOKUP_SOURCE": "DEBUG",
"CACHE_CREATED" : "DEBUG2",
"CACHE_FETCHED" : "DEBUG2",
"CACHED_RETURN" : "DEBUG3",
Expand Down
1 change: 1 addition & 0 deletions share/profile.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -474,6 +474,7 @@ test_levels:
TEST_CASE_END: DEBUG
TEST_CASE_START: DEBUG
SYSTEM:
ASN_LOOKUP_SOURCE: DEBUG
CACHED_RETURN: DEBUG3
CACHE_CREATED: DEBUG2
CACHE_FETCHED: DEBUG2
Expand Down
Loading
Loading