Skip to content

Commit

Permalink
Update after reviews
Browse files Browse the repository at this point in the history
Incorporate review comments
Add synopsis example for each method
  • Loading branch information
tgreenx committed Aug 30, 2023
1 parent 7c70935 commit 478b984
Show file tree
Hide file tree
Showing 16 changed files with 431 additions and 316 deletions.
18 changes: 7 additions & 11 deletions docs/Implementing_Tests.pod
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ to the framework.

=item Make as few assumptions as possible.

When the entry method in a test module is called, it can rely on only a handful of things about the testing process so far:
When the entry method in a test module is called, it can only rely on a handful of things about the testing process so far:

=over

Expand All @@ -35,8 +35,8 @@ The zone object it's given has nameservers.

=item Favor code clarity over performance.

Strive to make the test code as easily understood as possible. Ideally, it should be possible for a person with only a basic understanding of Perl to
read the code and verify that it correctly implements the corresponding test specification.
Strive to make the test code as easily understandable as possible. Ideally, it should be possible for a person with only a basic understanding of Perl to
read the code and verify that it correctly implements the corresponding Test Case specifications.

=back

Expand All @@ -58,18 +58,14 @@ message, but otherwise not used.
=item C<metadata()>

This method should return a reference to a hash. The keys of this hash should be the names of the individual test methods in the module, and the
values for each key should be a reference to an array listing all the possible messages that test method can emit.
values for each key should be a reference to an array listing all the possible messages that test methods can emit.

=back

=head2 Test-running method(s)

The framework expects to be able to run all tests in a module by calling a class method C<all()> with a L<Zonemaster::Engine::Zone> object as the argument.
The return value from this method is expected to be a list of L<Zonemaster::Engine::Logger::Entry> objects describing the results of the tests run. The
example test module has one possible implementation of this.

In the case where one test module wants to use a test in another, the top-level L<Zonemaster::Engine> module has a method C<test_method()> to request that.
This method will call the other test, if it is available and the current profile allows its execution. It's up to the caller to know what arguments
the other test needs, and what it returns.
The framework expects to be able to run all tests in a module by calling a class method L<all()|Zonemaster::Engine::Test/all()> with a
L<Zonemaster::Engine::Zone> object as the argument.
The return value from this method is expected to be a list of L<Zonemaster::Engine::Logger::Entry> objects describing the results of the tests run.

=cut
4 changes: 0 additions & 4 deletions docs/Translation.pod
Original file line number Diff line number Diff line change
Expand Up @@ -45,10 +45,6 @@ The format strings themselves, the comments and the line numbers of the __x
calls are used by the gettext tooling when updating the PO files with new
message ids and old message strings.

Every time you add, remove or modify a tag or its message id, re-run
C<make update-po> as described in the next section.
Make sure the message tag comments are properly added and up to date.

=head2 For Zonemaster package maintainers

In order to make a new translation usable, it must be compiled to C<mo> format
Expand Down
8 changes: 5 additions & 3 deletions lib/Zonemaster/Engine/Constants.pm
Original file line number Diff line number Diff line change
Expand Up @@ -160,7 +160,7 @@ An integer, used to define the size of the serial number space, as defined in RF
An integer, used to define the EDNS0 UDP packet size in EDNS queries.
=item * C<UDP_COMMON_EDNS_LIMIT>
=item * C<$UDP_COMMON_EDNS_LIMIT>
=item * C<@IPV4_SPECIAL_ADDRESSES>
Expand Down Expand Up @@ -216,11 +216,13 @@ Readonly::Array our @IPV6_SPECIAL_ADDRESSES => _extract_iana_ip_blocks($IP_VERSI
=item _extract_iana_ip_blocks()
my @array = _extract_iana_ip_blocks( $ip_version );
Internal method that is used to extract IP blocks details from IANA files for a given IP version (i.e. 4 or 6).
Takes an integer (IP version).
Returns an array of hashes - the keys of which are C<ip> (L<Net::IP::XS> object), C<name> (string) and C<reference> (string).
Returns a list of hashes - the keys of which are C<ip> (L<Net::IP::XS> object), C<name> (string) and C<reference> (string).
=back
Expand Down Expand Up @@ -264,4 +266,4 @@ sub _extract_iana_ip_blocks {
return @list;
} ## end sub _extract_iana_ip_blocks

1;
1;
4 changes: 2 additions & 2 deletions lib/Zonemaster/Engine/Overview.pod
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ If all you want to do is run tests, you need to care about four or five modules.

There are two ways that you can get the results of a test you've requested: the simple-but-inflexible way and the flexible-but-complex way.

The simple-but-inflexible way is that all the methods in L<Zonemaster::Engine> that run tests return lists of L<Zonemaster::Engine::Logger::Entry> objects. Those lists incude all the results that the writer of the test module(s) considered important enough to return by default. The advantage of this method is that it is extremely easy to use. The following is a functional (if not very useful) way to run a full test and print the results from a command-line prompt:
The simple-but-inflexible way is that all the methods in L<Zonemaster::Engine> that run tests return lists of L<Zonemaster::Engine::Logger::Entry> objects. Those lists include all the results that the writer of the test module(s) considered important enough to return by default. The advantage of this method is that it is extremely easy to use. The following is a functional (if not very useful) way to run a full test and print the results from a command-line prompt:

perl -MZonemaster::Engine -E 'say "$_" for Zonemaster::Engine->new->test_zone("example.org")'

Expand Down Expand Up @@ -49,7 +49,7 @@ If you want to develop a test module of your own, the standard set of modules se

As an entry point to the "inside" of the Zonemaster framework, you want to read L<Zonemaster::Engine::Zone> and follow references from there. Of particular interest after the L<Zone|Zonemaster::Engine::Zone> class should be the L<Zonemaster::Engine::Nameserver> and possibly L<Zonemaster::Engine::Recursor> classes.

If you do write your own test module, I would very much appreciate feedback on which parts were tricky to figure out, because I'm honestly not sure what I need to explain here. So please, if there's something that you think really needs to be written about, make an issue at L<https://github.com/zonemaster/zonemaster-engine/issues>.
If you do write your own test module, I would very much appreciate feedback on which parts were tricky to figure out, because I'm honestly not sure what I need to explain here. So please, if there's something that you think really needs to be written about, create an issue at L<https://github.com/zonemaster/zonemaster-engine/issues>.

=head2 For developers of the Zonemaster Test Framework

Expand Down
6 changes: 1 addition & 5 deletions lib/Zonemaster/Engine/Profile.pm
Original file line number Diff line number Diff line change
Expand Up @@ -922,7 +922,7 @@ https://github.com/zonemaster/zonemaster/blob/master/docs/specifications/tests/Z
Related to the REFRESH_MINIMUM_VALUE_LOWER message tag from this test case.
Default C<14400> (4 hours in seconds).
=head2 test_cases_vars04.zone.SOA_RETRY_MINIMUM_VALUE
=head2 test_cases_vars.zone04.SOA_RETRY_MINIMUM_VALUE
A positive integer value.
Recommended lower bound for SOA retry values (in seconds) in test case
Expand Down Expand Up @@ -987,8 +987,4 @@ C<net.ipv6> = true has this JSON representation:
Similar to the L</JSON REPRESENTATION> but uses a YAML format.
=over
=back
=cut
59 changes: 39 additions & 20 deletions lib/Zonemaster/Engine/Test.pm
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ tags that the Test Case can use in L<log entries|Zonemaster::Engine::Logger::Ent
=item tag_descriptions()
This must return a reference to a hash, the keys of which are the message tags and the corresponding values
are strings (message ids) corresponding to user-friendly English translations of those message tags.
are strings (message IDs) corresponding to user-friendly English translations of those message tags.
Keep in mind that the message ids will be used as keys to look up translations into other languages,
so think twice before editing them.
Expand All @@ -107,6 +107,8 @@ BEGIN {
=item _log_versions()
_log_versions();
Adds logging messages regarding the current version of some modules, specifically for L<Zonemaster::Engine> and other dependency modules (e.g. L<Zonemaster::LDNS>).
=back
Expand Down Expand Up @@ -136,8 +138,11 @@ sub _log_versions {
=item modules()
Returns a list with the names of all available Test modules except
L<Zonemaster::Engine::Test::Basic> (since that one is a bit special).
my @modules_array = modules();
Returns a list of strings containing the names of all available Test modules, with the
exception of L<Zonemaster::Engine::Test::Basic> (since that one is a bit special),
based on the content of the B<share/modules.txt> file.
=back
Expand All @@ -151,17 +156,23 @@ sub modules {
=item run_all_for()
Runs all Tests Cases in all Test modules found.
my @logentry_array = run_all_for( $zone );
Runs the L<default set of tests|/all()> of L<all Test modules found|/modules()> for the given zone.
The order in which the Test modules found will be executed is not defined,
except for L<Zonemaster::Engine::Test::Basic> that is always executed first.
If the L<Basic tests|Zonemaster::Engine::Test::Basic/TESTS> fail to indicate
an extremely minimal level of function (it must have a parent domain, and it must
have at least one functional name server) for the zone, no further tests will be executed.
This method always starts with the execution of the L<Basic Test module|Zonemaster::Engine::Test::Basic>.
If the L<Basic tests|Zonemaster::Engine::Test::Basic/TESTS> fail to indicate an extremely minimal
level of function for the zone (e.g., it must have a parent domain, and it must have at least one
functional name server), the testing suite is aborted. See L<Zonemaster::Engine::Basic::can_continue()>
for more details.
Otherwise, other Test modules are L<looked up and loaded|/modules()> from the B<share/modules.txt> file,
and executed in the order in which they appear in the file.
The default set of tests (Test Cases) is specified in the L<all()> method of each Test module. They
can be individually disabled by the L<profile|Zonemaster::Engine::Profile/test_cases>.
Takes a L<Zonemaster::Engine::Zone> object.
Returns an array of L<Zonemaster::Engine::Logger::Entry> objects.
Returns a list of L<Zonemaster::Engine::Logger::Entry> objects.
=back
Expand Down Expand Up @@ -221,13 +232,18 @@ sub run_all_for {
=item run_module()
Runs all default Test Cases of the given Test module for the given zone.
The requested Test module must be in the list of actively loaded modules
(that is, not a module disabled by the current profile)
my @logentry_array = run_module( $module, $zone );
Runs the L<default set of tests|/all()> of the given Test module for the given zone.
The Test module must be in the list of actively loaded modules (that is,
a module defined in the B<share/modules.txt> file).
The default set of tests (Test Cases) is specified in the L<all()> method of each Test module.
They can be individually disabled by the L<profile|Zonemaster::Engine::Profile/test_cases>.
Takes a string (module name) and a L<Zonemaster::Engine::Zone> object.
Returns an array of L<Zonemaster::Engine::Logger::Entry> objects.
Returns a list of L<Zonemaster::Engine::Logger::Entry> objects.
=back
Expand Down Expand Up @@ -279,14 +295,17 @@ sub run_module {
=item run_one()
Runs the given Test method of the given Test module for the given zone.
The requested Test module must be in the list of actively loaded modules
(that is, not a module disabled by the current profile), and the Test method
my @logentry_array = run_one( $module, $test_case, $zone );
Runs the given Test Case of the given Test module for the given zone.
The Test module must be in the list of actively loaded modules (that is,
a module defined in the B<share/modules.txt> file), and the Test Case
must be listed in the L<metadata|/metadata()> of the Test module exports.
Takes a string (module name), a string (test method name) and an array of L<Zonemaster::Engine::Zone> objects.
Takes a string (module name), a string (test case name) and an array of L<Zonemaster::Engine::Zone> objects.
Returns an array of L<Zonemaster::Engine::Logger::Entry> objects.
Returns a list of L<Zonemaster::Engine::Logger::Entry> objects.
=back
Expand Down Expand Up @@ -347,4 +366,4 @@ sub run_one {
return;
} ## end sub run_one

1;
1;
36 changes: 23 additions & 13 deletions lib/Zonemaster/Engine/Test/Address.pm
Original file line number Diff line number Diff line change
Expand Up @@ -23,20 +23,22 @@ Zonemaster::Engine::Test::Address - Module implementing tests focused on IP addr
=head1 SYNOPSIS
my @results = Zonemaster::Engine::Test::Address->all($zone);
my @results = Zonemaster::Engine::Test::Address->all( $zone );
=head1 METHODS
=over
=item all()
my @logentry_array = all( $zone );
Runs the default set of tests for that module, i.e. between L<two and three tests|/TESTS> depending on the tested zone.
If L<ADDRESS02|/address02()> passes, L<ADDRESS03|/address03()> is run.
Takes a L<Zonemaster::Engine::Zone> object.
Returns an array of L<Zonemaster::Engine::Logger::Entry> objects.
Returns a list of L<Zonemaster::Engine::Logger::Entry> objects.
=back
Expand All @@ -60,6 +62,8 @@ sub all {
=item metadata()
my $hash_ref = metadata();
Returns a reference to a hash, the keys of which are the names of all Test Cases in the module, and the corresponding values are references to
an array containing all the message tags that the Test Case can use in L<log entries|Zonemaster::Engine::Logger::Entry>.
Expand Down Expand Up @@ -158,9 +162,11 @@ Readonly my %TAG_DESCRIPTIONS => (
=item tag_descriptions()
my $hash_ref = tag_descriptions();
Used by the L<built-in translation system|Zonemaster::Engine::Translator>.
Returns a reference to a hash, the keys of which are the message tags and the corresponding values are strings (message ids).
Returns a reference to a hash, the keys of which are the message tags and the corresponding values are strings (message IDs).
=back
Expand All @@ -174,6 +180,8 @@ sub tag_descriptions {
=item version()
my $version_string = version();
Returns a string containing the version of the current module.
=back
Expand All @@ -188,6 +196,8 @@ sub version {
=item find_special_address()
my $hash_ref = find_special_address( $ip );
Verifies if an IP address is a special (private, reserved, ...) one.
Takes a L<Net::IP::XS> object.
Expand Down Expand Up @@ -224,13 +234,13 @@ sub find_special_address {
=item address01()
Test Case that verifies if IP addresses are not in private networks.
my @logentry_array = address01( $zone );
See L<Address01 specification|https://github.com/zonemaster/zonemaster/blob/master/docs/public/specifications/tests/Address-TP/address01.md> for more details.
Runs the L<Address01 Test Case|https://github.com/zonemaster/zonemaster/blob/master/docs/public/specifications/tests/Address-TP/address01.md>.
Takes a L<Zonemaster::Engine::Zone> object.
Returns a list of an array of L<Zonemaster::Engine::Logger::Entry> objects and a L<Zonemaster::Engine::Logger::Entry> object.
Returns a list of L<Zonemaster::Engine::Logger::Entry> objects.
=back
Expand Down Expand Up @@ -277,13 +287,13 @@ sub address01 {
=item address02()
Test Case that verifies if reverse DNS entries exist for each name server IP addresses.
my @logentry_array = address02( $zone );
See L<Address02 specification|https://github.com/zonemaster/zonemaster/blob/master/docs/public/specifications/tests/Address-TP/address02.md> for more details.
Runs the L<Address02 Test Case|https://github.com/zonemaster/zonemaster/blob/master/docs/public/specifications/tests/Address-TP/address02.md>.
Takes a L<Zonemaster::Engine::Zone> object.
Returns a list of an array of L<Zonemaster::Engine::Logger::Entry> objects and a L<Zonemaster::Engine::Logger::Entry> object.
Returns a list of L<Zonemaster::Engine::Logger::Entry> objects.
=back
Expand Down Expand Up @@ -350,13 +360,13 @@ sub address02 {
=item address03()
Test Case that verifies if reverse DNS entries match name server names.
my @logentry_array = address03( $zone );
See L<Address03 specification|https://github.com/zonemaster/zonemaster/blob/master/docs/public/specifications/tests/Address-TP/address03.md> for more details.
Runs the L<Address03 Test Case|https://github.com/zonemaster/zonemaster/blob/master/docs/public/specifications/tests/Address-TP/address03.md>.
Takes a L<Zonemaster::Engine::Zone> object.
Returns a list of an array of L<Zonemaster::Engine::Logger::Entry> objects and a L<Zonemaster::Engine::Logger::Entry> object.
Returns a list of L<Zonemaster::Engine::Logger::Entry> objects.
=back
Expand Down Expand Up @@ -430,4 +440,4 @@ sub address03 {
return ( @results, info( TEST_CASE_END => { testcase => (split /::/, (caller(0))[3])[-1] } ) );
} ## end sub address03

1;
1;
Loading

0 comments on commit 478b984

Please sign in to comment.