Use Zonemaster::Engine::Normalization module for input name normalization

[tgreenx]

Purpose

This PR introduces the usage of the Zonemaster::Engine::Normalization module for input DNS name normalization.
It also adds input validation to the --ns argument..

Context

The current specification can found here, and will replace the Basic00 Test Case.

Changes

• Make use of Zonemaster::Engine::Normalization::normalize_name()
• Significantly improve --ns argument validation:
  • Add check for several slash (/) characters
  • Add IP address validation which leverages both regex checks and Net::IP::XS::Error() for detailed error messages
• Remove Zonemaster::CLI::to_idn()
• List Readonly and Net::IP::XS as direct dependencies

How to test this PR

Testing should proceed as before, although more errors are now supported and several can be reported simultaneously:

$ zonemaster-cli --show-testcase --no-ipv6 --test basic/basic01 '@.İabcİ.İcc'
Ambiguous downcaseing of character "LATIN CAPITAL LETTER I WITH DOT ABOVE" in the domain name. Use all lower case instead.

$ zonemaster-cli --show-testcase --no-ipv6 --test basic/basic01
Must give the name of a domain to test.

$ zonemaster-cli --show-testcase --no-ipv6 --test basic/basic01 ''
Must give the name of a domain to test.

$ zonemaster-cli --show-testcase --no-ipv6 --test basic/basic01 ' '
Domain name is empty.

$ zonemaster-cli --show-testcase --no-ipv6 --test basic/basic01 ' @!.'
Domain name has an ASCII label ("@!") with a character not permitted.

$ zonemaster-cli --show-testcase --no-ipv6 --test basic/basic01 zonemaster.net --ns 'ns1/.zonemaster.net/10.0.0.1'
--ns must be a name or a name/ip pair.

$ zonemaster-cli --show-testcase --no-ipv6 --test basic/basic01 zonemaster.net --ns 'ns1.zonemaster.net/1'
Invalid IP address in --ns argument.

$ zonemaster-cli --show-testcase --no-ipv6 --test basic/basic01 zonemaster.net --ns 'ns1.zonemaster.net/10.0.0.Z'
Invalid IP address in --ns argument:
        Invalid chars in IP 10.0.0.Z
        
$ zonemaster-cli --show-testcase --no-ipv6 --test basic/basic01 zonemaster.net --ns 'ns1.zonemaster.net/fe10:::'
Invalid IP address in --ns argument:
        Invalid address fe10::: (More than one :: pattern)

$ zonemaster-cli --show-testcase --no-ipv6 --test basic/basic01 zonemaster.net --ns '[email protected]@.net!.'
Invalid name in --ns argument:
        Domain name has an ASCII label ("ns1@") with a character not permitted.
        Domain name has an ASCII label ("zonemaster@") with a character not permitted.
        Domain name has an ASCII label ("net!") with a character not permitted.

$ zonemaster-cli --show-testcase --no-ipv6 --test basic/basic01 zonemaster.net --ns 'ns1.zonemaster.net' --ns 'ns2.zonemaster.net/10.0.0.2' --ns '[email protected].!net.'
Invalid name in --ns argument:
        Domain name has an ASCII label ("ns3@") with a character not permitted.
        Domain name has an ASCII label ("!net") with a character not permitted.

Examples of (in)valid domain names can be found in the t/normalization.t unit test file in Engine, e.g:

#Valid
example。com.
café.example.com

#Invalid
example。.com.
bad:%;!$.example.com.
❤️．example．com

[tgreenx]

Seems like at first I forgot about the normalization of name server names. It is now done (commit 4ab0aaf). I also pushed another commit (0c8093e) that removes the now unused Zonemaster::CLI::to_idn() method.

By the way, in both the specification and the implementation (message ids) of Zonemaster::Engine::Normalization we might want to rename 'domain name' to 'DNS name' to avoid possible confusion, since name server names are also normalized by this module. Note that we do in fact already have a module named Zonemaster::Engine::DNSName.

Also, in this PR , for normalization of name server names, I added the message Invalid name in --ns argument: followed by indented error message(s) for clarity.

[mattias-p]

I like this change but I have a couple of nits.

[matsduf]

$ zonemaster-cli --show-testcase --no-ipv6 --test basic/basic01 '@.İabcİ.İcc'
Domain name has an ASCII label ("@") with a character not permitted.
Domain name has a non-ASCII label ("İabcİ") which is not a valid U-label.
Domain name has a non-ASCII label ("İcc") which is not a valid U-label.

The problem with "İabcİ" is that it cannot be downcased in an unambiguous way. The specification says "AMBIGUOUS_DOWNCASING" (https://github.com/zonemaster/zonemaster/blob/master/docs/public/specifications/tests/RequirementsAndNormalizationOfDomainNames.md).

[hannaeko]

I tested the PR with café.test and also got an INVALID_ASCII error. There seems to be missing decoding from utf8 (or whatever encoding the terminal is using), I added Encode::decode("utf8", $domain ) before passing the domain to nomalize_name and it worked.

[matsduf]

Does it work correctly? On FreeBSD I get

# zonemaster-cli råttgift.se
Domain name has a non-ASCII label ("råttgift") which is not a valid U-label.

Then I tested on Ubuntu 22.04 and get the same error.

[tgreenx]

Does it work correctly? On FreeBSD I get

# zonemaster-cli råttgift.se
Domain name has a non-ASCII label ("råttgift") which is not a valid U-label.

Then I tested on Ubuntu 22.04 and get the same error.

As suggested by @blacksponge, with proper decoding this should now work as expected. See commit ce9b025.

[hannaeko]

Looks fine to me! I tested some of the test commands and it works as expected.

[mattias-p]

🎉

[matsduf]

Nice to have it in place! This gives consistent and documented behavior.

[mattias-p]

v2023.2 release testing

I've tested this for v2023.2 and I could not find any deviations between the described behavior and the actual behavior.

However, upon reflection I think the behavior could be improved in a couple of ways. I created #364 for this.