Integration tests for certbot-dns-rfc2136

[bmw]

For #8134, I had to update our pinned version of dnspython to a new major version which includes some (what seems to be minor) API changes. I would like to verify that the plugin still works with this new version of dnspython.

Ideally, I think we should write simple, automated integration tests for this plugin. We can run our own RFC 2136 compliant DNS server locally. We may need to merge #7722 as part of this.

Alternatively, scripts/instructions for how to do this integration test manually would be useful. I've personally never done it before. If we go this route and we're going to figure out how to do it manually, I think we may as well shove it in our documentation somewhere to make this easier to do in the future.

[bmw]

@alexzorin, please don't spend much time on this unless you want to, but based on how I've seen you carefully recreate user's environments on the community forum over the years, this seems to me like the kind of thing I think you'd be good at/have experience with.

If you have any suggestions or pointers here, they'd be much appreciated. If not, no worries.

[alexzorin]

I only have superficial experience with setups of RFC2136, but it does seem like a necessary addition. Especially if something like #7244 is to be merged one day.

We could give BIND or Knot DNS the same treatment that Pebble and Boulder get in the integration tests.

How deeply would we want to go?

On one hand, it is probably straightforward to make integration tests for the plugin today, against just BIND, due to the plugin's simple implementation. Quite achievable to have assurances that upgrading libraries did not break things.

But I imagine we'd need something very comprehensive for something like that PR with recursion/CNAME chasing. We would want quite a big spread of zones/RPZs and perhaps a couple of different servers. Considering how complicated DNS can get compared to webserver configurations, I am not sure how good a job one could do. But that might be a commentary on that PR rather than testing in general, I'm not sure.

[adferrand]

The existing integration tests are composed of fundamental tests, covering most used user flows, and specific tests covering edge cases that have been added progressively in the certbot project lifetime.

I think we can start with something very simple, and enrich it the same way later on.

[bmw]

Thanks!

I agree with both of you that for what we need now, we should be able to start with something quite simple.

Right now I'm imagining something like its own directory in certbot-ci for these tests that sets up a DNS server along with pebble/boulder and uses #7722 to configure the ACME server to use the local DNS server. BIND seems like a good pick, but it doesn't seem like BIND has an official Docker image which would have been nice. Maybe we can use one of the 3rd party ones or a different DNS server altogether.

As for the actual test we run, for me to personally be satisfied with the initial version of this for #8134, I think it can be extremely simple. I think a successful run of certbot certonly using the RFC2136 plugin for a domain is probably a good enough start.

This is a fair bit of work, but since I think we should at least do something like it manually, I figure we may as well automate it.

[alexzorin]

but it doesn't seem like BIND has an official Docker image which would have been nice

Quite hidden from sight, but https://hub.docker.com/r/internetsystemsconsortium/bind9 appears to be an official ISC BIND image.

[alexzorin]

I've got something going in #8448 but it's probably quite far away. Basic design feedback would be welcome.