Small cleanups.new working demo driver, lots of docs (not complete)

README.md

@@ -20,7 +20,7 @@ the many changes in this release.
 - Support for SAN certificates.
 - Supports wildcard certificates.
 - Bundling certificates.
-- Supports [DNS](#dns-services-supported) and HTTP challenges.
+- Supports [DNS](dns-01) and HTTP challenges.
 - [Bring your own dns provider](#bring-your-own-dns-provider)
 - [Bring your own http provider](#bring-your-own-http-provider)
 - sewer is both a [command-line program](#cli) and a [Python library](#usage) for customization
@@ -257,11 +257,9 @@ The certificate, certificate key and account key will be saved in the directory
 that you run sewer from.
 
 ## Bring your own DNS provider          
-NB: This section is out of date.  It describes the Legacy DNS interface.
-Newer documentation, though not a worked example like this, can be found in
-the docs directory.
+>NB: This section is out of date.  It describes the Legacy DNS interface.
+>Newer documentation can be found in[docs/dns-01](docs/dns-01.md).
 
----
 It is very easy to use any dns provider with sewer.          
 All you have to do is create your own dns class that is a child class of [`sewer.BaseDns`](https://github.com/komuw/sewer/blob/master/sewer/dns_providers/common.py) and then implement the             
 `create_dns_record` and `delete_dns_record` methods.                     

docs/ACME.md

@@ -0,0 +1,48 @@
+## ACME, RFCs, and confusion, oh my!
+
+ACME grew out of early, ad-hoc procedures designed to let CAs issue large
+numbers of certificates with low overhead.  As described in RFC855, these
+would go something like this:
+
+> * Generate a PKCS#10 [RFC2986] Certificate Signing Request (CSR).
+> * Cut and paste the CSR into a CA's web page.
+> * Prove ownership of the domain(s) in the CSR by one of the following methods:
+>     + Put a CA-provided challenge at a specific place on the web server
+>     + Put a CA-provided challenge in a DNS record corresponding to the target domain
+>     + Receive a CA-provided challenge at (hopefully) an administrator-controlled email 
+>       address corresponding to the domain, and then respond to it on the CA's web page
+> * Download the issued certificate and install it on the user's web server
+>
+> With the exception of the CSR itself and the certificates that are
+> issued, these are all completely ad hoc procedures and are
+> accomplished by getting the human user to follow interactive natural
+> language instructions from the CA rather than by machine-implemented
+> published protocols.
+
+HTTP validation was the first mechanism, matching the first method of
+proving ownership in the above.  The rest of what
+[Let's Encrypt](https://letsencrypt.org)
+added was automating the process (and rearranging it a bit, having the proof
+of control happen before the CSR, etc.).  Years later, the IETF standardized
+the ACME protocol, and there are other variants that have been (or will be)
+standardized.
+
+## RFC8555
+
+The [IETF](https://www.ietf.org/) accepted(?) and published
+[RFC8555](https://tools.ietf.org/html/rfc8555) defining the ACME protocol
+for http-01 and dns-01 validations of dns-name authorizations.  These are
+the sort of ACME authorizations that we usually think of, and which sewer
+works with.  The RFC was published in the spring of 2019, but it wasn't
+until near the end of that year that Let's Encrypt adopted the full v.2 on
+only their *staging* server.  There's some elaborate and, from what I can
+make out, often-shifting schedule for various partial transitions, but I'm
+not going to try to make sense of them.  As of the beginning of 2020, the
+only immediate effect on sewer was that one could no longer run it against
+the *staging* server.  The next big change is when that same restriction is
+rolled out on LE's *production* server late in the year.  Since sewer
+v0.8.2, which implemented the final RFC8555 protocol at least well enough to
+work with LE's server implementation, our tl;dr is just this:
+
+> If you get a failure running an older version of sewer, get v0.8.2 or
+> later.  This is a known problem: v0.8.2 is the fix.

docs/Aliasing.md

@@ -0,0 +1,91 @@
+# Aliasing for ACME Validation
+
+The idea is presented (for dns-01 authorizations) in [an article at
+letsencrypt.org](https://letsencrypt.org/2019/10/09/onboarding-your-customers-with-lets-encrypt-and-acme.html)
+which shows an example of DNS aliasing
+and describes what was likely the original motivation - a hosting provider
+running the certificate process on behalf of his customers.
+Like all DNS aliasing, it uses a CNAME at the canonical name
+`_acme-challenge.domain.tld` to redirect the ACME server to a different fqdn
+that is more convenient for provisioning of the validation responses.
+I'm not going to try to convince you that you should use aliasing, because
+if you need it you probably already know that, or at least know that the
+process isn't working smoothly as-is.
+
+The `alias` option in sewer is available to drivers that derive from
+`DNSProviderBase`.
+
+**NB: neither drivers nor the cli app have support for `alias` in 0.8.2**
+
+## Isn't aliasing just for DNS?
+
+No.  HTTP has had, as a side effect of common web server and client behavior,
+a kind of aliasing since the very beginning of ACME.  Usually it's
+convenient enough to provision the validation at the canonical
+`/.well-known/acme-challenge/<token>` location.  But if it isn't, either
+`acme-challenge` or `.well-known` can be mapped (by server configuration or
+externally by symlink, usually) to some other location.  If it's desired to
+serve the validation file from some other domain or server altogether, an
+HTTP redirect can often be used (and that's also a third way to place the
+file elsewhere within the web server's accesible filesystem).
+
+The RFC says that (for http-01 challenges) the ACME server "SHOULD follow
+redirects", which would allow for an analogous aliasing.  Lets' Encrypt's
+servers [do follow redirects and
+CNAMES](https://letsencrypt.org/docs/challenge-types/).
+
+So aliasing can be used with HTTP validations, though it's probably less
+often needed since the privilege needed to directly configure the canonical
+response file is likely to be the same (or even less) than that needed to
+setup the new certificate.  And it's possible that you've already used it
+without thinking of it as _aliasing_ because it uses such basic HTTP
+behavior.
+
+## Preparing for DNS aliasing
+
+The first thing which you must have is a way to manage DNS TXT records.  In
+fact, you need to be able to control both the real domain's records (in
+order to setup the CNAME entries, but that's something that needs be done
+only once) as well as managing the alias domain records through the
+service-specific driver.  Generally, expect to need a new-model driver
+rather than an existing legacy driver, on the assumption that it's not much
+more work to migrate the legacy driver to the new interface while adding the
+alias support.  I have my fingers crossed, at any rate...
+
+With alias-capable driver in hand, you then setup CNAME records for every
+DNS name that you wish to use with the alias domain.  In traditional zone
+file form that might look something like this [excerpt]:
+
+    ; existing record for your web or other server
+    name.example.com.                  IN  A     111.222.333.444
+
+    ; the added CNAME at the ACME-prefixed name
+    _acme-challenge.name.example.com.  IN  CNAME name.example.com.alias.org
+
+In online domain editors, the names are usually given without the full
+domain suffix that's shown here (example.com).  The A record (or it could be
+a CNAME) for `name.example.com` that directs to your server is shown as an
+example here.
+The added CNAME record is the redirect from the conventional ACME challenge
+DNS name, pointing to the TXT record in the alias domain.  When it sees that
+CNAME, the ACME server will proceed to look for the challenge's TXT record
+at `name.example.com.alias.org`.  Since the alias-aware driver will have
+setup that TXT record, the server will retrieve it and validate your right
+to issue for `name.example.com`.
+
+Note that the alias domain can be ANY valid domain that you can manage.  In
+particular, it can be in a different tld (as shown here) or a different
+domain in the same tld, or even in a sub-domain (eg. 
+`validation.example.com`) of the target's domain that has been delgated to
+that more convenient DNS server.  And you can setup aliased TXT challenge
+records for names from any number of _real_ domains as long as the CNAME
+redirects can be provisioned.
+
+## Using DNS aliases in sewer
+
+This is really pretty short & sweet.
+All that's needed, once the setup is done, is to pass `alias=alias.org` to
+the alias-supporting driver when it's created.
+For users of the command line tool, you would add an option `--alias
+alias.org` as well as specifying a DNS driver that supports the alias
+method.

docs/DNS-Propagation.md

@@ -0,0 +1,99 @@
+## Waiting for Mr. DNS or Someone Like Him
+
+Q: How long does it take after you've setup the challenge response TXT records
+until they're actually accessible to the ACME server?
+
+Good Question!
+
+According to [Let's Encrypt](https://letsencrypt.org/docs/challenge-types/#dns-01-challenge)
+it can take up to an hour.  Depends on the DNS service.  Some provide a way
+to check that your changes are fully propagated to all their servers.  With
+many, however, you just have to wait.
+
+Sewer provides a flexible _delay until actually published_ mechanism through
+three optional driver parameters, `prop_delay`, prop_timeout`,
+`prop_sleep_times`, and the [unpropagated()method](unpropagated).
+Let's see how they're used in various circumstances.
+
+### No API support, no reliable way to check: just delay
+
+If you can't check that the TXT records are fully published, then all you
+can do is delay for a while.  Perhaps the DNS service will suggest a safe
+time.  If not, you'll have to start with a guess and adjust from there based
+on your experience.  Choosing the right number - long enough but not
+excessively long - can be hard, but applying it is easy.  Just add
+`prop_delay=SIMPLE_DELAY_TIME` to the driver's initialization parameters,
+and sewer's engine will add that many seconds of delay after the challenge
+setup returns before it signals the ACME server to validate those
+challenges.
+_[how does this work in the CLI?  TBD!]_
+
+### API support or can check: use a timeout
+
+If the DNS service gives you a way to check that the propagation is
+complete, or if there are not too many authoritative servers (viz., not an
+anycast system), you can use that actual check (implemented in the driver's
+`unpropagated` method) and the engine will run that check until it succeeds
+or until a timeout you specify is exceeded.  However the check is being
+done, you setup the timeout by adding `prop_timeout=MAX_WAIT_TIME` to the
+driver parameters.  If you know that it takes at least some minimum time to
+propagate, you may also pass `prop_delay` to make the engine delay that long
+before it starts checking.  And there's a delay between checks that has a
+hopefully sensible default, but which you can adjust if necessary through
+the `prop_sleep_times` parameter.
+
+### You probably don't need to change `prop_sleep_times`
+
+Unless you do, but if it's not obvious, just leave it.
+
+This parameter defines the lengths of sleeps the engine will add following a
+call to `unpropagated` that reports not ready.  As an optional parameter
+passed to the driver, `prop_sleep_times` can be an integer number of seconds
+or a list or tuple of such delays which will be used in order.  The final
+value in the sequence will be reused indefinitely.
+
+Example: the default value is (1, 2, 4, 8) which provides an exponential
+backoff up to an 8 second delay, then sticks there.  _[the values could
+change - it's just what seemed reasonable to me]_  So if there's no delay,
+and the check call takes no measurable time (and reports not ready each
+time), it will look something like this with `prop_timeout=20`:
+
+| time | action |
+| ---: | --- |
+| 0 | call unpropagated, sleep(1) |
+| 1 | call unproagagted, sleep(2) |
+| 2 | call unpropagated, sleep(4) |
+| 6 | call unpropagated, sleep(8) |
+| 14 | call unpropagated, sleep(8) |
+| 22 | call unpropagated, timeout! |
+
+This shows both the last value repeating and the way the timeout and sleeps
+interact.  The check for timeout is done only AFTER a call to unpropagated
+AND the chance to exit with success if it finally reports the challenges are
+ready.  So the timeout isn't a hard maximum time, but it's bounded to be no
+more than one sleep interval (plus actual time to run `unpropagated`, of
+course) over `prop_timeout`.
+
+### Other Notes and Advanced Use
+
+These values are setup through the Provider on the reasonable assumption
+that they will vary most directly with the choice of service provider, so
+the individual drivers are best suited to provide sensible defaults. 
+Sewer's engine implements the delay and check loop (with timeout) because
+the mechanism is the same for all providers (and may be useful for other
+than the DNS-based challenges for which it has been implemented).
+
+If you are using sewer as a library and find that you can make a better
+estimate of the propagation after the driver is setup (perhaps using a
+service-specific method to access part of the service's API or run some
+tests), you could adjust those parameters through the same-named attributes
+on the Provider instance.  This is solidly in the categories of don't do it
+unless you're sure you need to, and be prepared to own both the pieces if
+you break it!
+
+### Could this be used with non-DNS Providers?
+
+Yes!  I have no experience with http-01 in any setting where such a delay
+might be needed, but the mechanism is implemented in sewer's engine, and all
+that needs be done is to setup the parameters (and unpropagated if using
+more than just `prop_delay`) as described above and there you go!

docs/LegacyDNS.md

@@ -2,8 +2,8 @@
 
 ### `BaseDns` shim class
 
-A child of `ProviderBase` that acts as an adapter between the Provider
-interface and the Legacy DNS providers.
+A child of `DNSProviderBase` that acts as an adapter between the Provider
+interface and the Legacy DNS provider interface.
 
 #### `__init__(self, **kwargs: Any) -> None`
 
@@ -12,28 +12,28 @@ Injects chal_types=["dns-01"].
 
 #### `setup(self, challenges: Sequence[Dict[str, str]]) -> Sequence[Dict[str, str]]`
 
-Iterates over the challenges, extracting the values needed for dns-01 from
-each challenge in the list, and passing them to create_dns_record.
-Always returns an empty list since there is no error return from
-create_dns_record other than raising an exception.
+Iterates over the challenges, extracting the values needed for the Legacy
+DNS interface from each challenge in the list, and passing them to
+`create_dns_record`.  Always returns an empty list since there is no error
+return from `create_dns_record` other than raising an exception.
 
 #### `unpropagated(self, challenges: Sequence[Dict[str, str]]) -> Sequence[Dict[str, str]]`
 
 Always returns an empty list, signalling "all ready as far as I know".
-A DNS provider wishing to do something useful here must migrate to the new
-API.
+A Legacy DNS provider wishing to do something useful here MAY implement
+`unpropagated` without migrating the rest of its interface.
 
 #### `clear(self, challenges: Sequence[Dict[str, str]]) -> Sequence[Dict[str, str]]`
 
-Same as setup except it calls the legacy delete_dns_record, of course.
+Same as setup except it calls the legacy `delete_dns_record`, of course.
 
 ### Legacy DNS class
 
 #### `__init__(self, ...)`
 
 Args are explicitly named per provider; no provision for passing any to
 `super().__init__` - which makes sense, since there used not to be any the
-parent was prepared to receive.
+parent (original `BaseDns`) was prepared to receive.
 
 #### `def create_dns_record(self, domain_name, domain_dns_value)`
 
@@ -45,4 +45,11 @@ All very provider-dependent.
 
 In theory it should undo the effects of setup.
 In practice, at least one of the services is unable to do that
-(according to the comment).
+(according to the author's comment).
+
+### Legacy DNS vs Aliasing
+
+Legacy DNS providers MAY adapt to using the [aliasing](DNS-ALiasing), though
+a potentially fragile faking of the new-model challenge dict is required. 
+See the `unbound_ssh` example driver, and bear in mind that a change to the
+data type of the challenge items IS anticipated, perhaps in 0.9.