Finalize sprockets integration

[jgallagher]

As of #1128, bootstrap-agent to bootstrap-agent communications may (if an SP is present) take place in sprockets sessions, providing authentication and confidentiality. However, the current mechanism by which sprockets is integrated is as a TCP proxy sitting between the bootstrap-agent dropshot server and its client (see #1128 for more details). The dropshot server itself listens on localhost and implicitly trusts all incoming connections, expecting them to only be coming from the sprockets proxy. This needs to be locked down to guarantee requests to the dropshot server can only take place if the client has connected via sprockets. (A similar problem exists on the client side, where the sprockets proxy implicitly trusts all connections made to it before forwarding them to the corresponding serverside sprockets proxy.) Some possible options:

• Extend dropshot itself (and progenitor) allowing custom listeners/acceptors sufficiently for us to use hyper-sprockets, similarly to how dropshot/progenitor have built-in support for TLS.
• Switch from using TCP for the localhost connections to Unix domain sockets?
• Sufficiently lock down the TCP services (this point is vague because it's unclear to me what could be done here)?
• Move away from dropshot/progenitor entirely, and use something bespoke over "raw TCP" (inside a sprockets session)

My preference is the first, but it requires a currently-unknown amount of work; probably more on the progenitor side than dropshot due to progenitor's use of reqwest, which currently does not support custom clients.

CC @smklein @andrewjstone @ahl

[jgallagher]

After chatting with @andrewjstone about a different but related issue (described below), we're inclined to go with option 4 and use sprockets session directly as a wrapped TCP stream, eschewing dropshot and progenitor for bootstrap-agent-to-bootstrap-agent communications.

As described in RFD 238 § 4, during rack unlock, sleds must not only mutually authenticate via sprockets, but must additionally verify that the peer they're communicating with is in the set of DeviceIds provisioned during rack initialization. The remote identity is trivially available given direct access to a sprockets session, but it's less obvious how we would perform that check if we kept the proxies or used hyper-sprockets as the connection layer for dropshot/progenitor. In both cases we could push the device ID check into the connection layer itself (either the proxy or hyper-sprockets), but that would leave dropshot and its client in the awkward spot of appearing to implicitly trust all connections, as they'd be relying on the connection layer to (transparently to them) perform all auth checks. If we wanted to pursue the hyper-sprockets / integration path, it would be nicer to be able to expose at least evidence that the auth check had been performed to the dropshot endpoint and client, but that would require more work of unknown size, particularly since it's not obvious hyper supports that kind of out-of-band info.

[jgallagher]

Option 4 implented in #1173