Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proxy Nexus's external API out the tech port via wicketd #4224

Merged
merged 11 commits into from
Oct 18, 2023

Conversation

jgallagher
Copy link
Contributor

This implements one of the proposed solutions in RFD 431 (which I still need to get back to and refine a bit!):

  • Nexus gains another dropshot instance that serves the external API (using the same shared server context as the existing external API servers) on the underlay network
  • wicketd runs a TCP proxy on the tech port
  • When wicketd receives a new connection on that proxy socket, it:
    • looks up Nexus in internal DNS (these records already exist for the internal API, but that's good enough for us: we're just after the IP address here)
    • connects to any Nexus instance reported by the DNS query
    • proxies the connection between the tech port client and the Nexus instance

This is therefore exposing the "true" Nexus API, which still requires TLS, client auth, etc.

Testing this PR

This is currently deployed on madrid, and the racklette is set up using dogfood's certs, and I manually tested (and demo'd) it there.

I'd be happy to add integration tests, but am not sure what would make sense. I could add a test where I spin up an internal DNS server, seed it with a fake Nexus entry, and ensure that wicketd will proxy connections to that fake Nexus, maybe? Is it worth trying to add tests that Nexus exposes the same API on both the external and "techport" interfaces?

@morlandi7 morlandi7 added this to the 3 milestone Oct 9, 2023
nexus/src/lib.rs Outdated Show resolved Hide resolved
wicketd/src/http_entrypoints.rs Outdated Show resolved Hide resolved
Comment on lines +23 to +25
* `config/address` is not changing
* the dropshot server listening at `config/address` is not configured
for TLS
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could validate these assumptions in code, within the body of /reload-config? It might be worthwhile to fail-hard in these cases if the error would otherwise be confusing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it's super easy to validate http vs https from within the endpoint, but that's maybe okay: the curl invocation itself will fail hard. I added a check that config/address is not changing in 53e0654

wicketd/src/lib.rs Outdated Show resolved Hide resolved
@jgallagher jgallagher merged commit 5346472 into main Oct 18, 2023
23 checks passed
@jgallagher jgallagher deleted the wicketd-proxy-nexus branch October 18, 2023 16:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants